7
$\begingroup$

There is a mysterious "LogisticRegression" method in MMA, which could be applied as a substitution for missing multinomial logit functionality.

In version 11.3 I had the following simple 3-category classifier, which returns reasonable logit probabilities:

data = {{1, a}, {2, a}, {3, b}, {4, a}, {5, b}, {6, c}, {7, c}, {8, 
    b}, {9, c}};
cls = Classify[Rule @@@ data, Method -> "LogisticRegression"];
Quiet @ Plot[(Evaluate@
    Table[cls[x, {"Probability", i}], {i, {a, b, c}}]), {x, 0, 10}, 
  PlotLegends -> {"P(y=a | x)", "P(y=b | x)", "P(y=c | x)"}]

enter image description here

The same code produces weird results in version 12 (Windows 10, 64-bit).

enter image description here

As one may see, it does classify somehow, though with PlotRange -> {0,1} this guy looks as follows:

enter image description here

Question. Can someone reproduce this behavior? If yes, is it a bug?

The relevant question is also 163662

$\endgroup$
3
$\begingroup$

I don't think it is a bug. I executed your Code in M 11.1.1. Using ClassifierInformation[cls] you get some Information About the Classifier.

enter image description here

Now I use M 12.0.0 and pass the regularization coefficients explicitly to the Classifier.

data = {{1, a}, {2, a}, {3, b}, {4, a}, {5, b}, {6, c}, {7, c}, {8, b}, {9, c}}; 
cls = Classify[Rule @@@ data, Method -> {"LogisticRegression", "L1Regularization" ->0., "L2Regularization" -> 1.}]; 
Quiet@Plot[(Evaluate@Table[cls[x, {"Probability", i}], {i, {a, b, c}}]), {x, 0, 10},PlotLegends -> {"P(y=a | x)", "P(y=b | x)", "P(y=c | x)"}]

The result is exactely the same as in M 11. enter image description here

$\endgroup$
4
  • $\begingroup$ thank, you are right, it looks like a feature. $\endgroup$ – garej Jan 9 '20 at 12:03
  • $\begingroup$ @garej If you want to reproduce M12 behaviour in M11 you have to set L2Regularization 1000. Unfortunately I have no insight about the underlying logic regarding the automatic determination of regularization coefficients. $\endgroup$ – RMMA Jan 9 '20 at 12:18
  • 5
    $\begingroup$ If you just use Classify[Rule @@@ data, Method -> {"LogisticRegression"}, TimeGoal -> 10], V12 picks more sensible regularization value of 10. It seems like you need to give it some time to find a good value. $\endgroup$ – Sjoerd Smit Jan 9 '20 at 12:35
  • $\begingroup$ @SjoerdSmit, nice point, thank you. $\endgroup$ – garej Jan 9 '20 at 12:40

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.