7
$\begingroup$

There is a mysterious "LogisticRegression" method in MMA, which could be applied as a substitution for missing multinomial logit functionality.

In version 11.3 I had the following simple 3-category classifier, which returns reasonable logit probabilities:

data = {{1, a}, {2, a}, {3, b}, {4, a}, {5, b}, {6, c}, {7, c}, {8, 
    b}, {9, c}};
cls = Classify[Rule @@@ data, Method -> "LogisticRegression"];
Quiet @ Plot[(Evaluate@
    Table[cls[x, {"Probability", i}], {i, {a, b, c}}]), {x, 0, 10}, 
  PlotLegends -> {"P(y=a | x)", "P(y=b | x)", "P(y=c | x)"}]

enter image description here

The same code produces weird results in version 12 (Windows 10, 64-bit).

enter image description here

As one may see, it does classify somehow, though with PlotRange -> {0,1} this guy looks as follows:

enter image description here

Question. Can someone reproduce this behavior? If yes, is it a bug?

The relevant question is also 163662

Improve this question
$\endgroup$

1 Answer 1

Reset to default
3
$\begingroup$

I don't think it is a bug. I executed your Code in M 11.1.1. Using ClassifierInformation[cls] you get some Information About the Classifier.

enter image description here

Now I use M 12.0.0 and pass the regularization coefficients explicitly to the Classifier.

data = {{1, a}, {2, a}, {3, b}, {4, a}, {5, b}, {6, c}, {7, c}, {8, b}, {9, c}}; 
cls = Classify[Rule @@@ data, Method -> {"LogisticRegression", "L1Regularization" ->0., "L2Regularization" -> 1.}]; 
Quiet@Plot[(Evaluate@Table[cls[x, {"Probability", i}], {i, {a, b, c}}]), {x, 0, 10},PlotLegends -> {"P(y=a | x)", "P(y=b | x)", "P(y=c | x)"}]

The result is exactely the same as in M 11. enter image description here

Improve this answer
$\endgroup$
4
  • $\begingroup$ thank, you are right, it looks like a feature. $\endgroup$
    – garej
    Commented Jan 9, 2020 at 12:03
  • $\begingroup$ @garej If you want to reproduce M12 behaviour in M11 you have to set L2Regularization 1000. Unfortunately I have no insight about the underlying logic regarding the automatic determination of regularization coefficients. $\endgroup$
    – RMMA
    Commented Jan 9, 2020 at 12:18
  • 5
    $\begingroup$ If you just use Classify[Rule @@@ data, Method -> {"LogisticRegression"}, TimeGoal -> 10], V12 picks more sensible regularization value of 10. It seems like you need to give it some time to find a good value. $\endgroup$
    – Sjoerd Smit
    Commented Jan 9, 2020 at 12:35
  • $\begingroup$ @SjoerdSmit, nice point, thank you. $\endgroup$
    – garej
    Commented Jan 9, 2020 at 12:40

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.