# Logistic Regression classifier in Versions 12 vs 11.3

There is a mysterious "LogisticRegression" method in MMA, which could be applied as a substitution for missing multinomial logit functionality.

In version 11.3 I had the following simple 3-category classifier, which returns reasonable logit probabilities:

data = {{1, a}, {2, a}, {3, b}, {4, a}, {5, b}, {6, c}, {7, c}, {8,
b}, {9, c}};
cls = Classify[Rule @@@ data, Method -> "LogisticRegression"];
Quiet @ Plot[(Evaluate@
Table[cls[x, {"Probability", i}], {i, {a, b, c}}]), {x, 0, 10},
PlotLegends -> {"P(y=a | x)", "P(y=b | x)", "P(y=c | x)"}]


The same code produces weird results in version 12 (Windows 10, 64-bit).

As one may see, it does classify somehow, though with PlotRange -> {0,1} this guy looks as follows:

Question. Can someone reproduce this behavior? If yes, is it a bug?

The relevant question is also 163662

I don't think it is a bug. I executed your Code in M 11.1.1. Using ClassifierInformation[cls] you get some Information About the Classifier.

Now I use M 12.0.0 and pass the regularization coefficients explicitly to the Classifier.

data = {{1, a}, {2, a}, {3, b}, {4, a}, {5, b}, {6, c}, {7, c}, {8, b}, {9, c}};
cls = Classify[Rule @@@ data, Method -> {"LogisticRegression", "L1Regularization" ->0., "L2Regularization" -> 1.}];
Quiet@Plot[(Evaluate@Table[cls[x, {"Probability", i}], {i, {a, b, c}}]), {x, 0, 10},PlotLegends -> {"P(y=a | x)", "P(y=b | x)", "P(y=c | x)"}]


The result is exactely the same as in M 11.

• thank, you are right, it looks like a feature. – garej Jan 9 '20 at 12:03
• @garej If you want to reproduce M12 behaviour in M11 you have to set L2Regularization 1000. Unfortunately I have no insight about the underlying logic regarding the automatic determination of regularization coefficients. – RMMA Jan 9 '20 at 12:18
• If you just use Classify[Rule @@@ data, Method -> {"LogisticRegression"}, TimeGoal -> 10], V12 picks more sensible regularization value of 10. It seems like you need to give it some time to find a good value. – Sjoerd Smit Jan 9 '20 at 12:35
• @SjoerdSmit, nice point, thank you. – garej Jan 9 '20 at 12:40