Classify vs LogitModelFit

Question

Can someone explain me what is the difference between Classify[trainingset, Method -> "LogisticRegression"] function and LogitModelFit[trainingset, x, x] function in Wolfram Mathematica.

For example:

trainingset = {1 -> "A", 2 -> "A", 3.5 -> "B", 4 -> "B"};
c = Classify[trainingset, Method -> "LogisticRegression"];

c[2.6, "Probabilities"]
<|"A" -> 0.979033, "B" -> 0.0209672|>

But:

trainingset1  = {{1, 0}, {2, 0}, {3.5, 1}, {4, 1}};
c1 = LogitModelFit[trainingset1  , x, x] // Normal;

c1 /. {x -> 2.6}
0.0178003

Obviously, probabilities are not exactly the same. I also tried with ProbitModelFit, but again, probabilities are different.

Why is this the case?

Answer 1

This is an extended comment as I don't see that the documentation gives enough information to understand the differences.

The probability that you're getting from the LogitModelFit is the predicted probability of being from class "B" (the class you've assigned the value 1). To look at all probabilities in a specified range to see the differences in the two methods consider the following:

Plot[{c[x, {"Probability", "B"}], c1}, {x, 2.5, 3}, 
 PlotLegends -> {"Classify", "LogitModelFit"},
 Frame -> True, FrameLabel -> {"X", "Probability of B"}]

The fits are "similar" but certainly not the same (and certainly large enough to have nothing to do with round-off error).

However, if a threshold of probabilities less than 0.5 are considered "A" and greater than 0.5 are considered "B", then both approaches give exactly the same predictions. So they match perfectly on class predictions but not on the associated probabilities.