$Version
12.1.0 for Microsoft Windows (64-bit) (March 14, 2020)
trainingset = ExampleData[{"MachineLearning", "Mushroom"}, "TrainingData"];
testset = ExampleData[{"MachineLearning", "Mushroom"}, "TestData"];
samples = testset[[All, 1]];
trainingData = RandomSample[trainingset, 1000];
c = Classify[trainingData, Method -> "DecisionTree", FeatureExtractor -> "DimensionReducedVector"]
c[samples[[79]], "Probabilities"]
<|"edible" -> 0.975, "poisonous" -> 0.025|>
c[samples[[{1, 2, 79}]], "Probabilities"][[-1]]
<|"edible" -> 0.975, "poisonous" -> 0.025|>
No difference.
c = Classify[trainingData, Method -> "GradientBoostedTrees", FeatureExtractor -> "DimensionReducedVector"]
c[samples[[79]], "Probabilities"]
<|"edible" -> 0.998383, "poisonous" -> 0.00161739|>
c[samples[[{1, 2, 79}]], "Probabilities"][[-1]]
<|"edible" -> 0.99807, "poisonous" -> 0.00193042|>
0.998383 vs 0.99807
c = Classify[trainingData, Method -> "RandomForest", FeatureExtractor -> "DimensionReducedVector"]
c[samples[[79]], "Probabilities"]
<|"edible" -> 0.855689, "poisonous" -> 0.144311|>
c[samples[[{1, 2, 79}]], "Probabilities"][[-1]]
<|"edible" -> 0.875449, "poisonous" -> 0.124551|>
0.855689 vs 0.875449 - significant difference
The original version of the question
Tested at 12.1, the problem seems to be existed a long time...
trainingset = ExampleData[{"MachineLearning", "Mushroom"}, "TrainingData"];
testset = ExampleData[{"MachineLearning", "Mushroom"}, "TestData"];
samples = testset[[All, 1]];
trainingData = RandomSample[trainingset, 1000];
c = Classify[trainingData, Method -> "DecisionTree"]
also happen in GradientBoostedTrees method, you can try several times to re-train the model to make this happen.
Since the batch predict is extremely important in speed aspect, I think this problem is very serious...
Batch Predict means use a trained model to predict several examples like this way: model[{example1,example2,}], something like listable attributes in some function, which is much faster
c, to evaluate a single example , e.g.,c[ samples[[79]], "Probabilities" ], you get a different answer than if you use the trained classifier to evaluate a list of examples, e.g.,c[samples[[{1, 2, 79}]], "Probabilities"]. In fact, the problem is even worse than this: If you evaluate a list of examples of the same point one gets different answers, e.g.,c[ {samples[[79]], samples[[79]] }, "Probabilities"]returns the two different probability values, even though it is a list of the same examples. $\endgroup$