# Classify / Predict: ensemble of classifiers / predictors

Mathematica has undocumented functionality of combination classifiers / predictors in ensemble. How to get access to this functionality?

SeedRandom[1];
X = Table[
Join[
ToString /@ RandomReal[{-1, 1}, 1],
RandomReal[{-1, 1}, 4],
RandomSample[{"A", "B", "C"}, 2]
],
{1000}
];
Y = RandomInteger[{0, 1}, 1000];

c = Classify[X -> Y]


ClassifierInformation[c]


ClassifierMeasurements[c, X -> Y, "ConfusionMatrixPlot"]


• How is this not a duplicate of the question I posted? Dec 11 '16 at 18:13

c1 = Classify[X -> Y, Method -> "LogisticRegression"]
c2 = Classify[X -> Y, Method -> "NearestNeighbors"]

c = MachineLearningPackageScopeCombinePredictors[{c1, c2}]

ClassifierInformation[c]


p1 = Predict[X -> Y, Method -> "LinearRegression"]
p2 = Predict[X -> Y, Method -> "NearestNeighbors"]

p = MachineLearningPackageScopeCombinePredictors[{p1, p2}]

PredictorInformation[p]


• Great (+1) -- do you think you can find how to do a weighted combination of the classifiers? Jan 12 '17 at 15:05
• @AntonAntonov I am not sure. Right now Combiner can be only 'Geometric". GeneralUtilitiesPrintDefinitions@MachineLearningPackageScopeCombinePredictors Jan 12 '17 at 15:11
• What about 11.3? It doesn't seem to work anymore. Mar 22 '18 at 20:37
• @swish Yes, it stopped work in V11.2. Function was renamed to Combiner. But it doesn't work the old way. During exploration I found that Classify has the new method Combination. But I don't know what is wrong: Classify[X -> Y, Method -> {"Combination", "Methods" -> {"LogisticRegression", "NearestNeighbors"}, "CombinerMethod" -> "Geometric"}, "DirectTraining" -> True] See GeneralUtilitiesPrintDefinitions@MachineLearningPackageScopeTrainPredictor Mar 23 '18 at 4:12

The question formulation asks for access to the internals of Classify in order to get the ensembles, but there is way to make ensembles of classifiers (i.e. ClassifierFunction[___] functions) through the argument "Probabilities" -- see the (short) package ClassifierEnsembles.m.

Very detailed explanations for using and evaluating classifier ensembles made with that package are given in:

Here is an image of ROC curves for comparing the performance of a classifier ensemble with its individual classifiers:

• Very nice, that. Jan 12 '17 at 0:08
• @DanielLichtblau Thank you, good to hear. Jan 12 '17 at 3:36
• Anton, good answer! +1 here and in the Wolfram Community. But I can not accept your answer because, as you have noted, my question is about the internals of Classify. With Spelunking I have read Classify code but didn't understand how to call Combiner-function directly. Jan 12 '17 at 6:27
• @AlexeyGolyshev Thank you for your note! I understand your perspective. I posted this answer because with that package I am using the internals of Classify indirectly through "Probabilities". Jan 12 '17 at 11:26
• I have found solution! Jan 12 '17 at 15:03