I just prepared an (well known) example in machine learning, the weather dependent determination of whether to play golf or not, as shown in this picture (of a corresponding Dataset)
Now I set up this data to be classified as follows (one can copy and paste the data for easy verifying my example):
data =
{
{"outlook", "temperature", "humidity", "wind", "play"},
{"sunny", "hot", "high", "false", "no"},
{"sunny", "hot", "high", "true", "no"},
{"overcast", "hot", "high", "false", "yes"},
{"rainy", "mild", "high", "false", "yes"},
{"rainy", "cold", "normal", "false", "yes"},
{"rainy", "cold", "normal", "true", "no"},
{"overcast", "cold", "normal", "true", "yes"},
{"sunny", "mild", "high", "false", "no"},
{"sunny", "cold", "normal", "false", "yes"},
{"rainy", "mild", "normal", "false", "yes"},
{"sunny", "mild", "normal", "true", "yes"},
{"overcast", "mild", "high", "true", "yes"},
{"overcast", "hot", "normal", "false", "yes"},
{"rainy", "mild", "high", "true", "no"}
};
Now drop the header:
data = Rest@data;
Get the target
target = Last /@ data;
get the training data
training = Most /@ data
Now do the classification 100 times with automatic choosing of Method:
tab = Table[Classify[training -> target], {100}]
then....
tab = ClassifierInformation[#, Method] & /@ tab; Tally @ tab
And I get the following result (which I do not understand, because I thought the choosing of the Method in Classify is deterministic):
Can anyone give me a hint or explanation?
Automatic
actually tries multiple classifiers and chooses the best one, according to some criteria. And since some learners like random forests and neural networks aren't deterministic, that could lead to different results. I would be curious if anyone knows more about howAutomatic
works, though. $\endgroup$Classify
andPredict
will randomly choose a method from a weighted list that is determined by your data. So, if your data changes in some significant way (size, type, etc.), the possible methods change. The only way to get a consistent result is to useSeedRandom
which is true of any random process, or to specify the method yourself. $\endgroup$SeedRandom
" - does this extend to the"RandomForest"
method? e.g. {#, SameQ @@ Table[SeedRandom@10; Predict[ExampleData[{"MachineLearning", "BostonHomes"}, "Data"], Method -> #] // Query[1, "Models", 1, 2] // Hash, 2]} & /@ {"LinearRegression", "NearestNeighbors", "NeuralNetwork", "RandomForest", "GaussianProcess"} $\endgroup$