Extract details from PredictorFunction object

Question

For training data, the command Predict generates an object PredictorFunction, which can evaluate a prediction from variables. For instance, according to the help

d = Dataset[{<|"age" -> 32, "height" -> 160, "gender" -> "female"|>,
 <|"height" -> 183, "age" -> 41, "gender" -> "female"|>,
 <|"height" -> 123, "age" -> 30, "gender" -> "female"|>,
 <|"height" -> 175, "age" -> 21, "gender" -> "male"|>,
 <|"height" -> 150, "age" -> 11, "gender" -> "male"|>,
 <|"age" -> 52, "height" -> 164, "gender" -> "female"|>}]; 

 p = Predict[d -> "age", Method -> "DecisionTree"]

generates a prediction function p. The result may be simply tree. However, I cannot identify within p neither nodes nor branching rules. For instance, the command Normal@p contains technical information, but not the exact prediction formula.

The same problem I have with the choice Method -> "LinearRegression"], I cannot identify the coefficients from the PredictorFunction

Answer 1

I am not sure if this is documented or subject to change in the future, but you might be able to access the underlying data with the following code (using version 12.2.0.0 on Windows10)

p[[1]]

There, I get These keys:

In[77]:= p[[1]] // Keys

Out[77]= {"ExampleNumber", "Input", "Output", "LabelSplitter", \
"Prior", "Utility", "Threshold", "PerformanceGoal", \
"BatchProcessing", "Model", "TrainingInformation", "AnomalyDetector", \
"Log"}

Looking through the "Input" Association, you should be able to extract some details about the input preprocessing strategy.

Additionally, I see the model data can be viewed with:

tree = p[[1, "Model", "Tree"]];
tree[[1]] (* Print out the full Association *)

And data can be extracted as follows:

featureInddices = tree[[1, "FeatureIndices"]] // Normal
thresholds = tree[[1, "NumericalThresholds"]] // Normal
children = tree[[1, "Children"]] // Normal
leafValues = tree[[1, "LeafValues"]] // Normal

Out[88]= {2}

Out[89]= {0.160845}

Out[90]= {{-1, -2}}

Out[91]= {{-0.796448, 0.714051}, {0.796448, 0.733919}}

I have also been curious about this, if there are any convienent ways to store/export/analyze model and preprocessing parameters generated by the Classify and Predict functions.

I will ask Wolfram support and update my answer if they give me any more useful data. Good luck with your work!

Update: this is the response I got from support: "We don't have documentation for MachineLearning`DecisionTree, and there is no other way to implement it."

It seems there is no easy way to extract the algorithm using p[[1, "Model", "Tree"]] other than manually re-creating it. Hopefully in a future release this will be possible...