Classifying text and number features

Question

I am new to the ML algorithms and I am checking the Classify function of Mathematica. I understand the math for the random forest algorithm, but I have a question about the specific implementation by Mathematica.

I use both text and numeric features with Classify.

tr = 
  {{"sdf", 2} -> "A", {"sdaf", 3} -> "B", {"sddf", 4} -> "A", {"sdfas", 5} -> "B"};

c2 = Classify[tr, Method -> "RandomForest"];

and I get as outputs several results

c2[{"abcwer3425", 3}]

A

How does Mathematica use the text patterns on random forest?

Answer 1

I think in the case of your data all unique strings of the first feature, which are say of $n$ count, are converted into integers of the range $[1,n]$ and then the random trees are build on training data of $n+1$ features. (The second feature is being appended.)

Here is an indirect proof (which I used to come up with the conjecture above):

tr = {{"sdf", 2} -> "A", {"sdaf", 3} -> "B", {"sddf", 4} -> 
    "A", {"sdfas", 5} -> "B", {"sdaaf", 4} -> "B", {"sdf", 7} -> "A"};
c3 = Classify[Take[tr, 3], Method -> "RandomForest"];
c4 = Classify[Take[tr, 4], Method -> "RandomForest"];
c5 = Classify[Take[tr, 5], Method -> "RandomForest"];
c6 = Classify[Take[tr, 6], Method -> "RandomForest"];

Table[
 {Cases[cl[[1]]["Models"][[1]]["Processor"], 
   MachineLearning`SortedHashAssociation[x_], \[Infinity]],
  cl[[1]]["Models"][[1]]["FeatureNumber"]}, {cl, {c4, c5, c6}}]

(* {{{"SortedHashAssociation"["KeyNumber" -> 4, 
    "CollisionNumber" -> 0, "DefaultValue" -> -1]}, 5},    

    {{"SortedHashAssociation"["KeyNumber" -> 5, 
     "CollisionNumber" -> 0, "DefaultValue" -> -1]}, 6},           

    {{"SortedHashAssociation"["KeyNumber" -> 5, 
     "CollisionNumber" -> 0, "DefaultValue" -> -1]}, 6}} *)

You can further venture looking into c2[[1]] ...

From what I have seen in the Classify output internals, it is very likely that Classify uses some variant of Feature hashing.