# Classifying text and number features

I am new to the ML algorithms and I am checking the Classify function of Mathematica. I understand the math for the random forest algorithm, but I have a question about the specific implementation by Mathematica.

I use both text and numeric features with Classify.

tr =
{{"sdf", 2} -> "A", {"sdaf", 3} -> "B", {"sddf", 4} -> "A", {"sdfas", 5} -> "B"};

c2 = Classify[tr, Method -> "RandomForest"];


and I get as outputs several results

c2[{"abcwer3425", 3}]


A

How does Mathematica use the text patterns on random forest?

I think in the case of your data all unique strings of the first feature, which are say of $n$ count, are converted into integers of the range $[1,n]$ and then the random trees are build on training data of $n+1$ features. (The second feature is being appended.)

Here is an indirect proof (which I used to come up with the conjecture above):

tr = {{"sdf", 2} -> "A", {"sdaf", 3} -> "B", {"sddf", 4} ->
"A", {"sdfas", 5} -> "B", {"sdaaf", 4} -> "B", {"sdf", 7} -> "A"};
c3 = Classify[Take[tr, 3], Method -> "RandomForest"];
c4 = Classify[Take[tr, 4], Method -> "RandomForest"];
c5 = Classify[Take[tr, 5], Method -> "RandomForest"];
c6 = Classify[Take[tr, 6], Method -> "RandomForest"];

Table[
{Cases[cl[[1]]["Models"][[1]]["Processor"],
MachineLearningSortedHashAssociation[x_], \[Infinity]],
cl[[1]]["Models"][[1]]["FeatureNumber"]}, {cl, {c4, c5, c6}}]

(* {{{"SortedHashAssociation"["KeyNumber" -> 4,
"CollisionNumber" -> 0, "DefaultValue" -> -1]}, 5},

{{"SortedHashAssociation"["KeyNumber" -> 5,
"CollisionNumber" -> 0, "DefaultValue" -> -1]}, 6},

{{"SortedHashAssociation"["KeyNumber" -> 5,
"CollisionNumber" -> 0, "DefaultValue" -> -1]}, 6}} *)


You can further venture looking into c2[[1]] ...

From what I have seen in the Classify output internals, it is very likely that Classify` uses some variant of Feature hashing.

• Thank you for your answer. That makes sense!! Commented Jun 5, 2016 at 22:05
• @dimos No problem, good luck! Commented Jun 5, 2016 at 22:40