4
$\begingroup$

I am new to the ML algorithms and I am checking the Classify function of Mathematica. I understand the math for the random forest algorithm, but I have a question about the specific implementation by Mathematica.

I use both text and numeric features with Classify.

tr = 
  {{"sdf", 2} -> "A", {"sdaf", 3} -> "B", {"sddf", 4} -> "A", {"sdfas", 5} -> "B"};

c2 = Classify[tr, Method -> "RandomForest"];

and I get as outputs several results

c2[{"abcwer3425", 3}]

A

How does Mathematica use the text patterns on random forest?

$\endgroup$
4
$\begingroup$

I think in the case of your data all unique strings of the first feature, which are say of $n$ count, are converted into integers of the range $[1,n]$ and then the random trees are build on training data of $n+1$ features. (The second feature is being appended.)

Here is an indirect proof (which I used to come up with the conjecture above):

tr = {{"sdf", 2} -> "A", {"sdaf", 3} -> "B", {"sddf", 4} -> 
    "A", {"sdfas", 5} -> "B", {"sdaaf", 4} -> "B", {"sdf", 7} -> "A"};
c3 = Classify[Take[tr, 3], Method -> "RandomForest"];
c4 = Classify[Take[tr, 4], Method -> "RandomForest"];
c5 = Classify[Take[tr, 5], Method -> "RandomForest"];
c6 = Classify[Take[tr, 6], Method -> "RandomForest"];

Table[
 {Cases[cl[[1]]["Models"][[1]]["Processor"], 
   MachineLearning`SortedHashAssociation[x_], \[Infinity]],
  cl[[1]]["Models"][[1]]["FeatureNumber"]}, {cl, {c4, c5, c6}}]

(* {{{"SortedHashAssociation"["KeyNumber" -> 4, 
    "CollisionNumber" -> 0, "DefaultValue" -> -1]}, 5},    

    {{"SortedHashAssociation"["KeyNumber" -> 5, 
     "CollisionNumber" -> 0, "DefaultValue" -> -1]}, 6},           

    {{"SortedHashAssociation"["KeyNumber" -> 5, 
     "CollisionNumber" -> 0, "DefaultValue" -> -1]}, 6}} *)

You can further venture looking into c2[[1]] ...

From what I have seen in the Classify output internals, it is very likely that Classify uses some variant of Feature hashing.

$\endgroup$
  • $\begingroup$ Thank you for your answer. That makes sense!! $\endgroup$ – dimos Jun 5 '16 at 22:05
  • $\begingroup$ @dimos No problem, good luck! $\endgroup$ – Anton Antonov Jun 5 '16 at 22:40

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.