Strange result for decision-tree-classifier. Why?

Question

I'm still in touch with DT-Classification and try to understand how the Mathematica algorithms works and what tree is generated (related: click). I just tested the well-known "playing tennis" (sometimes "playing golf") example. The data is as follows:

 data = {{"outlook", "temperature", "humidity", "wind", 
    "play"}, {"sunny", "hot", "high", "weak", "no"}, {"sunny", "hot", 
    "high", "strong", "no"}, {"overcast", "hot", "high", "weak", 
    "yes"}, {"rainy", "mild", "high", "weak", "yes"}, {"rainy", 
    "cold", "normal", "weak", "yes"}, {"rainy", "cold", "normal", 
    "strong", "no"}, {"overcast", "cold", "normal", "strong", 
    "yes"}, {"sunny", "mild", "high", "weak", "no"}, {"sunny", "cold",
     "normal", "weak", "yes"}, {"rainy", "mild", "normal", "weak", 
    "yes"}, {"sunny", "mild", "normal", "strong", "yes"}, {"overcast",
     "mild", "high", "strong", "yes"}, {"overcast", "hot", "normal", 
    "weak", "yes"}, {"rainy", "mild", "high", "strong", "no"}};

the input for classification:

input = Rest @ Thread[data[[All, ;; 4]] -> data[[All, 5]]]

The attribute space we have:

allAttributes = Union /@ Table[data[[All, i]], {i, 1, 4}]

and all possible inputs (not only the given 14 ones):

allTuples = Tuples[allAttributes];

with an Automatic classification we get:

   cfAutomatic = Classify[input];
cfAutomatic /@ allTuples // Tally

(* {{"yes", 30}, {"no", 6}} *)

with a DecisionTree we get

    cfTree = Classify[input, Method -> "DecisionTree"];
cfTree /@ allTuples // Tally

(* {{"yes", 36}} *)

This is "interesting". The given example can be found very widespread in literature with appropriate DT calculated using, e.g. Gini-Index and ID3 or CART). This result is clearly not "correct" - or I made a stupid error. Can anyone give a hint what goes wrong or what I did wrong?

Answer 1

The construction of the space of possible values for each attribute must not include the attribute labels (feature names or variable names). To do this, the variable you have named allAttributes is calculated, using the restriction data[[2;;,i], as follows: allAttributes = Union /@ Table[data[[2;;, i]], {i, 1, 4}]. We get : {{"overcast", "rainy", "sunny"}, {"cold", "hot", "mild"}, {"high", "normal"}, {"strong", "weak"}} then apply. allTuples = Tuples[allAttributes] cfAutomatic = Classify[input] cfAutomatic /@ allTuples// Tally cfTree = Classify[input, Method -> "DecisionTree"] cfTree /@ allTuples// Tally With the new allAttributes set, the results obtained confirm the good behaviour of the Classify function for both cfAutomatic and cfTree : {{"yes", 24}, {"no", 12}}