# Transform Dataset so it can be used as training set for Classify

Trying to learn the new Dataset features, I am playing with the iris dataset:

iris=SemanticImport["http://aima.cs.berkeley.edu/data/iris.csv"];
iris[1;;3]


Now I would like to use Classify to predict the Species from the other Variables. From the documentation of Classify it seems I need to write a query that outputs something like

{{{5.1,3.5,1.4,0.2} -> "setosa"}, {{4.9,3.,1.4,.2} -> "setosa"}}


and so on. But here I am stuck. How do I write this query function? I tried

trainQuery = Function[row,
Rule[row[All, 1;;4], row[All, 5]]
];


but this fails.

Any help appreciated!

• And what is the source of the iris dataset? It's not available from ExampleData["Dataset"]. – m_goldberg Oct 15 '14 at 15:00
• Good point. Added source for iris. – Karsten W. Oct 15 '14 at 17:45
• @m_goldberg, it's actually built-in: ExampleData[{"Statistics", "FisherIris"}]. – J. M. is away Oct 21 '15 at 21:18

Try this:

iris[1 ;; 5, {Most@# -> Last@#} &]


You can use Normal to get it out as a List

Normal @ iris[1 ;; 5, {Most@# -> Last@#} &]

{{{5.1, 3.5, 1.4, 0.2} -> "setosa"}, {{4.9, 3., 1.4, 0.2} -> "setosa"},
{{4.7, 3.2, 1.3, 0.2} -> "setosa"}, {{4.6, 3.1, 1.5, 0.2} ->  "setosa"},
{{5., 3.6, 1.4, 0.2} -> "setosa"}}


To do it for all rows simply:

Normal @ iris[All, {Most@# -> Last@#} &]


Note: if you have to use it with Classify there's still one more step. Flatten:

Flatten[Normal @ iris[All, {Most@# -> Last@#} &], 1]


And if that's the case, the Flatten may be avoided all together by using Sequence

Normal @ iris[All, Sequence[Most@# -> Last@#] &]


Update

As per your question in the comments, you can use Association with Classify like this:

Classify @ Normal @ iris[Map[Association], Sequence[Last@# -> Most@#] &][Merge[#, Identity] &]


Or just use operator forms and Composition

Classify @ Normal @ iris[Map[Association] /* Merge[Identity], Sequence[Last@# -> Most@#] &]

• That is very helpful, thank you! Do you think it is possible to apply Classify without converting to a list? I tried <| Most@# -> Last@#|>& but this is not accepted by Classify. – Karsten W. Oct 15 '14 at 18:11
• @KarstenW. I haven't used Classify before, so I don't know anything about it. Sorry, I can look into it though. – RunnyKine Oct 15 '14 at 18:12
• @KarstenW., see my update. Does that help? – RunnyKine Oct 15 '14 at 18:57
• Yes it sure does. I still have to accommodate with this new Dataset feature, and start to see its elegance.. :-) Your examples help me understand. – Karsten W. Oct 16 '14 at 16:19
• @RunnyKine For what it is worth, Classify and Predict will both support the syntax Classify[Dataset[...] -> colname] in version 10.0.2. – Taliesin Beynon Oct 24 '14 at 18:58

Just for reference, here is the example for Classify I worked out with your help:

iris = SemanticImport["http://aima.cs.berkeley.edu/data/iris.csv"];
iris = iris[All, <|
"SepalLength" -> 1, "SepalWidth" -> 2, "PetalLength" -> 3,
"PetalWidth" -> 4, "Species" -> 5|>]
trainIdx = RandomSample[Range[1, Length[iris]], Round[Length[iris]*0.9]];
trainSet = Normal@iris[trainIdx, Most@# -> Last@# &];
testIdx = Complement[Range[1, Length[iris]], trainIdx];
testVal = Normal@iris[testIdx, Last];

clss = Classify[trainSet, Method -> "RandomForest"];
pred = clss[Normal@iris[testIdx, Most]];
Print["RF: ", Count[MapThread[Equal, {pred, testVal}] , False]]


I found it interesting to purposefully handicap the classifier by only giving it 50% of the data to train on, testing with the remaining 50%. Then looked at the results in a confusion matrix .

Add this to the bottom of Karsten's code:

testSet = Normal@iris[testIdx, Most@# -> Last@# &];
cm = ClassifierMeasurements[clss, testSet];
cm["ConfusionMatrixPlot"]


Yields: