Transform Dataset so it can be used as training set for Classify

Question

Trying to learn the new Dataset features, I am playing with the iris dataset:

iris=SemanticImport["http://aima.cs.berkeley.edu/data/iris.csv"];
iris[1;;3]

Now I would like to use Classify to predict the Species from the other Variables. From the documentation of Classify it seems I need to write a query that outputs something like

{{{5.1,3.5,1.4,0.2} -> "setosa"}, {{4.9,3.,1.4,.2} -> "setosa"}}

and so on. But here I am stuck. How do I write this query function? I tried

trainQuery = Function[row, 
   Rule[row[All, 1;;4], row[All, 5]]
];

but this fails.

Any help appreciated!

Answer 1

Try this:

iris[1 ;; 5, {Most@# -> Last@#} &]

You can use Normal to get it out as a List

Normal @ iris[1 ;; 5, {Most@# -> Last@#} &]

{{{5.1, 3.5, 1.4, 0.2} -> "setosa"}, {{4.9, 3., 1.4, 0.2} -> "setosa"},
 {{4.7, 3.2, 1.3, 0.2} -> "setosa"}, {{4.6, 3.1, 1.5, 0.2} ->  "setosa"},         
 {{5., 3.6, 1.4, 0.2} -> "setosa"}}

To do it for all rows simply:

Normal @ iris[All, {Most@# -> Last@#} &]

Note: if you have to use it with Classify there's still one more step. Flatten:

Flatten[Normal @ iris[All, {Most@# -> Last@#} &], 1]

And if that's the case, the Flatten may be avoided all together by using Sequence

Normal @ iris[All, Sequence[Most@# -> Last@#] &]

Update

As per your question in the comments, you can use Association with Classify like this:

Classify @ Normal @ iris[Map[Association], Sequence[Last@# -> Most@#] &][Merge[#, Identity] &]

Or just use operator forms and Composition

Classify @ Normal @ iris[Map[Association] /* Merge[Identity], Sequence[Last@# -> Most@#] &]

Answer 2

Just for reference, here is the example for Classify I worked out with your help:

iris = SemanticImport["http://aima.cs.berkeley.edu/data/iris.csv"];
iris = iris[All, <|
    "SepalLength" -> 1, "SepalWidth" -> 2, "PetalLength" -> 3, 
    "PetalWidth" -> 4, "Species" -> 5|>] 
trainIdx = RandomSample[Range[1, Length[iris]], Round[Length[iris]*0.9]];
trainSet = Normal@iris[trainIdx, Most@# -> Last@# &];
testIdx = Complement[Range[1, Length[iris]], trainIdx];
testVal = Normal@iris[testIdx, Last];

clss = Classify[trainSet, Method -> "RandomForest"];
pred = clss[Normal@iris[testIdx, Most]];
Print["RF: ", Count[MapThread[Equal, {pred, testVal}] , False]]

Answer 3

I found it interesting to purposefully handicap the classifier by only giving it 50% of the data to train on, testing with the remaining 50%. Then looked at the results in a confusion matrix .

Add this to the bottom of Karsten's code:

testSet = Normal@iris[testIdx, Most@# -> Last@# &];
cm = ClassifierMeasurements[clss, testSet];
cm["ConfusionMatrixPlot"]

Yields: