13
$\begingroup$

Trying to learn the new Dataset features, I am playing with the iris dataset:

iris=SemanticImport["http://aima.cs.berkeley.edu/data/iris.csv"];
iris[1;;3]

this is only a subset of iris

Now I would like to use Classify to predict the Species from the other Variables. From the documentation of Classify it seems I need to write a query that outputs something like

{{{5.1,3.5,1.4,0.2} -> "setosa"}, {{4.9,3.,1.4,.2} -> "setosa"}}

and so on. But here I am stuck. How do I write this query function? I tried

trainQuery = Function[row, 
   Rule[row[All, 1;;4], row[All, 5]]
];

but this fails.

Any help appreciated!

$\endgroup$
  • $\begingroup$ And what is the source of the iris dataset? It's not available from ExampleData["Dataset"]. $\endgroup$ – m_goldberg Oct 15 '14 at 15:00
  • $\begingroup$ Good point. Added source for iris. $\endgroup$ – Karsten W. Oct 15 '14 at 17:45
  • 3
    $\begingroup$ @m_goldberg, it's actually built-in: ExampleData[{"Statistics", "FisherIris"}]. $\endgroup$ – J. M. is away Oct 21 '15 at 21:18
12
$\begingroup$

Try this:

iris[1 ;; 5, {Most@# -> Last@#} &]

Mathematica graphics

You can use Normal to get it out as a List

Normal @ iris[1 ;; 5, {Most@# -> Last@#} &]
{{{5.1, 3.5, 1.4, 0.2} -> "setosa"}, {{4.9, 3., 1.4, 0.2} -> "setosa"},
 {{4.7, 3.2, 1.3, 0.2} -> "setosa"}, {{4.6, 3.1, 1.5, 0.2} ->  "setosa"},         
 {{5., 3.6, 1.4, 0.2} -> "setosa"}}

To do it for all rows simply:

Normal @ iris[All, {Most@# -> Last@#} &]

Note: if you have to use it with Classify there's still one more step. Flatten:

Flatten[Normal @ iris[All, {Most@# -> Last@#} &], 1]

And if that's the case, the Flatten may be avoided all together by using Sequence

Normal @ iris[All, Sequence[Most@# -> Last@#] &]

Update

As per your question in the comments, you can use Association with Classify like this:

Classify @ Normal @ iris[Map[Association], Sequence[Last@# -> Most@#] &][Merge[#, Identity] &]

Or just use operator forms and Composition

Classify @ Normal @ iris[Map[Association] /* Merge[Identity], Sequence[Last@# -> Most@#] &]
$\endgroup$
  • $\begingroup$ That is very helpful, thank you! Do you think it is possible to apply Classify without converting to a list? I tried <| Most@# -> Last@#|>& but this is not accepted by Classify. $\endgroup$ – Karsten W. Oct 15 '14 at 18:11
  • $\begingroup$ @KarstenW. I haven't used Classify before, so I don't know anything about it. Sorry, I can look into it though. $\endgroup$ – RunnyKine Oct 15 '14 at 18:12
  • $\begingroup$ @KarstenW., see my update. Does that help? $\endgroup$ – RunnyKine Oct 15 '14 at 18:57
  • $\begingroup$ Yes it sure does. I still have to accommodate with this new Dataset feature, and start to see its elegance.. :-) Your examples help me understand. $\endgroup$ – Karsten W. Oct 16 '14 at 16:19
  • 3
    $\begingroup$ @RunnyKine For what it is worth, Classify and Predict will both support the syntax Classify[Dataset[...] -> colname] in version 10.0.2. $\endgroup$ – Taliesin Beynon Oct 24 '14 at 18:58
3
$\begingroup$

Just for reference, here is the example for Classify I worked out with your help:

iris = SemanticImport["http://aima.cs.berkeley.edu/data/iris.csv"];
iris = iris[All, <|
    "SepalLength" -> 1, "SepalWidth" -> 2, "PetalLength" -> 3, 
    "PetalWidth" -> 4, "Species" -> 5|>] 
trainIdx = RandomSample[Range[1, Length[iris]], Round[Length[iris]*0.9]];
trainSet = Normal@iris[trainIdx, Most@# -> Last@# &];
testIdx = Complement[Range[1, Length[iris]], trainIdx];
testVal = Normal@iris[testIdx, Last];

clss = Classify[trainSet, Method -> "RandomForest"];
pred = clss[Normal@iris[testIdx, Most]];
Print["RF: ", Count[MapThread[Equal, {pred, testVal}] , False]]
$\endgroup$
3
$\begingroup$

I found it interesting to purposefully handicap the classifier by only giving it 50% of the data to train on, testing with the remaining 50%. Then looked at the results in a confusion matrix .

Add this to the bottom of Karsten's code:

testSet = Normal@iris[testIdx, Most@# -> Last@# &];
cm = ClassifierMeasurements[clss, testSet];
cm["ConfusionMatrixPlot"]

Yields:

enter image description here

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.