Suppose I have iris data In the Dataset Format. enter image description here

How to split the iris data Into training and testing for Machine learning?

For example,Transform the data for Classify.


2 Answers 2


Let's use the Titanic dataset

tit = ExampleData[{"Dataset", "Titanic"}];

Let's see it's columns

tit[Union, Keys]

Let's choose an objective

obj = "survived";

Let's add an id to each row

tit = tit[AssociationThread[Range@Length@#, #] &];

Let's create a database that splits the features and objective, to make this general.

titSplit = 
  tit[All, <|"Features" -> Values@*KeyDrop[obj], 
    "Objective" -> Key[obj]|>];

Let's split the rows arbitrarily to have a test and train set.

numTraining = 200;
ids = Range@tit[Length];
testIds = ids~RandomSample~numTraining;
trainIds = ids~Complement~testIds;

titUnclass = titSplit[<|"Test" -> testIds, "Train" -> trainIds|>];

Let's train the classifier

cfun = titUnclass["Train", Classify@*Values, #Features -> #Objective &];

Let's create a new dataset appending the classifications as a column in the Test dataset.

titClass = 
  titUnclass[{"Test" -> 
     Query[All, Append[#, <|"Classified as" -> cfun@#Features|>] &]}];

Or perhaps just have the results in a separate database results = titUnclass["Test", All, cfun@#Features &];

Let's ask for performance measures

cfm = titUnclass["Test", 
   ClassifierMeasurements[cfun, #] & @* Values, 
   #Features -> #Objective &];


(* 0.75 *)


This is based on the OP @PhilChang nice suggestion in the comments

RandomSample works on Datasets

rtit = RandomSample@ExampleData[{"Dataset", "Titanic"}];


obj = "survived"; numTrain = 200;
ctit = rtit[<|"Train" -> (;; numTrain), "Test" -> (numTrain + 1 ;;)|>];


cfun = ctit["Train", GroupBy[Key[obj] -> KeyDrop[obj]], Values][


cfm = ctit["Test", GroupBy[Key[obj] -> KeyDrop[obj]], Values][
   ClassifierMeasurements[cfun, #] &];
  • 2
    $\begingroup$ I get an Accuracy = 0.63. Naive splitting like this is often biased by the dataset layout, eg, look at the distribution of "class": trainTit[Map[First] /* Counts, "Features"] --> "1st" -> 200 versus testTit[Map[First]...] --> "1st" -> 123, "2nd" -> 277, "3rd" -> 709. Typically, bootstrap or other resampling methods are used to minimize the effects of clustering. Certainly you'd want to randomize the partition. $\endgroup$ Aug 12, 2014 at 19:49
  • 8
    $\begingroup$ We need to make this easier, probably with a new function. Especially making sure you preserve the frequencies of whatever classes when you do the split. $\endgroup$ Aug 12, 2014 at 20:30
  • 1
    $\begingroup$ Thank you all,I learned a lot from it. For this moment the best way maybe: 1. Random the dataset, randsample = RandomSample @ titanic; 2. get train and test data. train = RandSample[;;3000];test = randomSample[3001;;]; 3. transform the data set for classify use. train = titanic[GroupBy[Key[obj]], All, Values@*Normal@*KeyDrop[obj]]; $\endgroup$
    – PhilChang
    Aug 13, 2014 at 6:31
  • 3
    $\begingroup$ If Only Classify accept a Dataset as its argument:Classify[mydataset,y,{x1,x2,...,xp}] $\endgroup$
    – PhilChang
    Aug 13, 2014 at 6:36
  • 1
    $\begingroup$ @GordonCoale You can use the ValidationSet option to Classify and Predict to override our internal cross-validation if you have your own test set. Also, @Rojo, note that in 10.0.2 you can use the Classify[data -> out] shorthand to indicate that the column name or number is the one being predicted, so you don't have to split off the features from the output yourself. $\endgroup$ Feb 9, 2015 at 18:50

I think there is a resource function now...



Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.