proper use of PredictorMeasurements[]

Question

I've been playing around with Predict[] with multi-dimensional datasets and, for small training sets anyway, things seem to work correctly. For example,

trainingset = {<|"age" -> 47, "sex" -> "M", "height" -> 100, 
    "weight" -> 60|>, <|"age" -> 22, "sex" -> "M", "height" -> 90, 
    "weight" -> 55|>, <|"age" -> 43, "sex" -> "M", "height" -> 110, 
    "weight" -> 61|>, <|"age" -> 23, "sex" -> "F", "height" -> 100, 
    "weight" -> 41|>, <|"age" -> 33, "sex" -> "F", "height" -> 80, 
    "weight" -> 50|>, <|"age" -> 43, "sex" -> "F", "height" -> 70, 
    "weight" -> 51|>};
testset = {<|"age" -> 37, "sex" -> "M", "height" -> 100|>, <|
    "age" -> 22, "sex" -> "M", "height" -> 90|>, <|"age" -> 43, 
    "sex" -> "F", "height" -> 80|>, <|"age" -> 33, "sex" -> "F", 
    "height" -> 70|>};
p1 = Predict[trainingset -> "weight", PerformanceGoal -> "Quality", 
   Method -> "RandomForest"];

We can get predictions from the p1 PredictorFunction with

Map[Append[#, "prediction" -> p1[#]] &, testset] (* this works *)

I can then compute residuals, etc., myself.

Since version 10, Wolfram Language has included the function PredictorMeasurements[], and the documentation suggests that I should be able to get the predictions above, plus residual reports and other information, with

PredictorMeasurements[p1, testset]

But this does not work. I get the following error: PredictorMeasurements::bdfmt: Argument {<|age->37,sex->M,height->100,weight->60|>,<|age->22,sex->M,height->90|>,<|age->43,sex->F,height->80|>,<|age->33,sex->F,height->70|>} should be a rule or a list of rules.

What am I missing?

Answer 1

It appears that PredictorMeasurements requires exactly the same variables for the test set as Predict for the training set.

For example, when

testset2={<|"age"->37,"sex"->"M","height"->100,"weight"->49|>,<|"age"->22,"sex"->"M","height"->90,"weight"->58|>,<|"age"->43,"sex"->"F","height"->80,"weight"->55|>,<|"age"->33,"sex"->"F","height"->70,"weight"->54|>};

is called by

PredictorMeasurements[p1,testset2->"weight","StandardDeviation"]

the result is

5.64506