2
$\begingroup$

I would like to plot some PR curves for a few different applications:

  • results on a test set for a (multilabel multiclass) classifier
  • results on a test set for a CBIR function that returns ranked indices based on feature distance from a query image

Is there some way of using PredictorMeasurements or ClassifierMeasurements to do this?

  • There may be a way to transform the “ROCCurve” property from ClassifierMeasurements into aPR curve.
  • I'd like to show the averaging out of different PR curves if there are many test/query sets.

Perhaps someone can show a worked example of how to do this, or point to some package implementation?

References:

$\endgroup$
1
$\begingroup$

Anton Antonov provides a worked out example in a blog post of his, as mentioned in comments. See: Linear regression with ROC. Interesting material is also present in this post on the Wolfram Community: Basic example of using ROC with Linear regression.

$\endgroup$
  • $\begingroup$ Thanks, but PR curves are not the same as ROC curves. $\endgroup$ – M.R. May 29 '18 at 15:55
  • $\begingroup$ @M.R. This was mostly to preserve what seemed an interesting reference from comments. If you would rather the question still appeared unanswered though (perhaps to attract attention), I'd be happy to delete the answer. $\endgroup$ – MarcoB May 29 '18 at 16:06
  • $\begingroup$ No it’s definitely better than nothing $\endgroup$ – M.R. May 29 '18 at 16:12
  • $\begingroup$ I just wish ClassifierMeasurements supported them as PRCurve is probably the most important measurement. I have found that ROCCurves are more esoteric to people. $\endgroup$ – M.R. May 29 '18 at 16:15
  • $\begingroup$ @M.R. You can to Precision-Recall curves with the package used in the references, ROCFunctions.m. That package provides all functions in Wikipedia's article "Receiver Operating Characteristic". $\endgroup$ – Anton Antonov May 29 '18 at 17:07

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.