4
$\begingroup$

I'm trying to understand the evaluation mechanism for ClassifierFunction.

Here we train a simple linear SVM on some toy data:

training = {1 -> "A", 2 -> "A", 3.5 -> "B", 4 -> "B"};
Clear@c
c = Classify[training, Method -> {"SupportVectorMachine", "KernelType" -> "Linear"}]
c[2.6, "Probabilities"]

enter image description here

We can extract the weights, biases and other data from the trained svm model like so:

svmData = c[[1, "Models", 1]]

enter image description here

Here are all the relevant parameters from the model extracted:

sv = svmData[["TrainedModel", 1, "supportVectors"]]
sc = svmData[["TrainedModel", 1, "supportVectorCoefficients"]]
\[Rho] = svmData[["TrainedModel", 1, "rho"]]
ker = svmData[["SVMParameters", "KernelType"]]
\[DoubledGamma] = svmData[["SVMParameters", "GammaScalingParameter"]]
margin = svmData[["SVMParameters", "SoftMarginParameter"]]
deg = svmData[["SVMParameters", "PolynomialDegree"]]
pc = svmData[["ProbabilityCoefficients"]]

Now, I'd like to recompute the scores

c[2.6, "Probabilities"] 
 (* = <|"A"->0.520637,"B"->0.479363|>*)

by hand using the extracted values from the ClassifierFunction.

What are the steps to do this prediction and get the same values?

$\endgroup$
  • $\begingroup$ That's a typo on my part should be exactly the same as before you're right. $\endgroup$ – M.R. Mar 29 '16 at 19:47
  • 1
    $\begingroup$ @C.E. Typo fixed $\endgroup$ – M.R. Mar 29 '16 at 20:18
  • $\begingroup$ One motivation here is that if one can understand how mma computes the prediction then you can successfully export the svms generated by Classify. $\endgroup$ – M.R. Mar 29 '16 at 21:30
3
$\begingroup$

In order to find the source code that implements the computation of probabilities, evaluate

Needs["GeneralUtilities`"]
PrintDefinitions[MachineLearning`PackageScope`ClassifierEvaluation]

The relevant definition is second to last in the list. As we can see it knows two different ways to compute the probabilities. "One versus one" and "one versus all". The probabilities using the first method are computed with sigma. The probabilities using the second method - which is default, i.e. what you asked about - are computed with multiClassProbability. This function in its turn uses a function called pkpdProbabilities. I did some searching, and found out that pkpd stands for Price, Kner, Personnaz, and Dreyfus.

One text that explains how this method works can be found here.

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.