How do I access the parameters in Classify using Support Vector Machine as method?

Question

I have a dataset that I want to classify and I do the following:

clf = Classify[xtrain -> ytrain, Method -> "SupportVectorMachine"]

How can I use my own values for the parameters of the SupportVectorMachine ?

Answer 1

You can try setting them as suboptions

Classify[xtrain -> ytrain, 
 Method -> {"SupportVectorMachine", "KernelType" -> "Linear", 
   "SoftMarginParameter" -> 2}]

Answer 2

It is easy to get access to the parameters of a SVM. Let c be something like

c = Classify[
  Table[data1[[i, 1]] -> data1[[i, 2]], {i, 1, Length[data1]}], 
  Method -> {"SupportVectorMachine", 
    "KernelType" -> "RadialBasisFunction"}]

then we get the support vectors, support vector coefficients, rho and gamma scaling parameters as follows:

 GammaScalingParameter = 
      c[[1]]["Model"]["SVMParameters"]["GammaScalingParameter"];

    supportvectors = 
      c[[1]]["Model"]["TrainedModel"][[1]]["supportVectors"];

supportVectorCoefficients = 
 c[[1]]["Model"]["TrainedModel"][[1]]["supportVectorCoefficients"]

rho = c[[1]]["Model"]["TrainedModel"][[1]]["rho"]

and finally we can compute the decision function f

f[z_] :=Sum[supportVectorCoefficients[[i]] K[supportvectors[[i]], z], {i, 1, 
    Length[supportvectors]}] - rho

whereby K is the Radial basis kernel

K[x1_, x2_] := Exp[-GammaScalingParameter Norm[(x1 - x2)]^2]

I think the whole procedure works also for other kernels.

Here is an illustration. Here we have 2 groups with data (Fig.1)

Here we illustrate the decision areas of the SVM

and here we illustrate the values of the decision function of each data point: