2
$\begingroup$

I'm doing an experiment of objects classification on dataset CIFAR-10 and I'm doing this using a neural network. The basic code of this experiment is come from the demo on Mathematica website and I tried to add a part of evaluating the classifier I trained.

I used CNN as a classifier and this is what it look like:

    lenet = NetChain[ {ConvolutionLayer[20, 5], Ramp, PoolingLayer[2, 2],        ConvolutionLayer[50, 5], Ramp, PoolingLayer[2, 2], FlattenLayer[], 500, Ramp, 2, SoftmaxLayer[]}, "Output" -> NetDecoder[{"Class", classes}], "Input" -> NetEncoder[{"Image", {32, 32}}] ]

After training, I created a ClassifierMeasurementsObject using function ClassifierMeasurements:

    cm=ClassifierMeasurements[trained,testData]

where trained is the name of trained CNN module.

Then I use this code to see what I can do with the ClassifierMeasurementsObject:

    cm["Properties"]

and I got the output:

    {Accuracy, AccuracyRejectionPlot, ClassRejectionRate,   ConfusionFunction, ConfusionMatrix, ConfusionMatrixPlot, Error, FScore, InversePerplexity, Likelihood, LogLikelihood, Perplexity, Precision, Properties, Recall, RejectionRate}

what I got from the output is much less than the document for ClassifierMeasurements, something missed such as "AreaUnderROCCurve", "CorrectlyClassifiedExamples", "Examples", {"Examples",i->j}, "WorstClassifiedExamples", etc.

I don't know why would this happen. Is this because the algorithm I use or the dataset I use? Do I have any way to find and split out all the misclassified samples?

Thank you very much!

$\endgroup$
5
  • $\begingroup$ Could you add a link to where you see the missing properties? I don't see "AreaUnderROCCurve" in the documentation. $\endgroup$ Commented Jan 2, 2017 at 17:20
  • $\begingroup$ @xslittlegrass ,please see here, in the document of ClassifierMeasurements, reference.wolfram.com/language/ref/ClassifierMeasurements.html and here, the document of ClassifierMeasurementsObject, which is the output of ClassifierMeasurement,reference.wolfram.com/language/ref/… $\endgroup$
    – Chang Liu
    Commented Jan 2, 2017 at 17:25
  • $\begingroup$ I'm still not seeing "AreaUnderROCCurve", could you point out the exact place in the documentation? Thanks. $\endgroup$ Commented Jan 2, 2017 at 18:03
  • $\begingroup$ @xslittlegrass In the both documents, under the basic description of the functions' parameters, there is a hidden menu titled by "more information and options", and you can see the hidden content after clicking it, where you can find all the properties. $\endgroup$
    – Chang Liu
    Commented Jan 3, 2017 at 7:44
  • $\begingroup$ I find the properties such as {"Examples",i->j}, "Accuracy", ` "AccuracyRejectionPlot"`, etc. in "Details and Options", but I don't see the "AreaUnderROCCurve" property in there. $\endgroup$ Commented Jan 4, 2017 at 17:16

0

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Browse other questions tagged or ask your own question.