4
$\begingroup$

Tested@Mathematica 11.1 on Windows

ModelA

resource = ResourceObject["MNIST"];
trainingData = ResourceData[resource, "TrainingData"];
testData = ResourceData[resource, "TestData"];

model=Classify[trainingData,Method->"LogisticRegression"];
cm=ClassifierMeasurements[model,testData]
cm@"Accuracy"
0.9261

ModelB

model=Classify[trainingData,Method->"LogisticRegression",PerformanceGoal->"Quality"];
cm=ClassifierMeasurements[model,testData]
cm@"Accuracy"
0.8964

"Quality" maximize accuracy of the classifier.

model=Classify[trainingData,Method->"NearestNeighbors",PerformanceGoal->"Quality"];
cm=ClassifierMeasurements[model,testData]
cm@"Accuracy"
0.9659

Of course, glad to know the best parameters of Classify in LogisticRegression.

An interest thing: Quality is faster in some condition?

t1=SessionTime[];
model1=Classify[trainingData,Method->{"LogisticRegression","L2Regularization"->10},PerformanceGoal->"Quality"];
t2=SessionTime[];
t=t2-t1
80.5658897
t1=SessionTime[];
model2=Classify[trainingData,Method->{"LogisticRegression","L2Regularization"->10},PerformanceGoal->"TrainingSpeed"];
t2=SessionTime[];
t=t2-t1
166.8431739
$\endgroup$
2
$\begingroup$

The problem is PerformanceGoal->Quality will no only take effect on model-training parameters, but also the processor and other places. Classify thinks without doing DimensionReduce is better, actually doing DimensionReduce is better.

model1=Classify[trainingData,FeatureExtractor->"NumericVector", Method->{"LogisticRegression","L2Regularization"->1},PerformanceGoal->"Quality"];
cm=ClassifierMeasurements[model1,testData];
cm@"Accuracy"
0.9262

model2=Classify[trainingData, Method->{"LogisticRegression","L2Regularization"->1},PerformanceGoal->"Quality"];
cm=ClassifierMeasurements[model2,testData];
cm@"Accuracy"
0.8969

model3=Classify[trainingData, Method->{"LogisticRegression","L2Regularization"->1}];
cm=ClassifierMeasurements[model3,testData];
cm@"Accuracy"
0.9261

Without DimensionReduce, the SessionTime is of course faster.

Actually NumericVecotr by default is doing Linear DimensionReduce, but here it keep the all dimension, so FeatureDimension is 784. But model3 doesn't do DimensionReduce in Processor.

KeyTake[Options[model1][[1]]["Models"][[1]],"FeatureNumber"]
<|FeatureNumber->784|>
KeyTake[Options[model2][[1]]["Models"][[1]],"FeatureNumber"]
<|FeatureNumber->784|>
 KeyTake[Options[model3][[1]]["Models"][[1]],"FeatureNumber"]
<|FeatureNumber->393|>

Options[model1][[1]]//Keys
{Basic,Input,Output,Combiner,Decision,Models,Log}
Options[model1][[1]]["Input"]
<|Preprocessor->ToMLDataset,Processor->ConformImage->ImageExtractNumericalVector->DimensionReduceNumericalVector->Standardize|>
Options[model2][[1]]["Input"]
<|Preprocessor->ToMLDataset,Processor->ImputeMissing->ConformImage->ImageExtractNumericalVector|>
Options[model3][[1]]["Input"]
<|Preprocessor->ToMLDataset,Processor->ImputeMissing->ConformImage->ImageExtractNumericalVector|>
$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.