6
$\begingroup$

I have used Mathematica's automated Classify function:

class = Classify[trainlist, Method -> "NeuralNetwork", 
   PerformanceGoal -> "Quality"];

acc = ClassifierMeasurements[class, testlist];

'trainlist' has 720 examples and 'testlist' has 180 examples. I get 89% accuracy. Great. But now I would like build the network from scratch so I can understand it better. That turns out to be very difficult since Classify[] is like a black box.

enter image description here

enter image description here

The most information I can get about how the network is built is from these two tables. So I tried to build my own from this information:

net = NetChain[{LinearLayer[50, "Input" -> 251], 
   ElementwiseLayer["SELU"], LinearLayer[50, "Input" -> 50], 
   ElementwiseLayer["SELU"], LinearLayer[9, "Input" -> 50], 
   SoftmaxLayer["Input" -> {9}]}]

netin = NetInitialize[net]

nettrain = 
 NetTrain[netin, trainlist[[1 ;; 576]], Method -> "ADAM", 
  MaxTrainingRounds -> 1000, ValidationSet -> trainlist[[577 ;; 720]]]

In all there are about 15500 parameters (I assume they mean weights). But when I check the accuracy, it's only 0.1111. I have nine classifications so that means it only does as well as random guessing. Why is it so difficult to build a network that does as well as the automated function?

$\endgroup$
6
  • $\begingroup$ Did you have a look at the internals of your class? Just do class[[1]] and you can see it all. Currently in 12.2.0.0 (and I think also 12.1) you can extract the internal NetChain from the ClassifierFunction just by doing class[[1]]["Model"]["Network"] . By the way, Classify might also be doing Standardize on your training data too. $\endgroup$
    – flinty
    Mar 7, 2021 at 23:21
  • $\begingroup$ class[[1]] yields what is in the second picture. It is slightly more revealing than Information[] but still very cryptic. $\endgroup$
    – ngc1300
    Mar 7, 2021 at 23:28
  • $\begingroup$ Does ClassifierInformation not give you what you need? $\endgroup$
    – b3m2a1
    Mar 11, 2021 at 20:57
  • $\begingroup$ That’s what the first image gives. $\endgroup$
    – ngc1300
    Mar 11, 2021 at 22:35
  • $\begingroup$ @pmac Classify is not like a black box. class[[1]]["Model"]["Network"] gives you the NetChain used under the hood. Please try your training using net = class[[1]]["Model"]["Network"] and report the accuracy. $\endgroup$
    – flinty
    Mar 14, 2021 at 18:31

0

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Browse other questions tagged or ask your own question.