6
$\begingroup$

I have used Mathematica's automated Classify function:

class = Classify[trainlist, Method -> "NeuralNetwork", 
   PerformanceGoal -> "Quality"];

acc = ClassifierMeasurements[class, testlist];

'trainlist' has 720 examples and 'testlist' has 180 examples. I get 89% accuracy. Great. But now I would like build the network from scratch so I can understand it better. That turns out to be very difficult since Classify[] is like a black box.

enter image description here

enter image description here

The most information I can get about how the network is built is from these two tables. So I tried to build my own from this information:

net = NetChain[{LinearLayer[50, "Input" -> 251], 
   ElementwiseLayer["SELU"], LinearLayer[50, "Input" -> 50], 
   ElementwiseLayer["SELU"], LinearLayer[9, "Input" -> 50], 
   SoftmaxLayer["Input" -> {9}]}]

netin = NetInitialize[net]

nettrain = 
 NetTrain[netin, trainlist[[1 ;; 576]], Method -> "ADAM", 
  MaxTrainingRounds -> 1000, ValidationSet -> trainlist[[577 ;; 720]]]

In all there are about 15500 parameters (I assume they mean weights). But when I check the accuracy, it's only 0.1111. I have nine classifications so that means it only does as well as random guessing. Why is it so difficult to build a network that does as well as the automated function?

$\endgroup$
6
  • $\begingroup$ Did you have a look at the internals of your class? Just do class[[1]] and you can see it all. Currently in 12.2.0.0 (and I think also 12.1) you can extract the internal NetChain from the ClassifierFunction just by doing class[[1]]["Model"]["Network"] . By the way, Classify might also be doing Standardize on your training data too. $\endgroup$ – flinty Mar 7 at 23:21
  • $\begingroup$ class[[1]] yields what is in the second picture. It is slightly more revealing than Information[] but still very cryptic. $\endgroup$ – pmac Mar 7 at 23:28
  • $\begingroup$ Does ClassifierInformation not give you what you need? $\endgroup$ – b3m2a1 Mar 11 at 20:57
  • $\begingroup$ That’s what the first image gives. $\endgroup$ – pmac Mar 11 at 22:35
  • $\begingroup$ @pmac Classify is not like a black box. class[[1]]["Model"]["Network"] gives you the NetChain used under the hood. Please try your training using net = class[[1]]["Model"]["Network"] and report the accuracy. $\endgroup$ – flinty Mar 14 at 18:31

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Browse other questions tagged or ask your own question.