0
$\begingroup$

I'm using NetTrain to make a simple classifier network that takes integers 0-9 as inputs and classifies them into ten classes labeled by integers 0-9 as well. My training data maps the inputs to the identically labeled classes, i.e., a 3 would be mapped to class "3".

trainingData = {0 -> 0, 0 -> 0, 0 -> 1, 0 -> 0, 0 -> 0, 0 -> 4,  0 -> 0, 1 -> 1, 1 -> 1, 1 -> 1, 1 -> 1, 1 -> 2, 1 -> 1, 1 -> 1, 1 -> 1, 2 -> 2, 2 -> 2, 2 -> 2, 2 -> 2, 2 -> 2, 2 -> 2, 2 -> 2, 2 -> 2, 3 -> 1, 3 -> 3, 3 -> 3, 3 -> 3, 3 -> 3, 3 -> 3, 3 -> 3,  3 -> 3, 4 -> 1, 4 -> 4, 4 -> 9, 4 -> 4, 4 -> 4, 4 -> 4, 4 -> 4, 4 -> 4, 4 -> 4, 5 -> 5, 5 -> 5, 5 -> 5, 5 -> 5, 5 -> 5, 5 -> 5, 5 -> 5, 6 -> 6, 6 -> 6, 6 -> 6, 6 -> 6, 6 -> 6, 6 -> 6, 6 -> 6, 6 -> 6, 7 -> 7, 7 -> 1, 7 -> 7, 7 -> 7, 7 -> 7, 7 -> 5, 7 -> 7, 7 -> 7, 8 -> 8, 8 -> 8, 8 -> 8, 8 -> 8, 8 -> 3, 8 -> 8, 8 -> 8, 8 -> 8, 8 -> 8, 8 -> 8, 8 -> 8, 8 -> 8, 9 -> 9, 9 -> 9, 9 -> 9, 9 -> 9, 9 -> 8, 9 -> 9, 9 -> 9, 9 -> 9, 9 -> 9, 9 -> 9, 9 -> 9, 9 -> 9, 9 -> 9, 9 -> 9};
testData = {0 -> 0, 0 -> 0, 0 -> 0, 1 -> 1, 1 -> 1, 1 -> 1, 2 -> 2, 2 -> 2, 2 -> 2, 3 -> 3, 3 -> 3, 3 -> 3, 4 -> 4, 4 -> 4, 4 -> 4, 5 -> 5, 5 -> 5, 5 -> 5, 6 -> 6, 6 -> 6, 6 -> 6, 7 -> 7, 7 -> 7, 7 -> 7, 8 -> 8, 8 -> 8, 8 -> 1, 9 -> 9, 9 -> 9, 9 -> 9};

encoder = NetEncoder["Scalar"];
decoder = NetDecoder[{"Class", Range[10] - 1}];
net = NetChain[{LinearLayer[100], LinearLayer[10], SoftmaxLayer[]}, "Input" -> encoder, "Output" -> decoder]
init = NetInitialize[net]
results = NetTrain[init, trainingData, All]

However, when I test my network on the validation data set(which is pretty much a small sample of the training data), the confusion matrix indicates that my net simply outputs the same randomly chosen class for any input, and the accuracy is only around 10%. What went wrong here?

Inaccurate neural net predictions

$\endgroup$
  • $\begingroup$ Welcome to the Mathematica Stack Exchange! Could you post some of your training data - without it, we can't test your code. $\endgroup$ – Carl Lange Dec 27 '18 at 17:53
  • 1
    $\begingroup$ I've just tried your network myself using test data generated by doing trainingData = With[{r = RandomInteger[{0, 9}, 10000]}, Thread[r -> r]], and it appears to work perfectly with 100% accuracy. $\endgroup$ – Carl Lange Dec 27 '18 at 17:57
  • $\begingroup$ Thanks for adding your training data. Is that the whole dataset that you have, or is there more? As it is, there isn't really enough data to train an accurate network with. However, I tried to train with that data and got a top-3 accuracy (ClassifierMeasurements[results["TrainedNet"], testData, "Accuracy" -> 3]) of 96%. It's possible your attempt at training simply happened to stop at a different local minimum. Consider adding more data if you can. $\endgroup$ – Carl Lange Dec 28 '18 at 1:42

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Browse other questions tagged or ask your own question.