4
$\begingroup$

I would like to extract a graph of the history of a neural net's performance, as it is trained. This is so I can compare it with the equivalent from Tensorflow (for some reason an equivalent network in Mathematica is training much faster, and more accurately).

As an example, take the case study on MNIST digit classification from here: https://www.wolfram.com/language/11/neural-networks/digit-classification.html

Download training and testing data,

resource = ResourceObject["MNIST"]; 
trainingData = ResourceData[resource, "TrainingData"]; 
testData = ResourceData[resource, "TestData"];

Build a convolutional neural network,

lenet = NetChain[ {ConvolutionLayer[20, 5], Ramp, PoolingLayer[2, 2], 
         ConvolutionLayer[50, 5], Ramp, PoolingLayer[2, 2], FlattenLayer[], 500, 
         Ramp, 10, SoftmaxLayer[]}, 
         "Output" -> NetDecoder[{"Class", Range[0, 9]}], 
         "Input" -> NetEncoder[{"Image", {28, 28}, "Grayscale"}] ]

And train it,

lenet = NetTrain[lenet, trainingData, ValidationSet -> testData,
         MaxTrainingRounds -> 3];

You'll see a little box appear when this is run (on Mathematica 11; don't think it'll run on 10) like the following,

enter image description here

I would like to extract the data used to plot both the orange line (performance on the training set), and the blue line when it appears (yet to appear above but illustrates performance on the test set).

Does anyone know how to do this?

Best,

Ben

$\endgroup$
  • 3
    $\begingroup$ If I recall correctly at WTC2016 at one of the neural net talks there was mention of some new options coming to NetTrain that would allow you to capture intermediate results. It was said they could be for version 11.1 but we wont know for sure until it is released. $\endgroup$ – Edmund Nov 8 '16 at 1:35
  • $\begingroup$ Ok, thanks for the heads up! $\endgroup$ – ben18785 Nov 8 '16 at 1:42
2
$\begingroup$

You can either use the TrainingProgressFunction as suggested by Edmund, or simply use the property argument in NetTrain. For example, this will return the trained net and the batch loss and validation lost history:

{lenet, bloss, vloss} = 
  NetTrain[lenet, trainingData, 
   Automatic, {"TrainedNet", "BatchLossList", "ValidationLossList"}, 
   ValidationSet -> testData, MaxTrainingRounds -> 3];

batchsize=24;
round=10;

Show[
 ListLogPlot[bloss, DataRange -> {1, batchsize*round}, Joined -> True,
   PlotStyle -> Orange, Axes -> False, Frame -> True, 
  FrameLabel -> {"batch", "loss"}],
 ListLogPlot[vloss, Joined -> True, Mesh -> All]]

enter image description here

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.