I have a neural network that works fine, but I would like to obtain some results on the Validation Set which provides some metrics on the performance of the training. I am obtaining some strange output when using the NetMeasurements built-in function. I would expect some real numbers between 0 and 1, but instead I get an accuracy of 1 (doesn't seem plausible) and then some numbers and arrows. Can anyone advise on the reason for this, many thanks.
In the code I start by providing the training and validation data sets, I then train it and finally I attempt to analyze the validation set using NetMeasurements:
(*Training and Validation Data*)
TrainingData = {{9994, 6} -> 1, {9964, 36} -> 2, {9905, 95} ->
3, {9852, 148} -> 4, {9735, 265} -> 5, {9619, 381} ->
6, {9541, 459} -> 7, {9356, 644} -> 8, {9219, 781} ->
9, {9063, 937} -> 10, {8879, 1121} -> 11, {8644, 1356} ->
12, {8373, 1627} -> 13, {8144, 1856} -> 14, {8001, 1999} ->
15, {7690, 2310} -> 16, {7506, 2494} -> 17, {7152, 2848} ->
18, {6864, 3136} -> 19, {6634, 3366} -> 20, {6219, 3781} ->
21, {5975, 4025} -> 22, {5598, 4402} -> 23, {5331, 4669} ->
24, {4972, 5028} -> 25, {4693, 5307} -> 26, {4433, 5567} ->
27, {4019, 5981} -> 28, {3806, 6194} -> 29, {3398, 6602} ->
30, {3131, 6869} -> 31, {2888, 7112} -> 32, {2644, 7356} ->
33, {2311, 7689} -> 34, {2012, 7988} -> 35, {1778, 8222} ->
36, {1544, 8456} -> 37, {1327, 8673} -> 38, {1151, 8849} ->
39, {963, 9037} -> 40, {774, 9226} -> 41, {590, 9410} ->
42, {479, 9521} -> 43, {360, 9640} -> 44, {265, 9735} ->
45, {160, 9840} -> 46, {80, 9920} -> 47, {53, 9947} ->
48, {12, 9988} -> 49, {0, 10000} -> 50};
ValidationData = {{9991, 9} -> 1, {9959, 41} -> 2, {9902, 98} ->
3, {9849, 151} -> 4, {9764, 236} -> 5, {9623, 377} ->
6, {9497, 503} -> 7, {9342, 658} -> 8, {9213, 787} ->
9, {9073, 927} -> 10, {8826, 1174} -> 11, {8585, 1415} ->
12, {8420, 1580} -> 13, {8166, 1834} -> 14, {7926, 2074} ->
15, {7708, 2292} -> 16, {7508, 2492} -> 17, {7064, 2936} ->
18, {6758, 3242} -> 19, {6577, 3423} -> 20, {6304, 3696} ->
21, {5872, 4128} -> 22, {5661, 4339} -> 23, {5228, 4772} ->
24, {4936, 5064} -> 25, {4703, 5297} -> 26, {4387, 5613} ->
27, {4061, 5939} -> 28, {3786, 6214} -> 29, {3354, 6646} ->
30, {3227, 6773} -> 31, {2877, 7123} -> 32, {2661, 7339} ->
33, {2360, 7640} -> 34, {2033, 7967} -> 35, {1828, 8172} ->
36, {1552, 8448} -> 37, {1372, 8628} -> 38, {1138, 8862} ->
39, {958, 9042} -> 40, {770, 9230} -> 41, {612, 9388} ->
42, {479, 9521} -> 43, {355, 9645} -> 44, {259, 9741} ->
45, {160, 9840} -> 46, {81, 9919} -> 47, {36, 9964} ->
48, {14, 9986} -> 49, {0, 10000} -> 50};
(*Neural network architecture*)
net = NetChain[{LinearLayer[300], BatchNormalizationLayer[],
ElementwiseLayer["ReLU"], LinearLayer[300],
BatchNormalizationLayer[], ElementwiseLayer["ReLU"],
LinearLayer[50], SoftmaxLayer[] } ];
(*Training the network*)
trainedNet =
NetTrain[net, TrainingData, ValidationSet -> ValidationData,
BatchSize -> Automatic, MaxTrainingRounds -> 10000,
LearningRate -> 0.01]
(*Measurements of Validation Set*)
measurements =
NetMeasurements[trainedNet,
ValidationData, {"Accuracy", "Precision", "Recall", "F1Score"}]
{1., <|1 -> 1., 2 -> 1., 3 -> 1., 4 -> 1., 5 -> 1., 6 -> 1., 7 -> 1.,
8 -> 1., 9 -> 1., 10 -> 1., 11 -> 1., 12 -> 1., 13 -> 1., 14 -> 1.,
15 -> 1., 16 -> 1., 17 -> 1., 18 -> 1., 19 -> 1., 20 -> 1.,
21 -> 1., 22 -> 1., 23 -> 1., 24 -> 1., 25 -> 1., 26 -> 1.,
27 -> 1., 28 -> 1., 29 -> 1., 30 -> 1., 31 -> 1., 32 -> 1.,
33 -> 1., 34 -> 1., 35 -> 1., 36 -> 1., 37 -> 1., 38 -> 1.,
39 -> 1., 40 -> 1., 41 -> 1., 42 -> 1., 43 -> 1., 44 -> 1.,
45 -> 1., 46 -> 1., 47 -> 1., 48 -> 1., 49 -> 1.,
50 -> 1.|>, <|1 -> 1., 2 -> 1., 3 -> 1., 4 -> 1., 5 -> 1., 6 -> 1.,
7 -> 1., 8 -> 1., 9 -> 1., 10 -> 1., 11 -> 1., 12 -> 1., 13 -> 1.,
14 -> 1., 15 -> 1., 16 -> 1., 17 -> 1., 18 -> 1., 19 -> 1.,
20 -> 1., 21 -> 1., 22 -> 1., 23 -> 1., 24 -> 1., 25 -> 1.,
26 -> 1., 27 -> 1., 28 -> 1., 29 -> 1., 30 -> 1., 31 -> 1.,
32 -> 1., 33 -> 1., 34 -> 1., 35 -> 1., 36 -> 1., 37 -> 1.,
38 -> 1., 39 -> 1., 40 -> 1., 41 -> 1., 42 -> 1., 43 -> 1.,
44 -> 1., 45 -> 1., 46 -> 1., 47 -> 1., 48 -> 1., 49 -> 1.,
50 -> 1.|>, <|1 -> 1., 2 -> 1., 3 -> 1., 4 -> 1., 5 -> 1., 6 -> 1.,
7 -> 1., 8 -> 1., 9 -> 1., 10 -> 1., 11 -> 1., 12 -> 1., 13 -> 1.,
14 -> 1., 15 -> 1., 16 -> 1., 17 -> 1., 18 -> 1., 19 -> 1.,
20 -> 1., 21 -> 1., 22 -> 1., 23 -> 1., 24 -> 1., 25 -> 1.,
26 -> 1., 27 -> 1., 28 -> 1., 29 -> 1., 30 -> 1., 31 -> 1.,
32 -> 1., 33 -> 1., 34 -> 1., 35 -> 1., 36 -> 1., 37 -> 1.,
38 -> 1., 39 -> 1., 40 -> 1., 41 -> 1., 42 -> 1., 43 -> 1.,
44 -> 1., 45 -> 1., 46 -> 1., 47 -> 1., 48 -> 1., 49 -> 1.,
50 -> 1.|>}
NetDecoderto your softmax layer that returns the position of the largest (most probable class) element, you'll see theValidationSetis perfectly reconstructed. I.e. in your definition ofnetreplaceSoftmaxLayer[]withSoftmaxLayer["Output" -> dec]withdec = NetDecoder[{"Class", Range[50]}];$\endgroup$