3
$\begingroup$

Typically when you train a model you have three sets of data: training, validation, and testing - and they can’t be mixed, i.e. you can’t use images from one in the other.

I want to know if the samples that you give NetTrain through the option ValidationSet are used to effect the parameters of the network? Or is it considered a holdout that I can test on later, or is that data burnt after NetTrain completes?

For example, given that testData is already used in NetTrain:

trainingData = ResourceData["MNIST", "TrainingData"];
testData = ResourceData["MNIST", "TestData"];
lenetModel = NetModel["LeNet Trained on MNIST Data"]
n = NetTrain[NetModel["LeNet"], trainingData, ValidationSet -> testData]

Is this an ill-informed thing to do:

ClassifierMeasurements[n, testData]
$\endgroup$
2
$\begingroup$

Yes, you can do ClassifierMeasurements on the testData.

http://reference.wolfram.com/language/tutorial/NeuralNetworksRegularization.html

enter image description here

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.