2
$\begingroup$

I am trying to set up transfer learning using instructions from this repository page on EfficientNet. I am able to perform a stand-alone training and testing using the instructions on that page. The next step is to use cross-validate model from the resource functions page to evaluate the range of performance on my dataset.

The documentation for CVModel provides an example using NetTrain; however, it uses TimeGoal as stopping criterion during cross-valiation.

The relevant code setup based on instructions from that page:

cvEfficientNets = ResourceFunction["CrossValidateModel",Flatten[trainData],
Function[NetTrain[net, #, All, TimeGoal -> 10, 
TargetDevice -> {"GPU", All}]]

I wonder if that's appropriate if I am trying to compare performance of two different NNs for the same dataset, eg., ResNet vs. EfficientNet. What would be reasonable alternatives to stopping the training during cross-validation stage?

I tried TrainingStoppingCriterion->"Loss";however, WL complains that no validation set has been specified and defaults to using the training set for validation. I think that's probably not correct. If I specify ValidationSet->Scaled[0.1] as is usually done, I am uncertain if that's necessary. K-Fold validation should already have a partition selected for validation.

So, any advice is highly appreciated.

Improve this question
$\endgroup$

1 Answer 1

Reset to default
0
$\begingroup$

I think setting up MaxTrainingRounds to the same number should be a reasonable basis to compare the performance of different NNs during cross-validation stage.

Improve this answer
$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.