1
$\begingroup$

I would like to export the round loss during my model training for each training round. Currently, I am using the way shown in the Documentation Center around TrainingProgressFunction and PutAppend:

logFile = CreateTemporary[];
appendToLog = PutAppend[<|"Loss" -> #RoundLoss|>, logFile] &;
(*I have skipped the "Batch" -> #AbsoluteBatch output*)
trainingData = {#1, #2} -> {#1 + Sin[#1*#2]} & @@@ 
RandomReal[{-1, 1}, {50000, 2}];
NetTrain[
NetChain[{1000, Tanh, 1}, "Input" -> 2],trainingData, MaxTrainingRounds->10, 
TrainingProgressFunction -> appendToLog];
(*I am training for 10 rounds*)

Despite the fact that I am training for 10 rounds, the log file consists of 9 entries, the last round loss is omitted

logDataLength = Length[ReadList[logFile]]
9

Is there any alternative way to include also the last output? I can create a function appending the last output to the already existing list, but if I want to export for each and every round the weights and the biases of a large neural network (e.g. in total 15,000) this becomes quiet cumbersome.

Thanks!

$\endgroup$
  • $\begingroup$ I haven't tried this, but perhaps TrainingProgressFunction -> {appendToLog, "Interval"->Quantity[1,"Rounds"] or more likely TrainingProgressFunction -> {appendToLog, "Interval"->Quantity[.999,"Rounds"] would work? $\endgroup$ – Carl Lange Jan 7 at 11:40
  • $\begingroup$ TrainingProgressFunction -> {appendToLog, "Interval" -> Quantity[1, "Rounds"]} appears to work correctly. $\endgroup$ – Carl Lange Jan 7 at 11:43
  • $\begingroup$ @Carl, thanks a lot! However, I still get only 9 entries instead of 10 (11.1.1 for Mac OS X x86 (64-bit)). If I use "Interval"->Quantity[.999,"Rounds"] then the first entry, for the first iteration, is None. $\endgroup$ – demm Jan 7 at 13:46
  • $\begingroup$ I ended up having better luck with Quantity[.99,"Rounds"], but that approach kind of sucks anyway. I'm on 11.3 with no issue for Quantity[1,"Rounds"]. Potentially you could set BatchSize and do Quantity[batchsize-1,"Batches"]? $\endgroup$ – Carl Lange Jan 7 at 13:51
  • $\begingroup$ Another way to do this (only available after training) is using the third argument to NetTrain. nto = NetTrain[NetChain[{1000, Tanh, 1}, "Input" -> 2], trainingData, All, MaxTrainingRounds -> 10] and then nto["RoundLossList"]. $\endgroup$ – Carl Lange Jan 7 at 13:53

Your Answer

By clicking "Post Your Answer", you acknowledge that you have read our updated terms of service, privacy policy and cookie policy, and that your continued use of the website is subject to these policies.

Browse other questions tagged or ask your own question.