3
$\begingroup$

Solution: Pi is allowed in training sets, but not in batches generated by the batch generating function preamble:

I want to generate my own batches of training data for training a deep CNN.

ref/NetTrain says:

In NetTrain[net,f], the function f is applied to <|"BatchSize"->n,"Round"->r|> to generate each batch of training data in the form {Subscript[input, 1]->Subscript[output, 1],[Ellipsis]} or <|"Subscript[port, 1]"->data,[Ellipsis]|>.

How do i need to structure said f? I feel like i tried nothing (only Lego code) and I'm all out of ideas.

Question:

this code works, if and only if 64 is the batchsize. My question is how to generalize it st. it uses proper BatchSize and Round.

net = NetChain[{Ramp, 5, 1}, "Input" -> 64, "Output" -> 1]
net = NetInitialize[net]
tNet = NetTrain[
  net, 
  {
   ((RandomInteger[] & /@ Range[64 (*inputLength*)]) -> {1} & /@ 
      Range[64 (*Batchsize*)]) &, "RoundLength" -> 100000
  }, 
  All]

i tried replacing the batchsize 64 with BatchSize and n but it broke the code.

I think i didn't understand associations properly and have no idea how to extract said information. The wiki article is just confusing at this point since i have no idea where to access the association in this function f.

Solution: include parameters via #name in the function and append & or use Function[...]

net = NetChain[{Ramp, 5, 1}, "Input" -> 64, "Output" -> 1]
net = NetInitialize[net]
tNet = NetTrain[
  net, 
  {
   ((RandomInteger[] & /@ Range[64 (*inputLength*)]) -> {1} & /@ 
      Range[64 (*Batchsize*)]) &, "RoundLength" -> 100000
  }, 
  All]

But: A "Bug" (user error) arises when using functions which include Block.

net = NetChain[{Ramp, 5, 1}, "Input" -> 64, "Output" -> 1]
net = NetInitialize[net]
tNet = NetTrain[
  net, 
  Block[{},{
   ((RandomInteger[] & /@ Range[64 (*inputLength*)]) -> {1} & /@ 
      Range[64 (*Batchsize*)])] &, "RoundLength" -> 100000
  }, 
  All]

stating that the function would produce invalide data.

$\endgroup$
2
$\begingroup$

I think fTrain and/or fTrain2 in the code below are what you are looking for.

(Using real data in order to see that the thing is actually working...)

mnistData = 
  ExampleData[{"MachineLearning", "MNIST"}, "TrainingData"];

mnistTestData = 
  ExampleData[{"MachineLearning", "MNIST"}, "TrainingData"];

SeedRandom[232]
rDataBlocks = 
  Partition[RandomSample[mnistData], Floor[Length[mnistData]/300]];
MinMax[Length /@ rDataBlocks]   
(* {200, 200} *)

ClearAll[fTrain]
fTrain = Function[RandomSample[mnistData, #BatchSize]];

fTrain[<|"BatchSize" -> 12|>]

ClearAll[fTrain2]
fTrain2 = Function[RandomSample[rDataBlocks[[Mod[#Round, 300] + 1]], #BatchSize]];

fTrain2[<|"Round" -> 2, "BatchSize" -> 12|>]

lenet = NetChain[{
   ConvolutionLayer[20, 5], Ramp, PoolingLayer[2, 2],
   ConvolutionLayer[50, 5], Ramp, PoolingLayer[2, 2],
   FlattenLayer[], 500, Ramp, 10, SoftmaxLayer[]},
  "Output" -> NetDecoder[{"Class", Range[0, 9]}],
  "Input" -> NetEncoder[{"Image", {28, 28}, "Grayscale"}]
  ]

(* tNet = NetTrain[lenet, fTrain, MaxTrainingRounds -> 100] *)
tNet = NetTrain[lenet, fTrain2, MaxTrainingRounds -> 100]

Block[{t = RandomSample[mnistTestData, 100]},
   Tally[Equal @@@ Transpose[{tNet[t[[All, 1]]], t[[All, 2]]}]]
]

(* {{True, 95}, {False, 5}} *)
$\endgroup$
  • $\begingroup$ Thank you. I think i understood associations now. Do you know why the code breaks when the function is included in a Block or Module? $\endgroup$ – Gladaed May 29 '18 at 6:52
  • $\begingroup$ @Gladaed You have syntax error(s) in the code. This works: tNet = NetTrain[ net, {Block[{}, ((RandomInteger[] & /@ Range[64 (*inputLength*)]) -> {1} & /@ Range[64 (*Batchsize*)])] &, "RoundLength" -> 100000} , All] $\endgroup$ – Anton Antonov May 29 '18 at 7:54
  • $\begingroup$ just noticed it. Still got this problem in my original code which i don't know how to fix (function gives good dataset if called manually via [<|"BatchSize"->integer|>] but NetTrain complains about it not being able to generate proper dataset $\endgroup$ – Gladaed May 29 '18 at 7:58
  • $\begingroup$ generateDataBatch := Block[{trainingBatch}, n = nMax + 1; trainingBatch = If[n <= l - #BatchSize, nMax = nMax + #BatchSize; trainingSet[[n ;; nMax]], nMax = 0; trainingSet[[n ;;]] ]; trainingBatch ] &; is the troublemaker $\endgroup$ – Gladaed May 29 '18 at 8:00
  • $\begingroup$ @Gladaed Did you figure out the problem? I can look further into this in a day or two... $\endgroup$ – Anton Antonov May 31 '18 at 14:19

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.