Associations & generating training sets

Question

Solution: Pi is allowed in training sets, but not in batches generated by the batch generating function preamble:

I want to generate my own batches of training data for training a deep CNN.

ref/NetTrain says:

In NetTrain[net,f], the function f is applied to <|"BatchSize"->n,"Round"->r|> to generate each batch of training data in the form {Subscript[input, 1]->Subscript[output, 1],[Ellipsis]} or <|"Subscript[port, 1]"->data,[Ellipsis]|>.

How do i need to structure said f? I feel like i tried nothing (only Lego code) and I'm all out of ideas.

Question:

this code works, if and only if 64 is the batchsize. My question is how to generalize it st. it uses proper BatchSize and Round.

net = NetChain[{Ramp, 5, 1}, "Input" -> 64, "Output" -> 1]
net = NetInitialize[net]
tNet = NetTrain[
  net, 
  {
   ((RandomInteger[] & /@ Range[64 (*inputLength*)]) -> {1} & /@ 
      Range[64 (*Batchsize*)]) &, "RoundLength" -> 100000
  }, 
  All]

i tried replacing the batchsize 64 with BatchSize and n but it broke the code.

I think i didn't understand associations properly and have no idea how to extract said information. The wiki article is just confusing at this point since i have no idea where to access the association in this function f.

Solution: include parameters via #name in the function and append & or use Function[...]

net = NetChain[{Ramp, 5, 1}, "Input" -> 64, "Output" -> 1]
net = NetInitialize[net]
tNet = NetTrain[
  net, 
  {
   ((RandomInteger[] & /@ Range[64 (*inputLength*)]) -> {1} & /@ 
      Range[64 (*Batchsize*)]) &, "RoundLength" -> 100000
  }, 
  All]

But: A "Bug" (user error) arises when using functions which include Block.

net = NetChain[{Ramp, 5, 1}, "Input" -> 64, "Output" -> 1]
net = NetInitialize[net]
tNet = NetTrain[
  net, 
  Block[{},{
   ((RandomInteger[] & /@ Range[64 (*inputLength*)]) -> {1} & /@ 
      Range[64 (*Batchsize*)])] &, "RoundLength" -> 100000
  }, 
  All]

stating that the function would produce invalide data.

Answer 1

I think fTrain and/or fTrain2 in the code below are what you are looking for.

(Using real data in order to see that the thing is actually working...)

mnistData = 
  ExampleData[{"MachineLearning", "MNIST"}, "TrainingData"];

mnistTestData = 
  ExampleData[{"MachineLearning", "MNIST"}, "TrainingData"];

SeedRandom[232]
rDataBlocks = 
  Partition[RandomSample[mnistData], Floor[Length[mnistData]/300]];
MinMax[Length /@ rDataBlocks]   
(* {200, 200} *)

ClearAll[fTrain]
fTrain = Function[RandomSample[mnistData, #BatchSize]];

fTrain[<|"BatchSize" -> 12|>]

ClearAll[fTrain2]
fTrain2 = Function[RandomSample[rDataBlocks[[Mod[#Round, 300] + 1]], #BatchSize]];

fTrain2[<|"Round" -> 2, "BatchSize" -> 12|>]

lenet = NetChain[{
   ConvolutionLayer[20, 5], Ramp, PoolingLayer[2, 2],
   ConvolutionLayer[50, 5], Ramp, PoolingLayer[2, 2],
   FlattenLayer[], 500, Ramp, 10, SoftmaxLayer[]},
  "Output" -> NetDecoder[{"Class", Range[0, 9]}],
  "Input" -> NetEncoder[{"Image", {28, 28}, "Grayscale"}]
  ]

(* tNet = NetTrain[lenet, fTrain, MaxTrainingRounds -> 100] *)
tNet = NetTrain[lenet, fTrain2, MaxTrainingRounds -> 100]

Block[{t = RandomSample[mnistTestData, 100]},
   Tally[Equal @@@ Transpose[{tNet[t[[All, 1]]], t[[All, 2]]}]]
]

(* {{True, 95}, {False, 5}} *)