Training data generator?

Question

Python has generators which save memory, is there a technique for generating in memory examples for your training set "on the fly".

For example purposes, I constructed here a regressor for blur:

randomMask[img_] := 
 Module[{t, h, g, d = ImageDimensions[img]}, 
  t = Table[{PointSize@RandomReal[{0, .1}], 
     RandomChoice[{Point, 
        Rectangle[#, # + RandomReal[{-200, 200}, {2}]] &}]@
      RandomPoint[Rectangle[{0, 0}, d]]}, {RandomChoice[{0, 1, 2, 3, 
       4, 8, 14, 20, 50, 200}]}];
  g = Graphics[t, PlotRange -> Transpose[{{0, 0}, d}], ImageSize -> d];
  {g, Area@DiscretizeGraphics@g/Times @@ d}]

makeExample[img_] := Module[{g, v},
   {g, v} = randomMask[img];
   ImageCompose[img, SetAlphaChannel[Blur[img, 15], ColorNegate@g]] ->
     v
   ];

imgs = ConformImages[ExampleData /@ ExampleData["TestImage"], {100, 100}];

(* this is a large set that I don't want to precompute !!! *)
train = Table[makeExample@RandomChoice[imgs], {3000}]
test = Table[makeExample@RandomChoice[imgs], {500}];

convnet=NetChain[{
ConvolutionLayer[20,{5,5}],
ElementwiseLayer[Ramp],
PoolingLayer[{2,2},{2,2}],
ConvolutionLayer[50,{5,5}],
ElementwiseLayer[Ramp],
PoolingLayer[{2,2},{2,2}],
FlattenLayer[],
DotPlusLayer[500],
ElementwiseLayer[Ramp],
DotPlusLayer[50],
ElementwiseLayer[Ramp],
DotPlusLayer[1]
},
"Input"->NetEncoder[{"Image",{100,100}}],
"Output"->NetDecoder["Scalar"]
]

trainedConvnet = NetTrain[convnet, train, TargetDevice -> "GPU"]
output = trainedConvnet /@ Keys[test];

target = test // Values;
meanSquareLoss = Mean@Flatten[(#Output - #Target)^2, Infinity] &;

data = <|"Output" -> {{output}}, "Target" -> {{target}}|>;
N@meanSquareLoss@data

Answer 1

It's not very well tested but you can supply a "$AugmentationFunction" -> f option to the Image NetEncoder in which you can put a Blur or whatever (anything that takes an image and produces an image). This option is not officially supported and it'll probably be replaced with something superior in future.

EDIT: we plan on supporting the ability to generate batches via a callback function in 11.1. The training data spec will look something like {f, n}, where f is your callback function (which should take a batchsize and produce an association mapping port to input data) and n is the number of examples that should count as a training round.

SECOND EDIT: this functionality has landed in 11.1 builds, so you can now supply an arbitrary function to produce batches of data on demand. You might want to sign up to be a beta tester.

Answer 2

You can do out-of-core classification with the new function File (link1, link2).

I will simplify your code. For example, we have directory 'train' with 100 images.

CreateDirectory["train"];
Do[
 Export[
  "train\\" <> ToString[i] <> ".jpg",
  RandomImage[1, {100, 100}, ColorSpace -> "RGB"]
  ],
 {i, 100}
 ]

Let's compare the calculation speed of out-of-core File and classic Import.

SetDirectory["train"];

X1 = File /@ FileNames[];
X2 = Import /@ FileNames[];

Y = RandomInteger[1, 100];

Convolutional neural network:

convnet = NetChain[
   {
    ConvolutionLayer[20, {5, 5}],
    ElementwiseLayer[Ramp],
    PoolingLayer[{2, 2}, {2, 2}],
    ConvolutionLayer[50, {5, 5}],
    ElementwiseLayer[Ramp],
    PoolingLayer[{2, 2}, {2, 2}],
    FlattenLayer[],
    DotPlusLayer[500],
    ElementwiseLayer[Ramp],
    DotPlusLayer[50],
    ElementwiseLayer[Ramp],
    DotPlusLayer[1]
    },
   "Input" -> NetEncoder[{"Image", {100, 100}}],
   "Output" -> NetDecoder["Scalar"]
   ];

SeedRandom[123];
AbsoluteTiming[
 net1 = NetTrain[convnet, X1 -> Y, BatchSize -> 16, MaxTrainingRounds -> 1];
 ]

{5.79041, Null}

SeedRandom[123];
AbsoluteTiming[
 net2 = NetTrain[convnet, X2 -> Y, BatchSize -> 16, MaxTrainingRounds -> 1];
 ]

{5.54343, Null}

As we can see, the difference in the speed of calculations is very small.

But of course this is not an online augmentation of the dataset.

'In-the-storage' augmentation with function ImageFileApply:

augmentingFunctions = {# &, 1 - # &};
numberOfRounds = 3;

SeedRandom[123];
Do[
 Xaugm = File /@ (ImageFileApply[RandomChoice[augmentingFunctions], #] & /@ X1);
 net1 = NetTrain[convnet, Xaugm -> Y, BatchSize -> 16, MaxTrainingRounds -> 1];
 DeleteFile@FileNames["* at *.jpeg"],
 {numberOfRounds}
 ]