8
$\begingroup$

Python has generators which save memory, is there a technique for generating in memory examples for your training set "on the fly".

For example purposes, I constructed here a regressor for blur:

randomMask[img_] := 
 Module[{t, h, g, d = ImageDimensions[img]}, 
  t = Table[{PointSize@RandomReal[{0, .1}], 
     RandomChoice[{Point, 
        Rectangle[#, # + RandomReal[{-200, 200}, {2}]] &}]@
      RandomPoint[Rectangle[{0, 0}, d]]}, {RandomChoice[{0, 1, 2, 3, 
       4, 8, 14, 20, 50, 200}]}];
  g = Graphics[t, PlotRange -> Transpose[{{0, 0}, d}], ImageSize -> d];
  {g, Area@DiscretizeGraphics@g/Times @@ d}]

makeExample[img_] := Module[{g, v},
   {g, v} = randomMask[img];
   ImageCompose[img, SetAlphaChannel[Blur[img, 15], ColorNegate@g]] ->
     v
   ];

imgs = ConformImages[ExampleData /@ ExampleData["TestImage"], {100, 100}];

(* this is a large set that I don't want to precompute !!! *)
train = Table[makeExample@RandomChoice[imgs], {3000}]
test = Table[makeExample@RandomChoice[imgs], {500}];

convnet=NetChain[{
ConvolutionLayer[20,{5,5}],
ElementwiseLayer[Ramp],
PoolingLayer[{2,2},{2,2}],
ConvolutionLayer[50,{5,5}],
ElementwiseLayer[Ramp],
PoolingLayer[{2,2},{2,2}],
FlattenLayer[],
DotPlusLayer[500],
ElementwiseLayer[Ramp],
DotPlusLayer[50],
ElementwiseLayer[Ramp],
DotPlusLayer[1]
},
"Input"->NetEncoder[{"Image",{100,100}}],
"Output"->NetDecoder["Scalar"]
]

trainedConvnet = NetTrain[convnet, train, TargetDevice -> "GPU"]
output = trainedConvnet /@ Keys[test];

target = test // Values;
meanSquareLoss = Mean@Flatten[(#Output - #Target)^2, Infinity] &;

data = <|"Output" -> {{output}}, "Target" -> {{target}}|>;
N@meanSquareLoss@data
$\endgroup$
11
$\begingroup$

It's not very well tested but you can supply a "$AugmentationFunction" -> f option to the Image NetEncoder in which you can put a Blur or whatever (anything that takes an image and produces an image). This option is not officially supported and it'll probably be replaced with something superior in future.

EDIT: we plan on supporting the ability to generate batches via a callback function in 11.1. The training data spec will look something like {f, n}, where f is your callback function (which should take a batchsize and produce an association mapping port to input data) and n is the number of examples that should count as a training round.

SECOND EDIT: this functionality has landed in 11.1 builds, so you can now supply an arbitrary function to produce batches of data on demand. You might want to sign up to be a beta tester.

$\endgroup$
  • $\begingroup$ Where does one sign up for beta testing? I would be really interested in trying out the new neural network stuff $\endgroup$ – Sascha Oct 3 '16 at 20:43
  • $\begingroup$ @Sascha I'm not actually sure. Try contacting customer support and asking them. $\endgroup$ – Taliesin Beynon Oct 3 '16 at 21:30
  • $\begingroup$ @Sascha Ok I can get you on the list. If you want to email me at my first name and first letter of my last name at wolfram.com and you'll be added to to the beta program, which will be going live for 11.1 very soon. $\endgroup$ – Taliesin Beynon Oct 4 '16 at 16:00
10
$\begingroup$

You can do out-of-core classification with the new function File (link1, link2).

I will simplify your code. For example, we have directory 'train' with 100 images.

CreateDirectory["train"];
Do[
 Export[
  "train\\" <> ToString[i] <> ".jpg",
  RandomImage[1, {100, 100}, ColorSpace -> "RGB"]
  ],
 {i, 100}
 ]

Let's compare the calculation speed of out-of-core File and classic Import.

SetDirectory["train"];

X1 = File /@ FileNames[];
X2 = Import /@ FileNames[];

Y = RandomInteger[1, 100];

Convolutional neural network:

convnet = NetChain[
   {
    ConvolutionLayer[20, {5, 5}],
    ElementwiseLayer[Ramp],
    PoolingLayer[{2, 2}, {2, 2}],
    ConvolutionLayer[50, {5, 5}],
    ElementwiseLayer[Ramp],
    PoolingLayer[{2, 2}, {2, 2}],
    FlattenLayer[],
    DotPlusLayer[500],
    ElementwiseLayer[Ramp],
    DotPlusLayer[50],
    ElementwiseLayer[Ramp],
    DotPlusLayer[1]
    },
   "Input" -> NetEncoder[{"Image", {100, 100}}],
   "Output" -> NetDecoder["Scalar"]
   ];
SeedRandom[123];
AbsoluteTiming[
 net1 = NetTrain[convnet, X1 -> Y, BatchSize -> 16, MaxTrainingRounds -> 1];
 ]

{5.79041, Null}

SeedRandom[123];
AbsoluteTiming[
 net2 = NetTrain[convnet, X2 -> Y, BatchSize -> 16, MaxTrainingRounds -> 1];
 ]

{5.54343, Null}

As we can see, the difference in the speed of calculations is very small.

But of course this is not an online augmentation of the dataset.

'In-the-storage' augmentation with function ImageFileApply:

augmentingFunctions = {# &, 1 - # &};
numberOfRounds = 3;

SeedRandom[123];
Do[
 Xaugm = File /@ (ImageFileApply[RandomChoice[augmentingFunctions], #] & /@ X1);
 net1 = NetTrain[convnet, Xaugm -> Y, BatchSize -> 16, MaxTrainingRounds -> 1];
 DeleteFile@FileNames["* at *.jpeg"],
 {numberOfRounds}
 ]
$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.