# NetEncoders vs MXNet preprocessing

According to this great answer, the MXNet framework powers Mathematica's new neural net functions. So I wanted to check and make sure the results agree for a pretrained network (available here: https://github.com/dmlc/mxnet-model-gallery).

I loaded the pretrained networks from MXNet:

net = NeuralNetworksImportMXNetModel[
"~/InceptionV3/Inception-7-symbol.json",
"~/InceptionV3/Inception-7-0001.params"];


However, after comparing the outputs from both MXNet's python example code and Mathematica - I found that the softmax-layer outputs were the same, but that all the others were subtly different. Eventually, I figured out that the encoders don't yield the same values as the preprocessing script.

Here's how the python preprocessing script works:

Clear@mxPreprocess;
mxPreprocess[img_]:=Module[{shape,xx,yy,se,crop,re,d},
shape=ImageDimensions@img;
se=Min@shape;
yy=IntegerPart[(shape[[1]]-se)/2];
xx=IntegerPart[(shape[[2]]-se)/2];
crop=ImageTake[img,{xx,xx+se},{yy,yy+se}];
re=ImageResize[crop,{299,299}];
d=ImageData[re]-0.5;
d=d/0.5;
Return @ d
]


And here's how the Mathematica "preprocessing script" works:

enc = NetEncoder[{"Image", {299, 299}, "MeanImage" -> 0.5}]


And they are not the same operations, here's the difference:

I'd like to be able to reproduce the same encodings that the real MXNet program does (because that's presumably how it was trained), is there any way to do this?

## 1 Answer

Removing two lines from your code yields identical results:

Clear@mxPreprocess;
mxPreprocess[img_] :=
Module[{shape, xx, yy, se, crop, re, d},
shape = ImageDimensions@img;
se = Min@shape;
yy = IntegerPart[(shape[[1]] - se)/2];
xx = IntegerPart[(shape[[2]] - se)/2];
(* crop = ImageTake[img, {xx, xx + se}, {yy, yy + se}]; *)
re = ImageResize[(*crop*) img, {299, 299}] // Echo;
d = ImageData[re] - 0.5;
(* d = d/0.5 *);
Return@d]

img = ExampleData[{"TestImage", "Man"}]


  d = mxPreprocess@img // Image


dd = Transpose[enc@img, {3, 1, 2}]//Image


(note that the Transpose is necessary here to convert from three color planes to one image plane with color triplets)

ImageDifference[d, dd]


(ImageData[d] - ImageData[dd]) // MinMax
(* {-2.96855*10^-8, 0.} *)
`
• But that line of cropping code is in preprocessing.py for a reason: github.com/dmlc/mxnet-model-gallery/blob/master/… One can't simply ignore it. – user5601 Aug 30 '16 at 2:38
• I understanding that, and by identifying the lines in the code that NetEncoder doesn't seem to use one could see whether it is possible to modify it through its input or output to yield the same result. The last step is easy, but it looks like NetEncoder rescales the whole image and not a cropped version as mxPreProcess does. This means you have to put a padding around your input image to get the same result. That looks like much effort without any gain. Why don't you simply use mxPreProcess itself? – Sjoerd C. de Vries Aug 30 '16 at 6:42