7
$\begingroup$

According to this great answer, the MXNet framework powers Mathematica's new neural net functions. So I wanted to check and make sure the results agree for a pretrained network (available here: https://github.com/dmlc/mxnet-model-gallery).

I loaded the pretrained networks from MXNet:

net = NeuralNetworks`ImportMXNetModel[
  "~/InceptionV3/Inception-7-symbol.json",
  "~/InceptionV3/Inception-7-0001.params"];

However, after comparing the outputs from both MXNet's python example code and Mathematica - I found that the softmax-layer outputs were the same, but that all the others were subtly different. Eventually, I figured out that the encoders don't yield the same values as the preprocessing script.

Here's how the python preprocessing script works:

Clear@mxPreprocess;
mxPreprocess[img_]:=Module[{shape,xx,yy,se,crop,re,d},
    shape=ImageDimensions@img;
    se=Min@shape;
    yy=IntegerPart[(shape[[1]]-se)/2];
    xx=IntegerPart[(shape[[2]]-se)/2];
    crop=ImageTake[img,{xx,xx+se},{yy,yy+se}];
    re=ImageResize[crop,{299,299}];
    d=ImageData[re]-0.5;
    d=d/0.5;
    Return @ d
]

And here's how the Mathematica "preprocessing script" works:

enc = NetEncoder[{"Image", {299, 299}, "MeanImage" -> 0.5}]

And they are not the same operations, here's the difference:

enter image description here

I'd like to be able to reproduce the same encodings that the real MXNet program does (because that's presumably how it was trained), is there any way to do this?

$\endgroup$
4
$\begingroup$

Removing two lines from your code yields identical results:

Clear@mxPreprocess;
mxPreprocess[img_] := 
 Module[{shape, xx, yy, se, crop, re, d}, 
  shape = ImageDimensions@img;
  se = Min@shape;
  yy = IntegerPart[(shape[[1]] - se)/2];
  xx = IntegerPart[(shape[[2]] - se)/2];
  (* crop = ImageTake[img, {xx, xx + se}, {yy, yy + se}]; *)
  re = ImageResize[(*crop*) img, {299, 299}] // Echo;
  d = ImageData[re] - 0.5;
  (* d = d/0.5 *);
  Return@d]

img = ExampleData[{"TestImage", "Man"}]

Mathematica graphics

  d = mxPreprocess@img // Image

Mathematica graphics

dd = Transpose[enc@img, {3, 1, 2}]//Image

Mathematica graphics

(note that the Transpose is necessary here to convert from three color planes to one image plane with color triplets)

ImageDifference[d, dd]

Mathematica graphics

(ImageData[d] - ImageData[dd]) // MinMax
(* {-2.96855*10^-8, 0.} *)
$\endgroup$
  • $\begingroup$ But that line of cropping code is in preprocessing.py for a reason: github.com/dmlc/mxnet-model-gallery/blob/master/… One can't simply ignore it. $\endgroup$ – user5601 Aug 30 '16 at 2:38
  • 1
    $\begingroup$ I understanding that, and by identifying the lines in the code that NetEncoder doesn't seem to use one could see whether it is possible to modify it through its input or output to yield the same result. The last step is easy, but it looks like NetEncoder rescales the whole image and not a cropped version as mxPreProcess does. This means you have to put a padding around your input image to get the same result. That looks like much effort without any gain. Why don't you simply use mxPreProcess itself? $\endgroup$ – Sjoerd C. de Vries Aug 30 '16 at 6:42

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.