7
$\begingroup$

Mathematica Neural Networks based on MXNet. And there is lots of model in MXNet(MXNet Model Zoo).

For example,I follow a tutorial An introduction to the MXNet API — part 4, It uses Inception v3(net using MXNet).

I can use this code to load this pretrained model

<< NeuralNetworks`;
net = ImportMXNetModel["F:\\model\\Inception-BN-symbol.json", 
                       "F:\\model\\Inception-BN-0126.params"];
labels = Import["http://data.dmlc.ml/models/imagenet/synset.txt", "List"];
incepNet = NetChain[{ElementwiseLayer[#*255 &], net}, 
            "Input" -> NetEncoder[{"Image", {224, 224}, ColorSpace -> "RGB"}], 
            "Output" -> NetDecoder[{"Class", labels}]]

enter image description here

 TakeLargestBy[incepNet[Import["https://i.stack.imgur.com/893F0.jpg"],
  "Probabilities"], Identity, 5]//Dataset

enter image description here

Dynamic@Column[{#, incepNet[#]} &@CurrentImage[]]

enter image description here

FileByteCount[Export["Inception v3.wlnet", incepNet]]/2^20.
(*44.0188 MB*)

I want to share this model to others, but it's too big (about 44 MB) because there are lots of parameters in this pretrained model.

If I only want to share this model structure to others,I think deNetInitialize is necessary.

enter image description here

You can see UninitializedEvaluationNet only need very little memory.

In order to share nets like UninitializedEvaluationNet, can we define a deNetInitialize function?

$\endgroup$
  • 3
    $\begingroup$ There is an internal function which does precisely this: NeuralNetworks`NetDeinitialize $\endgroup$ – dan7geo Jul 16 '17 at 21:42
  • $\begingroup$ @dan7geo wow! interesting...Thank you! $\endgroup$ – partida Jul 17 '17 at 3:01
5
$\begingroup$

The weights in the neural network are stored in the form of RawArray. This can be seen by comparing their full form

NetInitialize@ConvolutionLayer[1, {2}, "Input" -> {1, 3}] // InputForm
(* ConvolutionLayer[<|"Type" -> "Convolution", 
  "Arrays" -> <|"Weights" -> RawArray["Real32", {{{0.4258739650249481, -0.9216004014015198}}}], 
    "Biases" -> RawArray["Real32", {0.}]|>, "Parameters" -> <|"OutputChannels" -> 1, "KernelSize" -> {2}, "Stride" -> {1}, 
    "PaddingSize" -> {0}, "Dilation" -> {1}, "Dimensionality" -> 1, "InputChannels" -> 1, "$GroupNumber" -> 1, "$InputSize" -> {3}, 
    "$OutputSize" -> {2}|>, "Inputs" -> <|"Input" -> TensorT[{1, 3}, RealT]|>, "Outputs" -> <|"Output" -> TensorT[{1, 2}, RealT]|>|>, 
 <|"Version" -> "11.1.1"|>] *)

ConvolutionLayer[1, {2}, "Input" -> {1, 3}] // InputForm
(* ConvolutionLayer[<|"Type" -> "Convolution", "Arrays" -> <|"Weights" -> TensorT[{1, 1, 2}, RealT], 
    "Biases" -> Nullable[TensorT[{1}, RealT]]|>, "Parameters" -> <|"OutputChannels" -> 1, "KernelSize" -> {2}, "Stride" -> {1}, 
    "PaddingSize" -> {0}, "Dilation" -> {1}, "Dimensionality" -> 1, "InputChannels" -> 1, "$GroupNumber" -> 1, "$InputSize" -> {3}, 
    "$OutputSize" -> {2}|>, "Inputs" -> <|"Input" -> TensorT[{1, 3}, RealT]|>, "Outputs" -> <|"Output" -> TensorT[{1, 2}, RealT]|>|>, 
 <|"Version" -> "11.1.1"|>] *)

To remove the weights, we only need to replace the RawArray with the corresponding place holders. Here is a solution using string manipulation

netUnInitialize[net_] := 
 Activate@Replace[
   ToExpression[
    StringReplace[
     ToString[InputForm[net]], {"NetChain" -> "Inactive[NetChain]", 
      "NetGraph" -> "Inactive[NetGraph]", 
      "RawArray" -> "Inactive[RawArray]"}]], 
   Inactive[RawArray][x_, y_] :> 
    NeuralNetworks`TensorT[Dimensions[y], 
     NeuralNetworks`RealT], ∞]

FileByteCount[Export["~/Downloads/incept.wlnet",netUnInitialize[incepNet]]]
(* 40877 *)
$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.