4
$\begingroup$

I'm attempting to implement this neural network in WL: https://github.com/lhelontra/squeeze-unet/blob/master/squeezeunet.py which is a variant of the SqueezeNet network that does image segmentation.

I feel as though I'm nearly there, but I'm getting this error message:

NetGraph::tyinc: Value for output of layer "fireup4" (a 64*50*50 3-tensor) is inconsistent with value for output of layer "concat" of layer "fireup4" (a 64*24*24 3-tensor).

This code for the fire module is taken from the SqueezeNet implementation on the Neural Network Repository, the implementation of which can be gotten using NetModel["SqueezeNet V1.1 Trained on ImageNet Competition Data", "ConstructionNotebook"].

fireModule[outputchannels_] := 
 NetGraph[<|

   "squeeze1x1" -> ConvolutionLayer[Floor[outputchannels/8], 1],
   "relu_squeeze1x1" -> ElementwiseLayer[Ramp],
   "batch" -> BatchNormalizationLayer[],
   "expand3x3" -> 
    ConvolutionLayer[Floor[outputchannels/2], 3, "PaddingSize" -> 1],
   "relu_expand3x3" -> ElementwiseLayer[Ramp],
   "expand1x1" -> ConvolutionLayer[Floor[outputchannels/2], 1],
   "relu_expand1x1" -> ElementwiseLayer[Ramp],
   "concat" -> CatenateLayer[]
   |>, {
   "squeeze1x1" -> 
    "relu_squeeze1x1" -> "batch" -> "expand3x3" -> "relu_expand3x3",
   "batch" -> "expand1x1" -> "relu_expand1x1",
   {"relu_expand1x1", "relu_expand3x3"} -> "concat"
   }]

The following is code for the "up" modules described in the Keras source.

upModule[dec_, fire_, decstride_] := 
 NetGraph[<|
   "cat" -> CatenateLayer[],
   "deconv" -> 
    DeconvolutionLayer[dec, 3, "Stride" -> {decstride, decstride}],
   "fire" -> fireModule[fire]
   |>, {

   {NetPort["Input1"] -> "deconv"},
   {{"deconv", NetPort["Input2"]} -> "cat" -> "fire"}
   }]

The code for the network follows. The first half (up to the DropoutLayer is about the same as the SqueezeNet implementation on the Neural Network Repository, the second half is taken from the Keras "SqueezeUNet" implementation I linked above.

squeezeUNet = 
 NetGraph[<|
   "conv1" -> ConvolutionLayer[64, 3, "Stride" -> 2],
   "relu_conv1" -> ElementwiseLayer[Ramp],
   "pool1" -> PoolingLayer[3, "Stride" -> 2],
   "fire2" -> fireModule[128],
   "fire3" -> fireModule[128],
   "pool3_pad" -> PaddingLayer[{{0, 0}, {0, 1}, {0, 1}}],
   "pool3" -> PoolingLayer[3, "Stride" -> 2],
   "fire4" -> fireModule[256],
   "fire5" -> fireModule[256],
   "pool5_pad" -> PaddingLayer[{{0, 0}, {0, 1}, {0, 1}}],
   "pool5" -> PoolingLayer[3, "Stride" -> 2],
   "fire6" -> fireModule[384],
   "fire7" -> fireModule[384],
   "fire8" -> fireModule[512],
   "fire9" -> fireModule[512],

   "drop9" -> DropoutLayer[0.5],

   "up1" -> upModule[192, 512, 1],
   "up2" -> upModule[128, 256, 1],
   "up3" -> upModule[64, 128, 2],
   "up4" -> upModule[32, 64, 2],

   "cat10" -> CatenateLayer[],
   "conv10" -> ConvolutionLayer[64, {3, 3}, "Stride" -> {1, 1}],
   "relu_conv10" -> ElementwiseLayer[Ramp],
   "conv11" -> ConvolutionLayer[1, 1],
   "sigmoid" -> ElementwiseLayer["Sigmoid"]
   |>,

  {{"conv1" ->
     "relu_conv1" ->
      "pool1" ->
       "fire2" ->
        "fire3" ->
         "pool3_pad" ->
          "pool3" ->
           "fire4" ->
            "fire5" ->
             "pool5_pad" ->
              "pool5" ->
               "fire6" ->
                "fire7" ->
                 "fire8" ->
                  "fire9" ->
                   "drop9"},

   {{"drop9", "fire8"} -> "up1"},
   {{"up1", "pool5"} -> "up2"},
   {{"up2", "pool3"} -> "up3"},
   {{"up3", "pool1"} -> "up4"},

   {"up4", "relu_conv1"} -> 
    "cat10" -> "conv10" -> "relu_conv10" -> "conv11" -> "sigmoid"},

  "Input" -> 
   NetEncoder[{"Image", {101, 101}, ColorSpace -> "Grayscale"}]
  ]

When I visualise this, it's the correct U-shape. However, I don't really understand the error message I'm getting when I try to evaluate the NetGraph.

NetGraph::tyinc: Value for output of layer "fireup4" (a 64*50*50 3-tensor) is inconsistent with value for output of layer "concat" of layer "fireup4" (a 64*24*24 3-tensor).

I don't think that I've specified the layer output shapes in such a way that would cause this error, but I freely admit that I just barely know what I'm doing here. I couldn't find any information about this error message in the documentation - could someone please help me understand it?

$\endgroup$
  • 1
    $\begingroup$ I don't have time to try things out, but one thing that stands out is that the Keras code you link to uses padding = "same" in its convolutional layers, which means that the output will have the same dimension as the input. The Mathematica code doesn't use padding, so the output is smaller than the input. $\endgroup$ – C. E. Aug 2 '18 at 14:04
  • $\begingroup$ Thanks, that's helpful. Unfortunately I don't know how to replicate padding="same" behaviour in WL, but it gets me a little closer to understanding the problem. $\endgroup$ – Carl Lange Aug 2 '18 at 14:49
  • 1
    $\begingroup$ They also use padding = "same" for the pooling layers. How to replicate this could be a good separate question. If there isn't a solution one could always compute the padding manually or inspect the Keras network to see what the dimensions are supposed to be. You can build the network layer by layer, side by side in Mathematica and Keras and make sure you have the same dimensions throughout. (This is an unsatisfying solution, I know...) $\endgroup$ – C. E. Aug 2 '18 at 17:00
  • 4
    $\begingroup$ They should add the string option "PaddingSize"->"Same" in v12! $\endgroup$ – user5601 Aug 2 '18 at 20:45
  • $\begingroup$ Thinking about it, a regular UNET requires images to have dimensions that are a power of two - that's possibly the case here also. $\endgroup$ – Carl Lange Aug 5 '18 at 23:11
0
$\begingroup$

One way to get around this is to resize the outputs of the upModule so that they match the dimensions of the layer you're joining to.

upModule[dec_, fire_, decstride_] := 
 NetGraph[<|"cat" -> CatenateLayer[], 
   "deconv" -> 
    DeconvolutionLayer[dec, 3, "Stride" -> {decstride, decstride}, 
     "PaddingSize" -> 1], 
   "fire" -> 
    fireModule[fire]|>, {{NetPort["Input1"] -> 
     "deconv"}, {{"deconv", NetPort["Input2"]} -> "cat" -> "fire"}}]

(note the "PaddingSize" addition)

And now a graph that at least doesn't fail to evaluate:

squeezeUNet = 
 NetGraph[<|
   "conv1" -> ConvolutionLayer[64, 3, "Stride" -> 2],
   "relu_conv1" -> ElementwiseLayer[Ramp],
   "pool1" -> PoolingLayer[3, "Stride" -> 2],
   "fire2" -> fireModule[128],
   "fire3" -> fireModule[128],
   "pool3_pad" -> PaddingLayer[{{0, 0}, {0, 1}, {0, 1}}], 
   "pool3" -> PoolingLayer[3, "Stride" -> 2],
   "fire4" -> fireModule[256],
   "fire5" -> fireModule[256],
   "pool5_pad" -> PaddingLayer[{{0, 0}, {0, 1}, {0, 1}}], 
   "pool5" -> PoolingLayer[3, "Stride" -> 2],
   "fire6" -> fireModule[384],
   "fire7" -> fireModule[384],
   "fire8" -> fireModule[512],
   "fire9" -> fireModule[512],
   "up1" -> upModule[192, 512, 1],
   "up2" -> {upModule[128, 256, 1], ResizeLayer[{28, 28}]},
   "up3" -> {upModule[64, 128, 1], ResizeLayer[{56, 56}]},
   "up4" -> {upModule[32, 64, 1], ResizeLayer[{113, 113}]},
   "cat10" -> CatenateLayer[],
   "conv10" -> ConvolutionLayer[64, {3, 3}, "Stride" -> {1, 1}],
   "relu_conv10" -> ElementwiseLayer[Ramp],
   "conv11" -> ConvolutionLayer[1, 1],
   "sigmoid" -> 
    ElementwiseLayer["Sigmoid"]|>, {{"conv1" -> 
     "relu_conv1" -> 
      "pool1" -> 
       "fire2" -> 
        "fire3" -> 
         "pool3_pad" -> 
          "pool3" -> 
           "fire4" -> 
            "fire5" -> 
             "pool5_pad" -> 
              "pool5" -> 
               "fire6" -> "fire7" -> "fire8" -> "fire9"}, {{"fire9", 
      "fire8"} -> "up1"},
   {{"up1", "pool5"} -> "up2"},
   {{"up2", "pool3"} -> "up3"},
   {{"up3", "pool1"} -> "up4"},
   {"up4", "relu_conv1"} -> 
    "cat10" -> "conv10" -> "relu_conv10" -> "conv11" -> "sigmoid"}, 
  "Input" -> 
   NetEncoder[{"Image", {227, 227}, ColorSpace -> "Grayscale"}]]

This is all pretty bad! The issue stems from the upModule output not having the same dimensions as the things it's joining to, so CatenateLayer can't catenate them. However, the error message is definitely quite hard to parse.

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.