3
$\begingroup$

EDIT: Looking closely at my network I have a similar problem of providing inputs to the CropLayer function. The output from the pooling layer (node 30) and node 87 (activation) should feed to the CropLayer. The question is how to feed outputs from one layer to other layers in the network

I am trying to implement UNET (https://lmb.informatik.uni-freiburg.de/people/ronneber/u-net/) in Mathematica.

I have the following code so far for generating the net partially:

(* encoder *)

encoder = NetEncoder[{"Image", "ImageSize" -> {168, 168}, "ColorSpace" -> "Grayscale"}];

(* decoder *)

decoder = NetDecoder[{"Image", "ColorSpace" -> Automatic}];

(* convolution module *)

Options[convolutionModule] = {"batchNorm" -> True, "downpool" -> False,
"uppool" -> False, "activationType" -> Ramp, "convolution" -> True};

convolutionModule[net_, kernelsize_, padsize_, stride_: {1, 1}, OptionsPattern[]] := 
With[{upPool = OptionValue["uppool"], activationType = OptionValue["activationType"], 
convolution = OptionValue["convolution"], batchNorm = OptionValue["batchNorm"],
downpool = OptionValue@"downpool"},

Block[{nnet = net},
If[upPool,
nnet = NetAppend[nnet, DeconvolutionLayer[1, {2, 2}, "PaddingSize" -> {0, 0}, 
   "Stride" -> {2, 2}]];
nnet = NetAppend[nnet, BatchNormalizationLayer[]];
If[activationType === Ramp,
 nnet = NetAppend[nnet, ElementwiseLayer[activationType]]
 ];
];
If[convolution,
 nnet = NetAppend[nnet, ConvolutionLayer[1, kernelsize, "Stride" -> stride,
   "PaddingSize" -> padsize]]
];
If[batchNorm,
nnet = NetAppend[nnet, BatchNormalizationLayer[]]
];
If[activationType === Ramp,
nnet = NetAppend[nnet, ElementwiseLayer[activationType]]
];
If[downpool,
nnet = NetAppend[nnet, PoolingLayer[{2, 2}, "Function" -> Max, "Stride" -> {2, 2}]]
];
nnet]
]

(* Crop Layer *)

CropLayer[netlayer_] := With[{p = NetExtract[netlayer, "Output"]},
PartLayer[{First@p, 1 ;; p[[2]], 1 ;; Last@p}] ]; 

(* UNET *)

UNET[] := 
Block[{nm, pool1, pool2, pool3, pool4, pool5, kernelsize = {3, 3}, 
padsize = {1, 1}, stride = {1, 1}},
nm = NetChain@
Join[{ConvolutionLayer[1, {3, 3}, 
   "Input" -> encoder]}, {BatchNormalizationLayer[], 
  ElementwiseLayer[Ramp],
  PoolingLayer[{2, 2}, "Function" -> Max, "Stride" -> {2, 2}]}];
pool1 = nm[[-1]];
nm = convolutionModule[nm, kernelsize, padsize, stride,"downpool" -> True];
pool2 = nm[[-1]];
nm = convolutionModule[nm, kernelsize, padsize, stride,"downpool" -> True];
pool3 = nm[[-1]];
nm = convolutionModule[nm, kernelsize, padsize, stride,"downpool" -> True];
pool4 = nm[[-1]];
nm = NetAppend[nm, DropoutLayer[]];
nm = convolutionModule[nm, kernelsize, padsize, stride, "downpool" -> True];
pool5 = nm[[-1]];
nm = convolutionModule[nm, kernelsize, padsize, stride, "uppool" -> True]; 
nm = convolutionModule[nm, kernelsize, padsize + 1, stride, "uppool" -> True]; 
nm = NetAppend[nm, CropLayer@pool3];

with NetInformation I can generate the net plot below:

NetInformation[(nm = UNET[]), "MXNetNodeGraphPlot"]

enter image description here

My problem is: how do i catenate the output from the pooling layer i.e. node 30 with the output from node 91.

I tried using NetGraph with CatenateLayer but could not find a way to connect node 30 within the NetChain with the second input of CatenateLayer.

enter image description here

$\endgroup$
  • $\begingroup$ Please try to reduce this to a minimal working example. $\endgroup$ – MarcoB Apr 24 '18 at 1:28
  • $\begingroup$ @MarcoB will do later in the day $\endgroup$ – Ali Hashmi Apr 24 '18 at 6:36
2
$\begingroup$

U-Net

conv[n_] := NetChain[
  {
   ConvolutionLayer[n, 3, "PaddingSize" -> {1, 1}],
   Ramp,
   BatchNormalizationLayer[],
   ConvolutionLayer[n, 3, "PaddingSize" -> {1, 1}],
   Ramp,
   BatchNormalizationLayer[]
   }
  ]

pool := PoolingLayer[{2, 2}, 2]

dec[n_] := NetGraph[
  {
   "deconv" -> DeconvolutionLayer[n, {2, 2}, "Stride" -> {2, 2}],
   "cat" -> CatenateLayer[],
   "conv" -> conv[n]
   },
  {
   NetPort["Input1"] -> "cat",
   NetPort["Input2"] -> "deconv" -> "cat" -> "conv"
   }
  ]


unet = NetGraph[
   <|
    "enc_1" -> conv[64],
    "enc_2" -> {pool, conv[128]},
    "enc_3" -> {pool, conv[256]},
    "enc_4" -> {pool, conv[512]},
    "enc_5" -> {pool, conv[1024]},
    "dec_1" -> dec[512],
    "dec_2" -> dec[256],
    "dec_3" -> dec[128],
    "dec_4" -> dec[64],
    "map" -> {ConvolutionLayer[1, {1, 1}], LogisticSigmoid}
    |>,
   {
    NetPort["Input"] -> "enc_1" -> "enc_2" -> "enc_3" -> "enc_4" -> "enc_5",
    {"enc_4", "enc_5"} -> "dec_1",
    {"enc_3", "dec_1"} -> "dec_2",
    {"enc_2", "dec_2"} -> "dec_3",
    {"enc_1", "dec_3"} -> "dec_4",
    "dec_4" -> "map"
    },
   "Input" -> NetEncoder[{"Image", {512, 512}, ColorSpace -> "Grayscale"}]
   ] // NetInitialize

enter image description here

img = RandomImage[1, {512, 512}];

unet@img

enter image description here

$\endgroup$
  • 1
    $\begingroup$ @AliHashmi Look how I defined decoder (with CatenateLayer in it) and connected it with other layers inside of NetGraph. $\endgroup$ – Alexey Golyshev Apr 24 '18 at 12:16
  • 1
    $\begingroup$ Many thanks. upvoted it. Will check your implementation thoroughly at night. I will accept the answer after a couple of days :) $\endgroup$ – Ali Hashmi Apr 24 '18 at 13:42
  • $\begingroup$ can we create a room and talk there? I have a few questions in mind if you can kindly help. $\endgroup$ – Ali Hashmi May 1 '18 at 11:14
  • 1
    $\begingroup$ @AliHashmi My email: alexey.golyshev@live.com. Write your questions. $\endgroup$ – Alexey Golyshev May 2 '18 at 3:39
  • $\begingroup$ kindly check the following question i have posted if it interests you? mathematica.stackexchange.com/questions/172481/… $\endgroup$ – Ali Hashmi May 3 '18 at 14:24

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.