Deconvolution approaches for 3D data

Question

Building neural nets in the WL is great. I was in the process of rewriting a previous GAN of mine in the WL to see if I can get better performance, and was surprised to find that the standard DeconvolutionLayer does not have the same coverage as ConvolutionLayer. There's no option on the DeconvolutionLayer to enter a {h, w, d}-sized kernel. My use is in the Architecture/Engineering field but this seems like an even larger oversight for people doing medical image analysis. Be that as it may, the search is now on for a good workaround. I'm looking for something similar in functionality to Tensorflow2 Keras' Conv3DTranspose.

There seems to be very little writing online about this particular corner of Mathematica. There is this 2018 post by Martijn Froeling on the Wolfram Community forums which I'm investigating. Instead of upscaling through a DeconvolutionLayer that keeps the trainable parameters, there's quite a lot of working around with RescaleLayer and ResizeLayer. I spent yesterday parsing its flow and looking up all the layers in the docs but there are still gaps in my knowledge. What's completely lost on me are the "whys". Why does this lead to this? Why did they choose to put this after that? I'm really missing the big picture here, and would like to have a better understanding of this workaround before I use a variation of it in my project.

If someone who can parse this code better than I can annotate it? And perhaps use it on an example Image3D resource? This is good information for the community I think, until the DeconvolutionLayer has better coverage to match TensorFlow2. Is there a chance that the WL neural network framework has implemented this functionality but it doesn't work as we expect?

DeconvLayer2D[n_, {dimInx_, dimIny_}] := Block[{sc = 2}, 
  NetChain[{
    DeconvolutionLayer[n, {sc, sc}, "Stride" -> {sc, sc}, "Input" -> {sc n, dimInx, dimIny}]
}]]

ResizeLayer2D[n_, {dimInx_, dimIny_}] := Block[{sc = 2}, 
  NetChain[{
    ResizeLayer[{Scaled[sc], Scaled[sc]}, "Input" -> {sc n, dimInx, dimIny}], 
    ConvolutionLayer[n, 1]
}]]

ResizeLayer3D[n_, {dimInx_, dimIny_, dimInz_}] := Block[{sc = 2},
 NetChain[ {
    FlattenLayer[1, "Input" -> {n sc, dimInx, dimIny, dimInz}],
    ResizeLayer[{Scaled[sc], Scaled[sc]}], 
    ReshapeLayer[{n sc, dimInx, sc dimIny, sc dimInz}],
    TransposeLayer[2 <-> 3], 
    FlattenLayer[1],
    ResizeLayer[{Scaled[sc], Scaled[1]}], 
    ReshapeLayer[{n sc, sc dimIny, sc dimInx, sc dimInz}],
    TransposeLayer[2 <-> 3], 
    ConvolutionLayer[n, 1]
}]]

{DeconvLayer2D[16, {2, 4}], ResizeLayer2D[16, {2, 4}], ResizeLayer3D[16, {2, 4, 6}]}

Answer 1

Even after offering a Stack Exchange bounty, unfortunately there was no interest from the community in adding further to the question. As a way of continuing to problem solve and hopefully elicit some help, I wanted to start off by trying to annotate some of the code myself.

(* n is the number of features in the kernel. sc is the scale of 
upsampling *)  
ResizeLayer3D[n_, {dimInx_, dimIny_, dimInz_}] :=
 Block[{sc = 2},
  NetChain[{
    (* Flattens the first two levels so because the next layer only works on 2D arrays *)        
    FlattenLayer[1, "Input" -> {n sc, dimInx, dimIny, dimInz}],
    
    (* Doubles the size of the last two dimensions *)        
    ResizeLayer[{Scaled[sc], Scaled[sc]}],
    
    (* Reshapes the array back to its original order but with the  last two dimensions scaled up from previous layer *)        
    ReshapeLayer[{n sc, dimInx, sc dimIny, sc dimInz}],
    
    (* Transposes 2nd and 3rd dimensions so that previously unscaled dimension can be actioned *)
    TransposeLayer[2 <-> 3],
    
    (* Again flatten array a level so that the next later can actually action it *)
    FlattenLayer[1],
    
    (* Scale only the dimension that hasn't been actioned yet *)        
    ResizeLayer[{Scaled[sc], Scaled[1]}],
    
    (* Reshape back to original structure but the array now has been 
    scaled up *)
    ReshapeLayer[{n sc, sc dimIny, sc dimInx, sc dimInz}],
    
    (* importantly, transpose the dimensions back to their original 
    order since it was changed above*)
    TransposeLayer[2 <-> 3],
    
    (* Now a convolution can be applied to the upsampled up data *)
    ConvolutionLayer[n, 1]}
   ]
  ]

Nets return an array of numbers and can be implemented like the example below. Sharing an animated ArrayPlot for convenience.

net = NetInitialize@NetChain[{
    ResizeLayer3D[1, 16],
    ElementwiseLayer["ELU"],
    ResizeLayer3D[1, 28],
    ElementwiseLayer["ELU"]
    }]

net[noise]

(* {{{{0.135027, 0.135027, 0.135027, 0.135027, 0.135027, 0.135027, 
    0.135027, 0.135027, 0.135027, 0.135027, 0.135027, 0.135027, 
    0.135027, 0.135027, ...... 0.194518, 
    0.204861, 0.192721, 0.163842, 0.119893, 0.0791787, 0.0570972, 
    0.0464499, 0.0538053, 0.0643428, 0.07991, 0.100084, 0.11526, 
    0.129162}, ..... }}}} *)