# Does RandomArrayLayer implement the reparameterization trick?

I want to backpropagate through random operations (e.g. learn the mean and variance of a Gaussian random variable). I assumed that

RandomArrayLayer[NormalDistribution[#\[Mu], #\[Sigma]] &];


implements the reparameterization trick. And therefore a RandomArrayLayer backpropagates a gradient to the mu and sigma inputs.

However, initial experiments seem to indicate that no such gradient is propagated.

Am I wrong in thinking that RandomArrayLayer is differentiable?

Thanks for any help.

It turns out the answer is no. However, Mathematica's great support for NNs means we can create our own. For example, to create a network layer of random variables that are normally distributed:

NormalReparameterizationLayer[initialMeans_List, initialStds_List]:=Module[
{\[Mu]Parameters, \[Sigma]Parameters, rl, rv, bnl},
rl = RandomArrayLayer[NormalDistribution[0,1],"Output"->Length[initialMeans]];
\[Mu]Parameters = NetArrayLayer["Array"->initialMeans,"Output"->Length[initialMeans]];
\[Sigma]Parameters = NetArrayLayer["Array"->initialStds,"Output"->Length[initialStds]];
rv = FunctionLayer[#Means+#Stds*#Random&,"Output"->Length[initialMeans]];
NetGraph[
{\[Mu]Parameters, \[Sigma]Parameters, rl, rv},
{1->4, 2->4, 3->4, NetPort[3,"Output"]->NetPort["Params"]}
]
]


Say that we want to create a reparameterization layer that generates 2 normally distributed variables with initial parameters {0.0, 10.0} and {30.0, 5.0}:

rpl = NormalReparamterizationLayer[{0.0, 30.0}, {10.0, 5.0}]


Running this layer yields a realisation of the two random variables:

In[]:= npl[]["Output"]

Out[]:= {4.42441, 23.87}


The layer is automatically differentiable. And so we can plug this layer into any NN architecture and learn the parameters of random variables.