I want to build the neural network in "Classifying and segmenting microscopy images with deep multiple instance learning". It is a fairly straightforward CNN, except that they define a custom layer which they describe as Noisy-AND.
I'm able to define this operation as a NetGraph but it's surprisingly convoluted to do so.
I'm wondering if anyone can help me express the same computation more elegantly.
The input is a vector of features for each output class (they are scaled
between zero and one, and represent probabilities.) The first step is to take the mean
of the feature vector for each class; call this $p_{i}$. This is easy enough with an AggregationLayer.
There are two learnable parameters: $a$, a single scalar for the whole
network, and $b$, a vector of parameters, one $b_i$ for each class. These can
be expressed using ConstantArrayLayer, although I need to use ReplicateLayer to express
multiplying a vector by the scalar $a$.
The noisy-AND function, for a given class $i$, is then
$ \begin{equation*} \frac{\sigma(a(p_i - b_i)) - \sigma(-a b_i)}{\sigma(a(1 - b_i)) - \sigma(- a b_i)} \end{equation*} $
That looks pretty simple and innocent. But here is some Mathematica code to generate a NetGraph. As you can see, it's pretty involved, although fortunately I can now just use it as a subcomponent in the much easier-to-define CNN.
Any way you can think of to simplify it would be helpful.
ClearAll[noisyAnd];
noisyAnd[numClasses_, numFeatures_] :=
NetGraph[<|
"a" -> ConstantArrayLayer["Array" -> {1.0}],
"b" ->
ConstantArrayLayer["Array" -> ConstantArray[0.5, numClasses]],
"mean" -> AggregationLayer[Mean],
"minus1numerator" -> ThreadingLayer[Subtract],
"multiply2" -> ThreadingLayer[Times],
"negatedSigmoid" -> ElementwiseLayer[LogisticSigmoid[-#] &],
"broadcast" -> ReplicateLayer[numClasses],
"flattenA" -> FlattenLayer[],
"minus2numerator" -> ThreadingLayer[Subtract],
"multiply1numerator" -> ThreadingLayer[Times],
"upperSigmoid" -> LogisticSigmoid,
"oneMinus" -> ElementwiseLayer[1.0 - # &],
"multiply1denominator" -> ThreadingLayer[Times],
"lowerSigmoid" -> LogisticSigmoid,
"minus2denominator" -> ThreadingLayer[Subtract],
"quotient" -> ThreadingLayer[Divide]|>,
{
"mean" -> "minus1numerator",
"b" -> "minus1numerator",
"a" -> "broadcast" -> "flattenA",
(*compute sigmoid(-abi)*)
"flattenA" -> "multiply2",
"b" -> "multiply2",
"multiply2" -> "negatedSigmoid",
(*compute numerator*)
"minus1numerator" -> "multiply1numerator",
"flattenA" -> "multiply1numerator",
"multiply1numerator" -> "upperSigmoid",
"upperSigmoid" -> "minus2numerator",
"negatedSigmoid" -> "minus2numerator",
(*compute denominator*)
"b" -> "oneMinus",
"flattenA" -> "multiply1denominator",
"oneMinus" -> "multiply1denominator",
"multiply1denominator" -> "lowerSigmoid",
"lowerSigmoid" -> "minus2denominator",
"negatedSigmoid" -> "minus2denominator",
"minus2numerator" -> "quotient",
"minus2denominator" -> "quotient"
}, "Input" -> {numClasses, numFeatures}]
