# How do I change activation function of LinearLayer[] in NetGraph

I want to analize a network such that all neurons of its hidden layers output LogisticSigmoid[w.x+b], which is one of the most common ouputs analized in a neuron. Is there a way to combine LinearLayer[] that gives output w.x+b and the LogisticSigmoid function? I looked for an option of changing the activation function in LinearLayer[] but I did not find it.

Note: I do not want to use twice ammount of hidden layers by first applying LinearLayer[] and then ElementWiseLayer[LogisticSigmoid], I would like the network to be more compact. How would I do that?

• But ElementWiseLayer doesn't contain any parameters, it's the same as changing activation function for LinearLayer. You don't even have to apply ElementWiseLayer, just LogisticSigmoid works. – swish Apr 10 '18 at 17:45

I guess maybe you know the function in Keras:

model = Sequential()


Note, that is Dense layer rather than linear layer, so it have the option that can define the activation function.

In MMA, that is simple.

Dense[outDims_, "activation" -> act_] :=
Sequence[LinearLayer[outDims],
Switch[act, "tanh", ElementwiseLayer[Tanh], "relu",
ElementwiseLayer[Ramp], "sigmoid",
ElementwiseLayer[LogisticSigmoid]]]

NetChain[{Dense[5, "activation" -> "sigmoid"],
Dense[5, "activation" -> "sigmoid"],
Dense[5, "activation" -> "sigmoid"]}, "Input" -> 10]