How do I change activation function of LinearLayer[] in NetGraph

Question

I want to analize a network such that all neurons of its hidden layers output LogisticSigmoid[w.x+b], which is one of the most common ouputs analized in a neuron. Is there a way to combine LinearLayer[] that gives output w.x+b and the LogisticSigmoid function? I looked for an option of changing the activation function in LinearLayer[] but I did not find it.

Note: I do not want to use twice ammount of hidden layers by first applying LinearLayer[] and then ElementWiseLayer[LogisticSigmoid], I would like the network to be more compact. How would I do that?

Answer 1

I guess maybe you know the function in Keras:

model = Sequential()
model.add(Dense(5, activation='sigmoid', input_shape=(10,)))
model.add(Dense(5, activation='sigmoid'))
model.add(Dense(5, activation='sigmoid'))

Note, that is Dense layer rather than linear layer, so it have the option that can define the activation function.

In MMA, that is simple.

Dense[outDims_, "activation" -> act_] := 
 Sequence[LinearLayer[outDims], 
  Switch[act, "tanh", ElementwiseLayer[Tanh], "relu", 
   ElementwiseLayer[Ramp], "sigmoid", 
   ElementwiseLayer[LogisticSigmoid]]]

NetChain[{Dense[5, "activation" -> "sigmoid"], 
          Dense[5, "activation" -> "sigmoid"], 
          Dense[5, "activation" -> "sigmoid"]}, "Input" -> 10]