# BasicRecurrentLayer with no bias and linear activation function?

Trying to design a small neural network for modeling a LTI system, I'd like to configure a BasicRecurrentLayer with these particular settings:

• No bias vector should be used (I've tried the option "Biases"-> None, but it does not work for a BasicRecurrentLayer)
• The output should be directly the weighted sum of its inputs (I'd like to use Identity instead of Tanh as activation function, which is the default value for this kind of layer)

I could apply a ElementwiseLayer[ArcTanh] at the output of the layer, to somehow undo the effect of Tanh, but still the recurrent connection going backwards to the input port would not be the desired one.

As far as I can see, I can't configure BasicRecurrentLayer to my own needs, so must I then create a new user-defined layer, and make the necessary recurrent connections manually into a NetGraph? How could I do it? I'm a newbie in the neural networks field, and any help is much appreciated.

• Could you please include the code you have developed so far in your attempts? That would give a valuable starting point for people to play around with and help you with your problem. Mar 4, 2022 at 13:18

SeedRandom[0];
batchSize = 16;
X = RandomReal[{-1, 1}, {batchSize, 10, 8}];
Y = RandomInteger[{0, 1}, {batchSize, 1}];

net = NetChain[
{
BasicRecurrentLayer[4],
SequenceLastLayer[],
LinearLayer[1],
ElementwiseLayer[LogisticSigmoid]
},
"Input" -> {"Varying", 8}
] // NetInitialize


NetExtract[net, {1, "Biases"}] // Normal


{0., 0., 0., 0.}

netT = NetTrain[net, X -> Y, MaxTrainingRounds -> 10];

NetExtract[netT, {1, "Biases"}] // Normal


{0.00993016, 0.00993503, -0.00995534, -0.00985054}

LearningRateMultipliers is useful here.

netT2 = NetTrain[net, X -> Y, MaxTrainingRounds -> 10,
LearningRateMultipliers -> {{1, "Biases"} -> 0}];

NetExtract[netT2, {1, "Biases"}] // Normal


{0., 0., 0., 0.}

You can also look at NetFoldOperator to create customized RNN.

• OK for the method to get rid of the biases, but why did you add a LogisticSigmoid at the end of the chain? I just need a linear function at the output. Mar 16, 2022 at 8:26
• Hello @Danel. You can delete LogisticSigmoid. In this example, Y is in the range [0, 1]. Mar 16, 2022 at 12:48