# Basic Feed Forward Network

I am trying to build a simple neural network, with an input layer of 250 nodes, 2 'hidden' layers each of 100 nodes, and an output layer with 1 node. I want each node to take in the linear combination of every node in the previous layer and perform the ReLU function. So far, my code is

layer1 = LinearLayer[100,"Input"->250]
layer2 = LinearLayer[100,"Input"->100]
layer3 = LinearLayer[1,"Input"->100]

NetInitialize@NetChain[{layer1,layer2,layer3}]


, but this is giving me an extra output layer and I am not sure how to encorporate the ReLU function.

• Just riffle them: NetChain[{layer1,ElementwiseLayer["ReLU"],layer2,ElementwiseLayer["ReLU"],layer3}] – M.R. Aug 28 '18 at 16:21
• Ok. I am confused though, because it looks like ElementwiseLayer["ReLU"] is adding another layer, whereas I want the ReLU function to be applied to each node in the layer – pmac Aug 28 '18 at 16:28
• And when you read about ElementwiseLayer, what does the documentation say that it does? It is the very thing you are looking for. I think it could help to think about neural networks, at least in Wolfram Language, not as layers with nodes but to think of each layer as a function. You're passing a tensor through a series of functions. It is completely natural from this point of view that activation functions are layers. – C. E. Aug 28 '18 at 22:39