0
$\begingroup$

I am trying to build a simple neural network, with an input layer of 250 nodes, 2 'hidden' layers each of 100 nodes, and an output layer with 1 node. I want each node to take in the linear combination of every node in the previous layer and perform the ReLU function. So far, my code is 

layer1 = LinearLayer[100,"Input"->250]
layer2 = LinearLayer[100,"Input"->100]
layer3 = LinearLayer[1,"Input"->100]

NetInitialize@NetChain[{layer1,layer2,layer3}]

, but this is giving me an extra output layer and I am not sure how to encorporate the ReLU function.

Improve this question
$\endgroup$
3
  • $\begingroup$ Just riffle them: NetChain[{layer1,ElementwiseLayer["ReLU"],layer2,ElementwiseLayer["ReLU"],layer3}] $\endgroup$
    – M.R.
    Aug 28, 2018 at 16:21
  • $\begingroup$ Ok. I am confused though, because it looks like ElementwiseLayer["ReLU"] is adding another layer, whereas I want the ReLU function to be applied to each node in the layer $\endgroup$
    – ngc1300
    Aug 28, 2018 at 16:28
  • 1
    $\begingroup$ And when you read about ElementwiseLayer, what does the documentation say that it does? It is the very thing you are looking for. I think it could help to think about neural networks, at least in Wolfram Language, not as layers with nodes but to think of each layer as a function. You're passing a tensor through a series of functions. It is completely natural from this point of view that activation functions are layers. $\endgroup$
    – C. E.
    Aug 28, 2018 at 22:39

0

Reset to default

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Browse other questions tagged or ask your own question.