How can I have two neural network layers act as the input to one layer?

Question

When using NetGraph[], I am try to specify the connections between 3 layers like so:

NetGraph[{...some layers...},{...{2,3}->4}]

But I receive the following error message:

"{2,3} is not a valid source for 4."

So it seems that Mathematica is having a hard time taking the input of two different layers into a single layer, which is strange to me, as an operation like:

...2->{3,4}...

Is considered valid, and will create 2 layers with the appropriate weights as output from the layer 2.

Does anyone know how I can have two layers be input to a single layer, while not losing the weighted connections of the first two layers to the third, such as in the following image: ? Thanks!

Answer 1

You can use CatenateLayer for this. Something close to your diagram might be like this:

NetGraph[{LinearLayer[4], LinearLayer[4], LinearLayer[4], 
  CatenateLayer[], LinearLayer[3]}, {1 -> 2, 1 -> 3, 2 -> 4, 3 -> 4, 
  4 -> 5}]