# Net layer that replicates last element

I am trying to design a neural network layer that takes in an array of dimensions n x 100, and produces an array of dimensions n x 100, where each of the n sublists is replaced by the nth. For example, this function would turn {{1,2,3,4,5,...},{4,5,6,7,8,...},{7,8,9,10,11,...}} into {{7,8,9,10,11,...},{7,8,9,10,11,...},{7,8,9,10,11,...}} - the same number of elements, but all replaced with the last sublist.

allRep[] :=
NetGraph[{"last" -> SequenceLastLayer[],
"repl" -> ReplicateLayer[Automatic, "Input" -> 100],
"switch" -> ThreadingLayer[#2 &]}, {"last" ->
"repl", {NetPort@"Input", "repl"} ->
"switch"}]


This actually works, and initializes, but when you try to pair it with any other layer you get an Internal error saying that it can't serialize something to do with the ReplicateLayer:

NetInitialize[NetChain[{allRep[], LongShortTermMemoryLayer[500]}]][
ConstantArray[0, {2, 100}]]


It seems the problem is that the ReplicateLayer can't figure out to replicate the input n times, even though it says it does in the NetGraph. A simple SequenceLastLayer and ReplicateLayerdoesn't automatically infer the first dimension (n) of the output either. Is this possible to do in Mathematica?

• Out of curiosity, why the you plug a NetPort@"Input" into a layer that always discards it (ThreadingLayer[#2 &])? – Fortsaint Oct 6 '19 at 23:07
• @Fortsaint It is to get the ReplicateLayer to infer the dimension automatically - try having just a SequenceLastLayer and ReplicateLayer[Automatic] and it’ll introduce an n2 rather than keep the n of the first dimension – Nico A Oct 6 '19 at 23:13
• ReplicateLayer doesn't support dynamic dimensions (yet), so you'll have to specify static dimensions for the inputs before NetInitialize. – Silvia Nov 22 '19 at 7:03