3
$\begingroup$

I am trying to design a neural network layer that takes in an array of dimensions n x 100, and produces an array of dimensions n x 100, where each of the n sublists is replaced by the nth. For example, this function would turn {{1,2,3,4,5,...},{4,5,6,7,8,...},{7,8,9,10,11,...}} into {{7,8,9,10,11,...},{7,8,9,10,11,...},{7,8,9,10,11,...}} - the same number of elements, but all replaced with the last sublist.

My original implementation was this:

allRep[] := 
 NetGraph[{"last" -> SequenceLastLayer[], 
   "repl" -> ReplicateLayer[Automatic, "Input" -> 100], 
   "switch" -> ThreadingLayer[#2 &]}, {"last" -> 
    "repl", {NetPort@"Input", "repl"} -> 
    "switch"}]

This actually works, and initializes, but when you try to pair it with any other layer you get an Internal error saying that it can't serialize something to do with the ReplicateLayer:

NetInitialize[NetChain[{allRep[], LongShortTermMemoryLayer[500]}]][
 ConstantArray[0, {2, 100}]]

It seems the problem is that the ReplicateLayer can't figure out to replicate the input n times, even though it says it does in the NetGraph. A simple SequenceLastLayer and ReplicateLayerdoesn't automatically infer the first dimension (n) of the output either. Is this possible to do in Mathematica?

$\endgroup$
  • $\begingroup$ Out of curiosity, why the you plug a NetPort@"Input" into a layer that always discards it (ThreadingLayer[#2 &])? $\endgroup$ – Fortsaint Oct 6 '19 at 23:07
  • 1
    $\begingroup$ @Fortsaint It is to get the ReplicateLayer to infer the dimension automatically - try having just a SequenceLastLayer and ReplicateLayer[Automatic] and it’ll introduce an n2 rather than keep the n of the first dimension $\endgroup$ – Nico A Oct 6 '19 at 23:13
  • $\begingroup$ ReplicateLayer doesn't support dynamic dimensions (yet), so you'll have to specify static dimensions for the inputs before NetInitialize. $\endgroup$ – Silvia Nov 22 '19 at 7:03

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Browse other questions tagged or ask your own question.