3
$\begingroup$

I am trying to design a neural network layer that takes in an array of dimensions n x 100, and produces an array of dimensions n x 100, where each of the n sublists is replaced by the nth. For example, this function would turn {{1,2,3,4,5,...},{4,5,6,7,8,...},{7,8,9,10,11,...}} into {{7,8,9,10,11,...},{7,8,9,10,11,...},{7,8,9,10,11,...}} - the same number of elements, but all replaced with the last sublist.

My original implementation was this:

allRep[] := 
 NetGraph[{"last" -> SequenceLastLayer[], 
   "repl" -> ReplicateLayer[Automatic, "Input" -> 100], 
   "switch" -> ThreadingLayer[#2 &]}, {"last" -> 
    "repl", {NetPort@"Input", "repl"} -> 
    "switch"}]

This actually works, and initializes, but when you try to pair it with any other layer you get an Internal error saying that it can't serialize something to do with the ReplicateLayer:

NetInitialize[NetChain[{allRep[], LongShortTermMemoryLayer[500]}]][
 ConstantArray[0, {2, 100}]]

It seems the problem is that the ReplicateLayer can't figure out to replicate the input n times, even though it says it does in the NetGraph. A simple SequenceLastLayer and ReplicateLayerdoesn't automatically infer the first dimension (n) of the output either. Is this possible to do in Mathematica?

Improve this question
$\endgroup$
3
  • $\begingroup$ Out of curiosity, why the you plug a NetPort@"Input" into a layer that always discards it (ThreadingLayer[#2 &])? $\endgroup$
    – Fortsaint
    Commented Oct 6, 2019 at 23:07
  • 1
    $\begingroup$ @Fortsaint It is to get the ReplicateLayer to infer the dimension automatically - try having just a SequenceLastLayer and ReplicateLayer[Automatic] and it’ll introduce an n2 rather than keep the n of the first dimension $\endgroup$
    – Nico A
    Commented Oct 6, 2019 at 23:13
  • $\begingroup$ ReplicateLayer doesn't support dynamic dimensions (yet), so you'll have to specify static dimensions for the inputs before NetInitialize. $\endgroup$
    – Silvia
    Commented Nov 22, 2019 at 7:03

0

Reset to default

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.