2
$\begingroup$

I have problems using a recurring network built using the NetFoldOperator in cascade to a LinearLayer. What happens is that the memory is reset at each input without keeping memory in the window.

This is the example:

W = NetInitialize@
   LinearLayer[1, "Input" -> 2, "Weights" -> {{1, 1}}, 
    "Biases" -> None];
core = NetGraph[<|
    "fun" -> 
     ThreadingLayer[#1 + #2 &]|>, {{NetPort["in"], NetPort["In2"]} -> 
     "fun" -> NetPort["out"]}];
dyn = NetFoldOperator[core, {"out" -> "In2"}];
complex = NetChain[{W, dyn}];

Output of the dyn network (* 1+0=1 => 1+2=3 => 3+3=6 *):

dyn[{1, 2, 3}] (* -> {1., 3., 6.} *)

Output of the complex network (* {1, 2}*W=3+0=3 => {1, 2}*W=3+0=3 => {1, 2}*W=3+0=3 *):

complex[{{1, 2}, {1, 2}, {1, 2}}] (* -> {{3.}, {3.}, {3.}} *)

The memory is reseted every input. How can this be avoided?

How can you get an output like complex[{{1, 2}, {1, 2}, {1, 2}}] (* -> {{3.}, {6.}, {9.}} *) ?

$\endgroup$

1 Answer 1

2
$\begingroup$

Your LinearLayer takes a length-2 vector as input, not the whole matrix. Because of that,

complex[{{1, 2}, {1, 2}, {1, 2}}] is interpreted as:

{complex[{1,2}],complex[{1,2}],complex[{1,2}]} instead.

In the end, the input of the dyn layer is just a single length-1 vector {3}, instead of {3,3,3}.

You want to apply the LinearLayer to each length-2 vector in the input, and output a list of the results to the dyn layer. For that, wrap your LinearLayer inside a NetMapOperator:

W = NetInitialize@
   LinearLayer[1, "Input" -> 2, "Weights" -> {{1, 1}}, 
    "Biases" -> None];
core = NetGraph[<|
    "fun" -> 
     ThreadingLayer[Plus]|>, {{NetPort["in"], NetPort["In2"]} -> 
     "fun" -> NetPort["out"]}, "out" -> 1];
dyn = NetFoldOperator[core, {"out" -> "In2"}];
complex = NetChain[{NetMapOperator[W], dyn}];

That way you have:

complex[{{1, 2}, {1, 2}, {1, 2}}] => {{3.}, {6.}, {9.}}

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.