I have a matrix $A_{m×n}$ and I want to assemble a network that performs $n$ different trainable linear transformations $T_{p×m}(i), i\in {1,2,...n}$ on its corresponding columns, (i.e. $B^i = T(i)A^i)$. However, the LinearLayer[] in mathematica will apply to all the columns. It's possible to use n PartLayer[] to extract every column in $A$, but that may not work well when $n$ is big. Is there any easy way to implement this?
My solution looks like this:enter image description here First time using StackExchange, thank you so much :)

  • 1
    $\begingroup$ Are you using NetChain? $\endgroup$
    – flinty
    Apr 10 at 15:33
  • $\begingroup$ I'm using a NetGraph consists of PartLayer[1], PartLayer[2], PartLayer[3]..., and n LinearLayer[p], and each PartLayer[i] is connected with a LinearLayer[p]. $\endgroup$
    – user79372
    Apr 10 at 15:54

This might not be the most compact or elegant to represent the NetGraph, but at least the size of the diagram is constant no matter how large the parameters n,m and p are.

trynet = With[{p = 5, n = 200, m = 100}, With[{},
     "WeightsLayer" -> 
      NetArrayLayer["Output" -> {Automatic, p, Automatic}],
     "Transpose" -> TransposeLayer[],
     "T Layer" -> 
           InputPorts -> <|"Weights" -> Automatic, 
             "Input" -> Automatic|>], <|"Weights" -> 1|>], 
         AggregationLayer[Total, 2]}, {NetPort["Input"] -> 
          NetPort[1, "Input"], 
         NetPort["Weights"] -> NetPort[1, "Weights"], 
         1 -> 2}], {"Input" -> 1, "Weights" -> 1}]
     |>, {
     "WeightsLayer" -> NetPort["T Layer", "Weights"],
     NetPort["Input"] -> "Transpose" -> NetPort["T Layer", "Input"]
     }, "Input" -> {m, n}]]]



Code autogeneration ^_^

dims = {10, 5};

n = 8; (*LinearLayer*)

 Join[Table[{PartLayer[{All, i}], LinearLayer[n]}, {i, dims[[2]]}], {CatenateLayer[]}],
 Join[Table[NetPort["Input"] -> i, {i, dims[[2]]}], {Range[dims[[2]]] -> dims[[2]] + 1}],
 "Input" -> dims

enter image description here


Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.