10
$\begingroup$

I am using a pre-trained neural network which I downloaded from an ONNX repository. I would like to study the output of a given internal layer in the network.

Here is a simple dummy example of what I would like to achieve.

net = NetInitialize@NetGraph[{LinearLayer[2], Ramp, LinearLayer[3], Tanh,CatenateLayer[]}, {NetPort["Input1"] -> 1 -> 2, NetPort["Input2"] -> 3 -> 4, {2, 4} -> 5}, "Input1" -> 6, "Input2" -> 7];
Export["onnx_model.onnx", net];
ONNXnet = Import["onnx_model.onnx"]

The imported network from the ONNX file looks like this

enter image description here

I would now like to add an additional output to layer 2 such that the new network has an extra output port connected to layer 2 : enter image description here

The only way I have found is by taking the imported network apart and then rebuild it with the extra link to the output port

layerlist=Normal[ONNXnet]

enter image description here

NetGraph[layerlist, {NetPort["Input1"] -> 1 -> 3,NetPort["Input2"] -> 2 -> 4, {3, 4} -> 5, 3 -> NetPort["Extra_port"]}]

enter image description here

In case of a simple network like the above example, this procedure is not too difficult. But when the original imported network has many layers and complicated links, rebuilding it becomes tedious and is error prone.

I have seen examples of new netgraphs being build on top of existing netgraphs but I don't know how to access a given layer of the existing netgraph being used. I only know how to connect something to the input and output ports of the existing NetGraph, not the internal layers. Something like the following does not work :

NetGraph[<|"net"->ONNXnet|>,{NetPort["net",2]->NetPort["Extra_Port"]}]

So, is there an easy way to connect an intermediate layer to a new output port in an imported network without having to rebuild it from its individual components? I am doing this on MMA 12.2

$\endgroup$
4
  • 1
    $\begingroup$ Please post the code for specifying your first NetGraph to facilitate a close example. $\endgroup$ Mar 3, 2021 at 18:50
  • $\begingroup$ I added a reworked dummy example with code. $\endgroup$
    – PeterB
    Mar 4, 2021 at 9:16
  • $\begingroup$ Interesting problem. This may be a limit of NetReplacePart. On Mathematica 12.1, doing something like rep = NetGraph[{ElementwiseLayer[Ramp]}, {1 -> NetPort["Output"]}]; NetReplacePart[net, 4 -> rep] works, but trying to put multiple outputs for the replacement part fails. For example: rep2 = NetGraph[ {ElementwiseLayer[Ramp]}, {1 -> NetPort["Output"], 1 -> NetPort["foo"]}]; NetReplacePart[net, 4 -> rep2] returns an error about the extra output port. $\endgroup$ Mar 4, 2021 at 14:36
  • $\begingroup$ Crossposted here. $\endgroup$ Mar 5, 2021 at 14:02

1 Answer 1

4
$\begingroup$

Taking a neural network apart and then re-assembling it with extra links from internal layers to new output ports can be easily done with the VertexList and EdgeList commands. Adding the extra port to layer 3 in the above example can be achieved by

NetGraph[VertexList[ONNXnet],Append[EdgeList[ONNXnet], 3 -> NetPort["Extra_port"]]]

The resulting NetGraph then looks like this enter image description here

If you want to specify the layer to which you want to attached a new link by its name instead of layer number, used the following instead

NetGraph[Normal[ONNXnet],Append[EdgeList[ONNXnet], "some layer name" -> NetPort["Extra_port"]]]

I have tried this with more complex networks (e.g. RESNET-34) and I could easily add multiple new output ports to different internal layers using the same approach.

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.