How to add an extra output port to an existing NetGraph?

Question

I am using a pre-trained neural network which I downloaded from an ONNX repository. I would like to study the output of a given internal layer in the network.

Here is a simple dummy example of what I would like to achieve.

net = NetInitialize@NetGraph[{LinearLayer[2], Ramp, LinearLayer[3], Tanh,CatenateLayer[]}, {NetPort["Input1"] -> 1 -> 2, NetPort["Input2"] -> 3 -> 4, {2, 4} -> 5}, "Input1" -> 6, "Input2" -> 7];
Export["onnx_model.onnx", net];
ONNXnet = Import["onnx_model.onnx"]

The imported network from the ONNX file looks like this

I would now like to add an additional output to layer 2 such that the new network has an extra output port connected to layer 2 :

The only way I have found is by taking the imported network apart and then rebuild it with the extra link to the output port

layerlist=Normal[ONNXnet]

NetGraph[layerlist, {NetPort["Input1"] -> 1 -> 3,NetPort["Input2"] -> 2 -> 4, {3, 4} -> 5, 3 -> NetPort["Extra_port"]}]

In case of a simple network like the above example, this procedure is not too difficult. But when the original imported network has many layers and complicated links, rebuilding it becomes tedious and is error prone.

I have seen examples of new netgraphs being build on top of existing netgraphs but I don't know how to access a given layer of the existing netgraph being used. I only know how to connect something to the input and output ports of the existing NetGraph, not the internal layers. Something like the following does not work :

NetGraph[<|"net"->ONNXnet|>,{NetPort["net",2]->NetPort["Extra_Port"]}]

So, is there an easy way to connect an intermediate layer to a new output port in an imported network without having to rebuild it from its individual components? I am doing this on MMA 12.2

Answer 1

Taking a neural network apart and then re-assembling it with extra links from internal layers to new output ports can be easily done with the VertexList and EdgeList commands. Adding the extra port to layer 3 in the above example can be achieved by

NetGraph[VertexList[ONNXnet],Append[EdgeList[ONNXnet], 3 -> NetPort["Extra_port"]]]

The resulting NetGraph then looks like this

If you want to specify the layer to which you want to attached a new link by its name instead of layer number, used the following instead

NetGraph[Normal[ONNXnet],Append[EdgeList[ONNXnet], "some layer name" -> NetPort["Extra_port"]]]

I have tried this with more complex networks (e.g. RESNET-34) and I could easily add multiple new output ports to different internal layers using the same approach.