The documentation for NetPortGradient says

net[<|..., NetPortGradient[oport] -> ograd|>, NetPortGradient[iport]] can be used to impose a gradient at an output port oport that will be backpropogated to calculate the gradient at iport.

I understand that this introduces a constraint x = ograd. What exactly is x? The gradient of oport with respect to what?

## TL;DR

The NetPortGradient[oport] -> ograd is not

the gradient of oport with respect to what,

it's

the gradient of what with respect to oport.

## Over-simplified explanation

For a simple scalar example, suppose we have a neural network $$\mathrm{net}$$ mapping $$in$$ to $$out$$:

$$out=\mathrm{net}(in)\text{ ,}$$

and some post-computing $$f$$ (loss-function or following-up network, name it) involving $$out$$ as:

$$\mathcal{E}=f(out)\text{ .}$$

Then we have:

$$\frac{\mathrm{d}\,\mathcal{E}}{\mathrm{d}\,in}=\mathrm{net}'(in)\;\frac{\mathrm{d}\,\mathcal{E}}{\mathrm{d}\,out}\text{ .}$$

For back-propagation, we know in advance:

• At which particular value (say, $$in=3$$) of $$in$$ we are doing the computing.
• The numerical value (say, $$7$$) of the endpoint's gradient at $$in = 3$$, i.e. $$\left.\frac{\mathrm{d}\,\mathcal{E}}{\mathrm{d}\,out}\right|_{in=3}=7$$. Note that we don't necessarily need to know the "formula" or "computing method" of $$\frac{\mathrm{d}\,\mathcal{E}}{\mathrm{d}\,out}$$ in general way, just a numerical value 7 from whatever source we trust.

And we would like to know $$\left.\frac{\mathrm{d}\,\mathcal{E}}{\mathrm{d}\,in}\right|_{in=3}$$.

Now in Mathematica, that query is written as:

net[
<| "in" -> 3, NetPortGradient["out"] -> 7 |>,
]


When $$f$$ is the Identity function (i.e. $$\mathcal{E}=out$$), we have

net[<|"in"->3, NetPortGradient["out"]->1|>, NetPortGradient["in"]]


which is what the default way is computing (i.e. $$\left.\frac{\mathrm{d}\,out}{\mathrm{d}\,in}\right|_{in=3}$$):

net[<|"in"->3|>, NetPortGradient["in"]]


## Example

net = ElementwiseLayer[#^2 &];