# Gradient of neural network output with respect to input

Context: I'm studying adversarial attacks on neural networks. These attacks are grouped into targeted, where a legitimate input is changed by some imperceptible amount and the new input is miss-classified by the network. E.g.

Source: Goodfellow IJ, Shlens J, Szegedy C. Explaining and Harnessing Adversarial Examples [Internet]. arXiv [stat.ML]. 2014. Available from: http://arxiv.org/abs/1412.6572

And to non-targeted, where you just want to come up with some random input that leads to a specific output, like:

A fast method for constructing a targeted-example is $$x_\text{adv} = x + \underbrace{\epsilon sign\left({\nabla_x J(x)}\right)}_{\text{perturbation factor}}$$, where $$x$$ is the legitimate input you target, $$\epsilon$$ some small number and $$J(x)$$ the cost as a function of input $$x$$.

For non-targeted attacks, the value of $$x_\text{adv}$$ can be found via gradient descent as the one that minimizes the following definition of cost function $$J$$, starting with some random value for $$x$$.

$$\newcommand{\norm}[1]{\left\lVert#1\right\rVert} J(x) = \frac{1}{2}\norm{y(x)-y_\text{adv}}_2^2 =\frac{1}{2}\norm{h_\Theta(x) - y_\text{adv}}_2^2$$

$$y_\text{adv}$$ is the goal value (e.g. $$y_\text{adv} = [0,0,0,1,0,0,0,0,0,0]$$ in the above image), $$h_\Theta(x)$$ is the output of the network for some input $$x$$ and $$x$$ gets updated with:

$$x_{j,\text{new}} = x_{j,\text{old}} - \alpha \frac{\partial }{\partial x_j}J(x)$$

Questions:

1. Is it true that net[x0, NetPortGradient["Input"]] returns the value of $$\frac{\partial }{\partial x}h_\Theta(x)$$ for some value $$x=x_0$$?

2. Is it normal for NetPortGradient[] to return values very close to zero as we go deeper into the network? Does this have anything to do with vanishing gradients, although that term usually refers to the gradient of loss function with respect to weights rather than input?

plg[x_, inp_] := ListPlot[#, Joined -> True, PlotRange -> All,
InterpolationOrder -> 1, Frame -> {True, True, False, False},
FrameLabel -> {"Layer number",
"Total@Abs@Gradient\nwrt to input"}, Filling -> Bottom] &@

Useful link on NetPortGradient[].