Context: I'm studying adversarial attacks on neural networks. These attacks are grouped into targeted, where a legitimate input is changed by some imperceptible amount and the new input is miss-classified by the network. E.g.

enter image description here Source: Goodfellow IJ, Shlens J, Szegedy C. Explaining and Harnessing Adversarial Examples [Internet]. arXiv [stat.ML]. 2014. Available from: http://arxiv.org/abs/1412.6572

And to non-targeted, where you just want to come up with some random input that leads to a specific output, like:

enter image description here

A fast method for constructing a targeted-example is $x_\text{adv} = x + \underbrace{\epsilon sign\left({\nabla_x J(x)}\right)}_{\text{perturbation factor}}$, where $x$ is the legitimate input you target, $\epsilon$ some small number and $J(x)$ the cost as a function of input $x$.

For non-targeted attacks, the value of $x_\text{adv}$ can be found via gradient descent as the one that minimizes the following definition of cost function $J$, starting with some random value for $x$.

$ \newcommand{\norm}[1]{\left\lVert#1\right\rVert} J(x) = \frac{1}{2}\norm{y(x)-y_\text{adv}}_2^2 =\frac{1}{2}\norm{h_\Theta(x) - y_\text{adv}}_2^2 $

$y_\text{adv}$ is the goal value (e.g. $y_\text{adv} = [0,0,0,1,0,0,0,0,0,0]$ in the above image), $h_\Theta(x)$ is the output of the network for some input $x$ and $x$ gets updated with:

$ x_{j,\text{new}} = x_{j,\text{old}} - \alpha \frac{\partial }{\partial x_j}J(x) $


  1. Is it true that net[x0, NetPortGradient["Input"]] returns the value of $\frac{\partial }{\partial x}h_\Theta(x)$ for some value $x=x_0$?

  2. Is it normal for NetPortGradient[] to return values very close to zero as we go deeper into the network? Does this have anything to do with vanishing gradients, although that term usually refers to the gradient of loss function with respect to weights rather than input?

    plg[x_, inp_] := ListPlot[#, Joined -> True, PlotRange -> All, 
     InterpolationOrder -> 1, Frame -> {True, True, False, False}, 
     FrameLabel -> {"Layer number", 
       "Total@Abs@Gradient\nwrt to input"}, Filling -> Bottom] &@
    Table[Total@Abs@Flatten@NetTake[x, k][inp, NetPortGradient["Input"]],
    {k, 1, NetInformation[x, "LayersCount"]}];

enter image description here

  1. If it is typical for the gradient of the last layer with respect to input to be practically zero, how would the update formulas work?

Useful link on NetPortGradient[].

  • $\begingroup$ MMA might not be the proper tool to use here, it doesn't have true support for GANs (yet..) $\endgroup$ – M.R. Jul 22 at 17:49

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Browse other questions tagged or ask your own question.