7
$\begingroup$

I'd like to use Mathematica to visualize the receptive field (RF) of given neuron in a given layer. I'd like to know if there is any built in functionality in version 11 to achieve this?

This paper by Torralba describes the RF and how they are computed in detail:

enter image description here

This is what the RF looks like for particular neurons from different layers of AlexNet:

enter image description here

And here's a link to Matlab code that computes receptive fields of units: http://places.csail.mit.edu/

Background:

Instead of having each neuron receive connections from all neurons in the previous layer, CNNs use a receptive field-like layout in which each neuron receives connections only from a subset of neurons in the previous (lower) layer. The receptive field of a neuron in one of the lower layers encompasses only a small area of the image, while the receptive field of a neuron in subsequent (higher) layers involves a combination of receptive fields from several (but not all) neurons in the layer before (i. e. a neuron in a higher layer "looks" at a larger portion of the image than does a neuron in a lower layer). In this way, each successive layer is capable of learning increasingly abstract features of the original image. The use of receptive fields in this fashion is thought to give CNNs an advantage in recognizing visual patterns when compared to other types of neural networks.

$\endgroup$
  • $\begingroup$ You might want to elaborate the term "receptive field" a little becuase in the context of ConvNets this term is used for the convolution kernel size. $\endgroup$ – Sascha Dec 20 '16 at 15:38
  • $\begingroup$ @Sascha Thanks, I added a reference for the explanation $\endgroup$ – user5601 Dec 20 '16 at 16:10
8
$\begingroup$

The receptive field usually refers to that of the neurons in the convolutional neural network. A convolutional neural network usually has the pyramid like structure, where the width and height of the output become smaller and smaller (See the following image, taken from here).

We compress the information as the output becomes smaller, this means that one pixel (or neuron) in the output now corresponding to not a single pixel but a region of pixels in the input image. This region is called the receptive field of that pixel. Since the information of the input image is compressed more and more in the later layers of the network, we expect to see that the receptive field is larger for later layers.

enter image description here

One idea to measure the receptive field of a particular neuron is to look at the gradient of the output of that neuron with respect to the input. The gradient would be nonzero if the pixel is inside the receptive field and zero otherwise.

We now take a simple two layer neural network and try to calculate receptive field for the pixel with index {5,6} using this idea.

NetChain[{
  ConvolutionLayer[1, {5, 5}],
  PoolingLayer[{2, 2}, {2, 2}, "Function" -> Mean]
  }, "Input" -> {1, 30, 30}]

In order to calculate the gradient with respect to that pixel, we use PartLayer to extract the value and use NetPortGradient to calculate the gradient.

net = NetInitialize@NetChain[{
    ConvolutionLayer[1, {5, 5}],
    PoolingLayer[{2, 2}, {2, 2}, "Function" -> Mean],
    PartLayer[1],
    PartLayer[5],
    PartLayer[6]
    }, "Input" -> {1, 30, 30}]

Map[If[# == 0., 0., 1.] &, 
   net[Table[0., {1}, {30}, {30}], NetPortGradient["Input"]], {-1}][[
  1]] // MatrixPlot

enter image description here

The square shows the receptive field for that neuron.

We can do the same thing for a more complicated neural network. The following plots show the neuron with index {3,3} on every convolution and pooling layers in the VGG network:

vgg = NetModel["VGG-16 Trained on ImageNet Competition Data"];
vggcut = NetChain[
   Cases[Table[
     vgg[[n]], {n, 1, 31}], _ConvolutionLayer | _PoolingLayer]];
data = Table[
   Map[If[# == 0., 0., 1.] &, 
     NetChain[{Take[vggcut, {1, n}], PartLayer[1], PartLayer[3], 
        PartLayer[3]}][Table[0., {3}, {224}, {224}], 
      NetPortGradient["Input"]], {-1}][[1]]
   , {n, 2, 18}
   ];
Grid@Partition[ArrayPlot /@ data, UpTo[6]]

enter image description here

As expected, the receptive field for later layers is larger.

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.