4
$\begingroup$

The famous 2013 paper from Zeiler and Fergus "Visualizing and Understanding Convolutional Networks" proposes a method to understand the behavior of CNN using one (or more) DeConv networks coupled with the original CNN.

The DeConv Nets used employs a set of unpooling and deconvolutional layers to reconstruct the features in the input image that are responsible for the activation of a given feature map in a given layer.

These, however, use "max location switches" to reverse the max-pooling operation. These are, if I'm correct, pooling layers who applies an ArgMax operation, allowing to map which are the positions the pooled maxima come from.

Unfortunately, PoolingLayer does not accept ArgMax as Function option.

Is it possible to workaround this limitation and extract the "max location switches" in another way? Or is there any other technique applicable in Mathematica to produce a visualization similar to the one proposed by Zeiler and Fergus to understand which are the features that activate a give layer?

$\endgroup$
  • $\begingroup$ Does this help you at all? $\endgroup$ – Carl Lange Apr 3 at 10:32
  • 1
    $\begingroup$ It may be helpful to give a precise example of the expected input and output :) $\endgroup$ – Carl Lange Apr 3 at 12:02
  • $\begingroup$ I think, the input would be the desired output of NetDrop[someNet, -n] and the output would be an argument that solves approximately NetDrop[someNet, -n][argument] == input and differ as less as possible from a given argumentOriginal. That is how I understand what is written in the paper. I think the mechanism is similar to the one used in Google's deep dream optimisation - but I am not aware of any MMA implementations of that. $\endgroup$ – MK. Apr 3 at 17:14
  • $\begingroup$ In the original paper linked, they use this kind of "reverse CNN" to understand which are the features in the original image that are responsible for the activation of a given feature map in a give layer. To do so, you need to create an "inverse network" starting from the point you want to study all the way back to the beginning. I'm trying to apply this to YOLO to understand how specialized are the features extracted in the finals layer to understand how deep I need to retrain it to repurpose it. $\endgroup$ – Luca Amerio Apr 5 at 9:37
  • $\begingroup$ It's quite hard to explain which output I expect. Let's say that I want to input an image into the original network and obtain as output an image showing which are the parts of the original image responsible for the activation of a given feature layer. This could be repeated for any feature in any layer. $\endgroup$ – Luca Amerio Apr 5 at 9:38

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Browse other questions tagged or ask your own question.