I have a symmetric image img. For simplicity, I would use the below image as my example:

img = Rasterize@Graphics[{Red, Rectangle[]}]

As you can see, img is an image of a pure red square with four thin white margins. Now I put it into a neural network with a pooling layer only (and the PartLayer which is used to extract the red color), and then extract the gradient.

pool = NetChain[{PoolingLayer[10, "Input" -> "Image"], PartLayer[1]}]
grad = pool[img, NetPortGradient["Input"]];
gIm = grad // NetDecoder["Image"]

As you can see, the gradient image has a "black-L" at the margin.

The input image is symmetric, while the pooling layer (and the PartLayer) have no parameters to be trained. That means everything should be symmetric, but the gradient is asymmetric. Why?

Many thanks!