# How do I remove grid from this photo?

I found this cat a month ago, and I'm not his owners' friend so the mosquito grid was the inevitable problem. I would like to post-process this photo to get rid of it, but I don't know exactly how this should be done and it would be interesting to see what can you do with this.

UPD: link to higher resolution (4928x3264) and quality (fixed) (fixed again after Google Drive probably disabled direct links)

• Maybe find the Fourier transform of the mosquito grid and subtract that from the frequency domain? It does seem to be quite a regular pattern. Jul 17, 2015 at 14:50
• My immediate thought is to use Inpaint, but then the problem becomes: how do I find an appropriate mask for Inpaint... Jul 17, 2015 at 14:58
• Somewhat ... i1 = Import["http://i.stack.imgur.com/XroGQ.jpg"]; truncate[data_, f_] := Module[{i, j}, {i, j} = Floor[Dimensions[data]/Sqrt[f]]; PadRight[Take[data, i, j], Dimensions[data], 0.] ]; id = Transpose[ImageData[i1, "Byte"], {3, 2, 1}]; t = FourierDCT /@ ((256 - #) & /@ id); fdct = FourierDCT[truncate[#, 50], 3] & /@ t; rfdct = Round[fdct]; ImageReflect[ ColorCombine[Image[#, "Byte"] & /@ ((256 - #) & /@ rfdct)], Left -> Top] Jul 17, 2015 at 15:47
• The above by using FourierDCT docs example Jul 17, 2015 at 15:47
• That cat looks pissed.
– kale
Jul 17, 2015 at 16:08

Here's a crude first attempt: First find the mosquito grid using RidgeFilter

img = Import["http://i.stack.imgur.com/XroGQ.jpg"];


(Note that I'm using ColorConvert after RidgeFilter, so RidgeFilter can find ridges in all color channels. Since RidgeFilter is nonlinear, the order makes a difference.)

Next, binarize with a low threshold to get a mask:

mask = MorphologicalBinarize[ridges, {0.05, 0.5}]


And finally: use Inpaint magic (where Diffusion is a compromise between quality and time):

Inpaint[img, mask, Method -> "Diffusion"]


I've played around with a few alternatives for mask, but none of them produced significantly better results, so I'm sticking with the KISS version. Maybe someone else can use this as a basis for a better reconstruction.

ADD In response to @Rahul's comment, here's a different mask that removes more of the grid, and also darker parts of the grid.

I'm using two separate LoG filters for the X- and Y-parts of the grid

logX = ImageData@LaplacianGaussianFilter[img, {50, {1, 20}}];
logY = ImageData@LaplacianGaussianFilter[img, {50, {20, 1}}];


I then use the square (to get dark and bright details)...

{logX, logY} = Map[Total, #^2, {2}] & /@ {logX, logY};


and rescale the resulting grid with the "average grid brightness" in the area, to get a more or less homogeneous image of the grid:

{logX, logY} =
Rescale[#/(GaussianFilter[#, 10] + 10^-10)] & /@ {logX, logY};

grid = Image[Rescale@(logX + logY)];


which I then binarize:

mask = MorphologicalBinarize[Image@grid, {0.15, 0.5}]


and use for inpainting:

res = Inpaint[img, Dilation[mask, 1], Method -> "Diffusion"]


A zoom on the cat's face shows that the grid is mostly gone:

ImageTrim[res, {{1130, 630}}, 200]


but so are details of the whiskers, and every edge in the image has "grid-shaped artifacts" from the inpainting.

• the cat looks well satisfied, +1.
– user9660
Jul 17, 2015 at 17:24
• That's a pretty fine result for a "first crude attempt"! +1, despite the first sub-result looking like Satan's cat ;-)
– ciao
Jul 17, 2015 at 21:39
• Well, if you zoom in, you can still see the grid, and the whiskers look more like "dashed" lines. Room for improvement, but I don't think you can get much further with Inpaint Jul 18, 2015 at 6:25
• I think the reason the grid is still left is because RidgeFilter looks for thin bright ridges, so in the regions where the grid is darker than the background, RidgeFilter is selecting parts not in the grid to be part of the mask. For illustration, take a look at ImageMultiply[img, mask] and zoom in on the white window frame behind the cat. Not sure what would be a better way of doing it though.
– user484
Jul 22, 2015 at 3:46
• Thank you for answer. You might be willing to apply and tweak your solution to the same photo with higher resolution and quality. I've used Google Drive hosting instead of default Imgur -- see question UPD. Jul 23, 2015 at 10:34

I tried the FFT idea just to see what it looks like. Here is a very quick and dirty version

t = 80;
fft = Table[Fourier[img[[All, All, x]]], {x, 1, 3}];
fft[[All, t ;; 1356 - t, All]] = 0;
fft[[All, All, t ;; 2048 - t]] = 0;
red = Image[Re[InverseFourier[fft[[1, All, All]]]]];
green = Image[Re[InverseFourier[fft[[2, All, All]]]]];
blue = Image[Re[InverseFourier[fft[[3, All, All]]]]];