# Help find a bright object on Mars!

In today's news, scientists found a bright object on one of Curiosity's photos (it's near the bottom of the picture below). It's a bit tricky to find - I actually spent quite some time staring at the picture before I saw it.

The question, then, is how one can systematically search for such anomalies. It should be harder than famous How do i find Waldo problem, as we do not necessarily know what we are looking for upfront!

Unfortunately, I know next to nothing about image processing. Playing with different Mathematica functions, I managed to find a transformation which makes the anomaly more visible at the third image after color separation -- but I knew what I was looking for already, so I played with the numerical parameter for Binarize until I found a value (0.55) that separated the bright object from the noise nicely. I'm wondering how can I do such analysis in a more systematic ways.

img = Import["http://www.nasa.gov/images/content/694809main_pia16225-43_946-710.jpg"];
Colorize @ MorphologicalComponents @ Binarize[#, .55] & /@ ColorSeparate[img]


Any pointers would be much appreciated!

-
To me it doesn't look bright so much as it looks less brown than the surroundings. ColorSeparate[ ColorConvert[Import["http://i.stack.imgur.com/z2jmA.jpg"], "HSB"]][[2]] –  Rahul Oct 10 '12 at 4:06
Great tip, @RahulNarain - extracting saturation does certainly bring it up. I agree that it's not that bright. –  Victor K. Oct 10 '12 at 4:12
I should have included an image in my last comment: i.stack.imgur.com/cE26t.png –  Rahul Oct 10 '12 at 4:14
Keep in mind those images are photoshoped from the original B&W images they receive and analyse. Images only get color and non-fisheye compensantion when they are sending out press releases. –  gcb Oct 11 '12 at 0:53
@gcb: Maybe "Hazcam Left" is only for obstacle avoidance? The "mastcam" images on e.g. mars.jpl.nasa.gov/msl/multimedia/raw/?s=32 seem to be in color. They're fisheyed, though –  nikie Oct 17 '12 at 14:14

Here's another, slightly more scientific method. One that works for many kinds of anomalies (darker, brighter, different hue, different saturation).

First, I use a part of the image that only contains sand as my training set (I use the high-res image from the NASA site instead of the one linked in the question. The results are similar, but I get much saner probabilities without the JPEG artifacts):

img = Import["http://www.nasa.gov/images/content/694811main_pia16225-43_full.jpg"];
sandSample = ImageTake[img, {0, 200}, {1000, 1200}]


We can visualize the distribution of the R/G channels in this sample:

SmoothHistogram3D[sandPixels[[All, {1, 2}]], Automatic, "PDF",  AxesLabel -> {"R", "G", "PDF"}]


The histogram looks a bit skewed, but it's close enough to treat it as gaussian. So I'll assume for simplicity that the "sand" texture is a gaussian random variable where each pixel is independent. Then I can estimate it's distribution like this:

sandPixels = Flatten[ImageData[sandSample], 1];
dist = MultinormalDistribution[{mR, mG, mB}, {{sRR, sRG, sRB}, {sRG, sGG, SGB}, {sRB, sGB, sBB}}];
edist = EstimatedDistribution[sandPixels, dist];
logPdf = PowerExpand@Log@PDF[edist, {r, g, b}]


Now I can just apply the PDF of this distribution to the complete image (I use the Log PDF to prevent overflows/underflows):

rgb = ImageData /@ ColorSeparate[GaussianFilter[img, 3]];
p = logPdf /. {r -> rgb[[1]], g -> rgb[[2]], b -> rgb[[3]]};


We can visualize the negative log PDF with an appropriate scaling factor:

Image[-p/20]


Here we can see:

• The sand areas are dark - these pixels fit the estimated distribution from the sand sample
• Most of the Curiosity area in the image is very bright - it's very unlikely that these pixels are from the same distribution
• The shadows of the Curiosity probe are gray - they're not from the same distribution as the sand sample, but still closer than the anomaly
• The anomaly we're looking is very bright - It can be detected easily

To find the sand/non-sand areas, I use MorphologicalBinarize. For the sand pixels, the PDF is > 0 everywhere, for the anomaly pixels, it's < 0, so finding a threshold isn't very hard.

bin = MorphologicalBinarize[Image[-p], {0, 10}]


Here, areas where the Log[PDF] < -10 are selected. PDF < e^-10 is very unlikely, so you won't have to check too many false positives.

Final step: find connected components, ignoring components above 10000 Pixels (that's the rover) and mark them in the image:

components =
ComponentMeasurements[bin, {"Area", "Centroid", "CaliperLength"},
10 < #1 < 10000 &][[All, 2]]
Show[Image[img],
Graphics[{Red, AbsoluteThickness[5], Circle[#[[2]], 2 #[[3]]] & /@ components}]]


Obviously, the assumption that "sand pixels" are independent gaussian random variables is a gross oversimplification, but the general method would work for other distributions as well. Also, r/g/b values alone are probably not the best features to find alien objects. Normally you'd use more features (e.g. a set of Gabor filters)

-
Shut up and take my money! –  Soner Gönül Oct 10 '12 at 22:17
I created an account solely to upvote this wonderful answer –  kibibu Oct 11 '12 at 1:31
@kibibu - Welcome - we hope you stick around to enjoy more of what Mathematica has to offer. –  Verbeia Oct 11 '12 at 6:37
I was always wondering how stuff like this works and seeing all the formulas and their output makes the process seem understandable. Thank you! –  Dennis G Oct 11 '12 at 7:29

Let's define a filtering chain:

isolateTheSand[x_Image] := ColorNegate@
Dilation[Closing[EdgeDetect[EntropyFilter[x, 1], 10], 100], 30];

getBrightObjects[x_Image] := ColorSeparate[ (* Credit Rahul's comment *)
ColorConvert[ImageMultiply[x, isolateTheSand[x]], "HSB"]][[2]];

Binarize@getBrightObjects[x]), 10];



And now use it on your image:

getBrighAndFoolSand@Import["http://i.stack.imgur.com/z2jmA.jpg"]


Edit

It's always useful to be able to visualize the steps taken in an image transformation. Designing the process as a set of stages, each one resulting in a visible outcome helps a lot when debugging:

GraphicsRow[{#, isolateTheSand@#, getBrightObjects@#, makeMask@#, getBrighAndFoolSand@#} &@
Import["http://i.stack.imgur.com/z2jmA.jpg"]]


-
Excellent, thanks! –  Victor K. Oct 10 '12 at 7:51
@VictorK. You don't need to accept an answer so fast. Leaving the question open for (say) 48 hrs., encourages others to post more solutions –  belisarius Oct 10 '12 at 8:09
Yes, I now realized that - and I gave the accepted answer to @nikie, hope you don't mind :). –  Victor K. Oct 10 '12 at 22:03
@VictorK. Of course not. His answer is outstanding –  belisarius Oct 10 '12 at 22:10
@belisarius +1 for taking the time to get the GraphicsRow and visualizing each step, and naming them intuitively. This is how Image Processing should be explained! –  Ram Narasimhan Nov 30 '12 at 23:44

## protected by The Toad♦Oct 10 '12 at 21:16

Thank you for your interest in this question. Because it has attracted low-quality answers, posting an answer now requires 10 reputation on this site.