Take the 2-minute tour ×
Mathematica Stack Exchange is a question and answer site for users of Mathematica. It's 100% free, no registration required.

In today's news, scientists found a bright object on one of Curiosity's photos (it's near the bottom of the picture below). It's a bit tricky to find - I actually spent quite some time staring at the picture before I saw it.

Bright object

The question, then, is how one can systematically search for such anomalies. It should be harder than famous How do i find Waldo problem, as we do not necessarily know what we are looking for upfront!

Unfortunately, I know next to nothing about image processing. Playing with different Mathematica functions, I managed to find a transformation which makes the anomaly more visible at the third image after color separation -- but I knew what I was looking for already, so I played with the numerical parameter for Binarize until I found a value (0.55) that separated the bright object from the noise nicely. I'm wondering how can I do such analysis in a more systematic ways.

img = Import["http://www.nasa.gov/images/content/694809main_pia16225-43_946-710.jpg"];
Colorize @ MorphologicalComponents @ Binarize[#, .55] & /@ ColorSeparate[img]

enter image description here

Any pointers would be much appreciated!

share|improve this question
5  
To me it doesn't look bright so much as it looks less brown than the surroundings. ColorSeparate[ ColorConvert[Import["http://i.stack.imgur.com/z2jmA.jpg"], "HSB"]][[2]] –  Rahul Narain Oct 10 '12 at 4:06
    
Great tip, @RahulNarain - extracting saturation does certainly bring it up. I agree that it's not that bright. –  Victor K. Oct 10 '12 at 4:12
3  
I should have included an image in my last comment: i.stack.imgur.com/cE26t.png –  Rahul Narain Oct 10 '12 at 4:14
1  
Keep in mind those images are photoshoped from the original B&W images they receive and analyse. Images only get color and non-fisheye compensantion when they are sending out press releases. –  gcb Oct 11 '12 at 0:53
1  
@gcb: Maybe "Hazcam Left" is only for obstacle avoidance? The "mastcam" images on e.g. mars.jpl.nasa.gov/msl/multimedia/raw/?s=32 seem to be in color. They're fisheyed, though –  nikie Oct 17 '12 at 14:14
show 11 more comments

2 Answers 2

up vote 197 down vote accepted

Here's another, slightly more scientific method. One that works for many kinds of anomalies (darker, brighter, different hue, different saturation).

First, I use a part of the image that only contains sand as my training set (I use the high-res image from the NASA site instead of the one linked in the question. The results are similar, but I get much saner probabilities without the JPEG artifacts):

img = Import["http://www.nasa.gov/images/content/694811main_pia16225-43_full.jpg"];
sandSample = ImageTake[img, {0, 200}, {1000, 1200}]

enter image description here

We can visualize the distribution of the R/G channels in this sample:

SmoothHistogram3D[sandPixels[[All, {1, 2}]], Automatic, "PDF",  AxesLabel -> {"R", "G", "PDF"}]

enter image description here

The histogram looks a bit skewed, but it's close enough to treat it as gaussian. So I'll assume for simplicity that the "sand" texture is a gaussian random variable where each pixel is independent. Then I can estimate it's distribution like this:

sandPixels = Flatten[ImageData[sandSample], 1];
dist = MultinormalDistribution[{mR, mG, mB}, {{sRR, sRG, sRB}, {sRG, sGG, SGB}, {sRB, sGB, sBB}}];
edist = EstimatedDistribution[sandPixels, dist];
logPdf = PowerExpand@Log@PDF[edist, {r, g, b}]

Now I can just apply the PDF of this distribution to the complete image (I use the Log PDF to prevent overflows/underflows):

rgb = ImageData /@ ColorSeparate[GaussianFilter[img, 3]];
p = logPdf /. {r -> rgb[[1]], g -> rgb[[2]], b -> rgb[[3]]};

We can visualize the negative log PDF with an appropriate scaling factor:

Image[-p/20]

enter image description here

Here we can see:

  • The sand areas are dark - these pixels fit the estimated distribution from the sand sample
  • Most of the Curiosity area in the image is very bright - it's very unlikely that these pixels are from the same distribution
  • The shadows of the Curiosity probe are gray - they're not from the same distribution as the sand sample, but still closer than the anomaly
  • The anomaly we're looking is very bright - It can be detected easily

To find the sand/non-sand areas, I use MorphologicalBinarize. For the sand pixels, the PDF is > 0 everywhere, for the anomaly pixels, it's < 0, so finding a threshold isn't very hard.

bin = MorphologicalBinarize[Image[-p], {0, 10}]

enter image description here

Here, areas where the Log[PDF] < -10 are selected. PDF < e^-10 is very unlikely, so you won't have to check too many false positives.

Final step: find connected components, ignoring components above 10000 Pixels (that's the rover) and mark them in the image:

components = 
 ComponentMeasurements[bin, {"Area", "Centroid", "CaliperLength"}, 
   10 < #1 < 10000 &][[All, 2]]
Show[Image[img], 
 Graphics[{Red, AbsoluteThickness[5], Circle[#[[2]], 2 #[[3]]] & /@ components}]]

enter image description here

Obviously, the assumption that "sand pixels" are independent gaussian random variables is a gross oversimplification, but the general method would work for other distributions as well. Also, r/g/b values alone are probably not the best features to find alien objects. Normally you'd use more features (e.g. a set of Gabor filters)

share|improve this answer
30  
Shut up and take my money! –  Soner Gönül Oct 10 '12 at 22:17
22  
I created an account solely to upvote this wonderful answer –  kibibu Oct 11 '12 at 1:31
1  
@kibibu - Welcome - we hope you stick around to enjoy more of what Mathematica has to offer. –  Verbeia Oct 11 '12 at 6:37
1  
I was always wondering how stuff like this works and seeing all the formulas and their output makes the process seem understandable. Thank you! –  Dennis G Oct 11 '12 at 7:29
add comment

Let's define a filtering chain:

isolateTheSand[x_Image] := ColorNegate@
                           Dilation[Closing[EdgeDetect[EntropyFilter[x, 1], 10], 100], 30];

getBrightObjects[x_Image] := ColorSeparate[ (* Credit Rahul's comment *)
                             ColorConvert[ImageMultiply[x, isolateTheSand[x]], "HSB"]][[2]];

makeMask[x_Image] := Dilation[ImageSubtract[#, DeleteSmallComponents@#] &@(ColorNegate@
                              Binarize@getBrightObjects[x]), 10];  

getBrighAndFoolSand[x_Image] := ImageMultiply[x, makeMask[x]];

And now use it on your image:

getBrighAndFoolSand@Import["http://i.stack.imgur.com/z2jmA.jpg"]

Mathematica graphics

Edit

It's always useful to be able to visualize the steps taken in an image transformation. Designing the process as a set of stages, each one resulting in a visible outcome helps a lot when debugging:

GraphicsRow[{#, isolateTheSand@#, getBrightObjects@#, makeMask@#, getBrighAndFoolSand@#} &@
                                                Import["http://i.stack.imgur.com/z2jmA.jpg"]]

Mathematica graphics

share|improve this answer
2  
Excellent, thanks! –  Victor K. Oct 10 '12 at 7:51
11  
@VictorK. You don't need to accept an answer so fast. Leaving the question open for (say) 48 hrs., encourages others to post more solutions –  belisarius Oct 10 '12 at 8:09
    
Yes, I now realized that - and I gave the accepted answer to @nikie, hope you don't mind :). –  Victor K. Oct 10 '12 at 22:03
6  
@VictorK. Of course not. His answer is outstanding –  belisarius Oct 10 '12 at 22:10
3  
@belisarius +1 for taking the time to get the GraphicsRow and visualizing each step, and naming them intuitively. This is how Image Processing should be explained! –  Ram Narasimhan Nov 30 '12 at 23:44
add comment

protected by rm -rf Oct 10 '12 at 21:16

Thank you for your interest in this question. Because it has attracted low-quality answers, posting an answer now requires 10 reputation on this site.

Would you like to answer one of these unanswered questions instead?

Not the answer you're looking for? Browse other questions tagged or ask your own question.