In today's news, scientists found a bright object on one of Curiosity's photos (it's near the bottom of the picture below). It's a bit tricky to find - I actually spent quite some time staring at the picture before I saw it.

Bright object

The question, then, is how one can systematically search for such anomalies. It should be harder than famous How do i find Waldo problem, as we do not necessarily know what we are looking for upfront!

Unfortunately, I know next to nothing about image processing. Playing with different Mathematica functions, I managed to find a transformation which makes the anomaly more visible at the third image after color separation -- but I knew what I was looking for already, so I played with the numerical parameter for Binarize until I found a value (0.55) that separated the bright object from the noise nicely. I'm wondering how can I do such analysis in a more systematic ways.

img = Import["http://www.nasa.gov/images/content/694809main_pia16225-43_946-710.jpg"];
Colorize @ MorphologicalComponents @ Binarize[#, .55] & /@ ColorSeparate[img]

enter image description here

Any pointers would be much appreciated!

  • 6
    $\begingroup$ To me it doesn't look bright so much as it looks less brown than the surroundings. ColorSeparate[ ColorConvert[Import["https://i.sstatic.net/z2jmA.jpg"], "HSB"]][[2]] $\endgroup$
    – user484
    Commented Oct 10, 2012 at 4:06
  • $\begingroup$ Great tip, @RahulNarain - extracting saturation does certainly bring it up. I agree that it's not that bright. $\endgroup$
    – Victor K.
    Commented Oct 10, 2012 at 4:12
  • 4
    $\begingroup$ I should have included an image in my last comment: i.sstatic.net/cE26t.png $\endgroup$
    – user484
    Commented Oct 10, 2012 at 4:14
  • 2
    $\begingroup$ Keep in mind those images are photoshoped from the original B&W images they receive and analyse. Images only get color and non-fisheye compensantion when they are sending out press releases. $\endgroup$
    – gcb
    Commented Oct 11, 2012 at 0:53
  • 1
    $\begingroup$ @gcb: Maybe "Hazcam Left" is only for obstacle avoidance? The "mastcam" images on e.g. mars.jpl.nasa.gov/msl/multimedia/raw/?s=32 seem to be in color. They're fisheyed, though $\endgroup$ Commented Oct 17, 2012 at 14:14

2 Answers 2


Here's another, slightly more scientific method. One that works for many kinds of anomalies (darker, brighter, different hue, different saturation).

First, I use a part of the image that only contains sand as my training set (I use the high-res image from the NASA site instead of the one linked in the question. The results are similar, but I get much saner probabilities without the JPEG artifacts):

img = Import["http://www.nasa.gov/images/content/694811main_pia16225-43_full.jpg"];
sandSample = ImageTake[img, {0, 200}, {1000, 1200}]

enter image description here

We can visualize the distribution of the R/G channels in this sample:

SmoothHistogram3D[sandPixels[[All, {1, 2}]], Automatic, "PDF",  AxesLabel -> {"R", "G", "PDF"}]

enter image description here

The histogram looks a bit skewed, but it's close enough to treat it as gaussian. So I'll assume for simplicity that the "sand" texture is a gaussian random variable where each pixel is independent. Then I can estimate it's distribution like this:

sandPixels = Flatten[ImageData[sandSample], 1];
dist = MultinormalDistribution[{mR, mG, mB}, {{sRR, sRG, sRB}, {sRG, sGG, SGB}, {sRB, sGB, sBB}}];
edist = EstimatedDistribution[sandPixels, dist];
logPdf = PowerExpand@Log@PDF[edist, {r, g, b}]

Now I can just apply the PDF of this distribution to the complete image (I use the Log PDF to prevent overflows/underflows):

rgb = ImageData /@ ColorSeparate[GaussianFilter[img, 3]];
p = logPdf /. {r -> rgb[[1]], g -> rgb[[2]], b -> rgb[[3]]};

We can visualize the negative log PDF with an appropriate scaling factor:


enter image description here

Here we can see:

  • The sand areas are dark - these pixels fit the estimated distribution from the sand sample
  • Most of the Curiosity area in the image is very bright - it's very unlikely that these pixels are from the same distribution
  • The shadows of the Curiosity probe are gray - they're not from the same distribution as the sand sample, but still closer than the anomaly
  • The anomaly we're looking is very bright - It can be detected easily

To find the sand/non-sand areas, I use MorphologicalBinarize. For the sand pixels, the PDF is > 0 everywhere, for the anomaly pixels, it's < 0, so finding a threshold isn't very hard.

bin = MorphologicalBinarize[Image[-p], {0, 10}]

enter image description here

Here, areas where the Log[PDF] < -10 are selected. PDF < e^-10 is very unlikely, so you won't have to check too many false positives.

Final step: find connected components, ignoring components above 10000 Pixels (that's the rover) and mark them in the image:

components = 
 ComponentMeasurements[bin, {"Area", "Centroid", "CaliperLength"}, 
   10 < #1 < 10000 &][[All, 2]]
 Graphics[{Red, AbsoluteThickness[5], Circle[#[[2]], 2 #[[3]]] & /@ components}]]

enter image description here

Obviously, the assumption that "sand pixels" are independent gaussian random variables is a gross oversimplification, but the general method would work for other distributions as well. Also, r/g/b values alone are probably not the best features to find alien objects. Normally you'd use more features (e.g. a set of Gabor filters)

  • 44
    $\begingroup$ Shut up and take my money! $\endgroup$ Commented Oct 10, 2012 at 22:17
  • 28
    $\begingroup$ I created an account solely to upvote this wonderful answer $\endgroup$
    – kibibu
    Commented Oct 11, 2012 at 1:31
  • 1
    $\begingroup$ @kibibu - Welcome - we hope you stick around to enjoy more of what Mathematica has to offer. $\endgroup$
    – Verbeia
    Commented Oct 11, 2012 at 6:37
  • 1
    $\begingroup$ I was always wondering how stuff like this works and seeing all the formulas and their output makes the process seem understandable. Thank you! $\endgroup$
    – Dennis G
    Commented Oct 11, 2012 at 7:29
  • 1
    $\begingroup$ As this is kind of old question it would be great to know how this answer would look today. Are there maybe some refinements build into mathematica nowadays ot what is the state of the art? $\endgroup$ Commented May 26, 2018 at 6:28

Let's define a filtering chain:

isolateTheSand[x_Image] := ColorNegate@
                           Dilation[Closing[EdgeDetect[EntropyFilter[x, 1], 10], 100], 30];

getBrightObjects[x_Image] := ColorSeparate[ (* Credit Rahul's comment *)
                             ColorConvert[ImageMultiply[x, isolateTheSand[x]], "HSB"]][[2]];

makeMask[x_Image] := Dilation[ImageSubtract[#, DeleteSmallComponents@#] &@(ColorNegate@
                              Binarize@getBrightObjects[x]), 10];  

getBrighAndFoolSand[x_Image] := ImageMultiply[x, makeMask[x]];

And now use it on your image:


Mathematica graphics


It's always useful to be able to visualize the steps taken in an image transformation. Designing the process as a set of stages, each one resulting in a visible outcome helps a lot when debugging:

GraphicsRow[{#, isolateTheSand@#, getBrightObjects@#, makeMask@#, getBrighAndFoolSand@#} &@

Mathematica graphics

  • 2
    $\begingroup$ Excellent, thanks! $\endgroup$
    – Victor K.
    Commented Oct 10, 2012 at 7:51
  • 13
    $\begingroup$ @VictorK. You don't need to accept an answer so fast. Leaving the question open for (say) 48 hrs., encourages others to post more solutions $\endgroup$ Commented Oct 10, 2012 at 8:09
  • $\begingroup$ Yes, I now realized that - and I gave the accepted answer to @nikie, hope you don't mind :). $\endgroup$
    – Victor K.
    Commented Oct 10, 2012 at 22:03
  • 8
    $\begingroup$ @VictorK. Of course not. His answer is outstanding $\endgroup$ Commented Oct 10, 2012 at 22:10
  • 6
    $\begingroup$ @belisarius +1 for taking the time to get the GraphicsRow and visualizing each step, and naming them intuitively. This is how Image Processing should be explained! $\endgroup$ Commented Nov 30, 2012 at 23:44

Not the answer you're looking for? Browse other questions tagged or ask your own question.