Processing images from NASA's Juno Jupiter Orbiter

Question

Question

Can you help me improve what I’ve done to create an enhanced-color picture from an image provided by NASA’s JunoCam? [My work is shown in the penultimate image of Jupiter in this post.]

I’d particularly like to know how to obtain better fine detail while significantly reducing grain—equal to or better than what can be seen in the image shown at the end (posted to the Juno site by David Marriott), which shows a very nice combination of detail, smoothness, and clarity, and is (IMO) pleasingly glossy. [By comparison, my current result has somewhat poorer fine detail, and much more grain.] But I'm open to seeing whatever anyone else comes up with and, more broadly, I’d like to learn of better approaches to this task than what I’ve used. I tried using variations on code shown in answers to other questions about image processing, but was unable to find anything that helped me with this problem.

I played with several different Mathematica image-processing functions, finally settling on the sequence given below. But I’m new to Mathematica’s image-processing capabilities, so I look forward to learning what more experienced hands can accomplish. [I'm particularly interested to see what can be done using the individual color channels, rather than the combined image I worked with.] I’d also be interested to learn how Mathematica’s capabilities, ease-of-use, and general suitability for this work compare with those of specialized image-processing software, like Photoshop.

Background Info. on JunoCam Images, and Test Case

JunoCam is the public-outreach wide-field visible-light camera on NASA’s Juno Jupiter orbiter. It uses a pushframe imager, which means it generates separate R, G, and B image strips that can then be assembled into spatially composited images. When you download, from the JunoCam website, the files associated with an imaging pass, you get a folder containing the following:

1) A “raw” png image containing all these individual strips.

2) Three assembled png images (R, G, and B), in which NASA has stitched together the strips for each color channel.

3) A “map projected” png image, which combines the three color-channel images into one.

More technical info. on JunoCam is available here; select "About JunoCam Images": https://www.missionjuno.swri.edu/junocam/processing

I suspect that essentially no one bothers working with #1, instead letting NASA do the work of stitching the images together. That’s certainly my choice. The options are then to work with either the individual R, G, and B files, or to work with the combined map projected file. I tried the former and had no luck, so instead used the latter.

For my test case, I’m using the Southern Timelapse Sequence, Mission Phase PERIJOVE 10, from 12/16/2017. It can be downloaded here: https://www.missionjuno.swri.edu/junocam/processing?id=JNCE_2017350_10C00038_V01

Additionally, here’s some nice general background on image-processing approaches for these files (using specialized image-processing software, not Mathematica) provided by Jim Plaxco on his Artsnova website: http://www.artsnova.com/Juno-JunoCam-Image-Processing.html

And there's also this from Jesse Dohmann at Wolfram itself: http://blog.wolfram.com/2018/01/12/slicing-silhouettes-of-jupiter-processing-junocam-images/

Mathematica Code

I renamed the map projected png image as “mapprojected.png”, and ran the following code. This yields the image shown under "Final Result."

img = Import["~path_to_file/mapprojected.png"]
imgS = ImageAdjust[Sharpen[img, 100], .4] (*Need to sharpen before converting to HSB, since sharpening doesn't work with HSB; note that here we're also increasing the contrast (using a value of 0.4)*)
imgH = ColorConvert[imgS, "HSB"];
imgA = ImageApply[# {1.03, 2.2, 1.1} &, imgH] (*Adjusting hue, saturation, and brightness, respectively*)
imgP = ImagePad[imgA, 600];
imgR = Rotate[imgP, -.23 Pi] (*Have to use Rotate instead of ImageRotate b/c the latter doesn't support HSB images: “”.*)

In summary, I significantly increased the sharpness, contrast, and color saturation, and modestly increased the brightness. Since I shifted the hue only slightly (towards blue), I'd characterize this as an enhanced-color, rather than false-color, image.

Other attempts:

I tried improving sharpness/reducing grain by instead using ImageDeconvolve (experimenting with various methods and kernels), as well as Blur (computing an unsharp mask), but none of these gave me an improved result over Sharpen:

ImageDeconvolve[img, Normalize[DiskMatrix[a], Total[#, 2] &], Method -> "Wiener"]
ImageDeconvolve[img, GaussianMatrix[a], Method -> "Wiener"]
ImageDeconvolve[img, GaussianMatrix[a]]
ImageDeconvolve[img, Normalize[DiskMatrix[a], Total[#, 2] &], Method -> {"Hybrid", "Preconditioned" -> True}, MaxIterations -> 500]

img2 = img - Blur[img, 100]; (*"compute unsharp mask"*)
img + 4 img2 (*"add unsharp mask to original image"*) (*quotes are from Wolfram documentation*)

Wolfram's documentation for ImageDeconvolve explains: "A kernel used for deconvolution is often referred to as a 'point spread function' (commonly abbreviated 'psf') and is assumed to model the blur whose removal from an image is being attempted. If the point spread function does not match the blur actually present in an image, deconvolution will fail to recover details and may even add spurious artifacts." Alas, I don't have any experience choosing appropriate blur kernels, so I just experimented blindly; but perhaps there is a way to analyze the image to determine which is the best method and kernel to use.

In addition, I tried first upsampling the image (experimenting with various methods), and then repeating all the above attempts but, again, didn't improve the result. [Interestingly, this changed the color and contrast, so I had to adjust for this in my subsequent code.]:

ImageResize[img, Scaled[4], Resampling -> {"CatmullRom"}]
ImageResize[img, Scaled[4], Resampling -> {"Lanczos", 20}]
ImageResize[img, Scaled[4], Resampling -> {"Connes", 8, 2}]

Starting Point: "Map Projected" Image Provided by NASA

Final Result

Here I took the image produced by the above code, and then cropped and padded the bottom. I had to do this final cropping & padding (cropping the bottom to get a straight line, and then padding the bottom with a black border) outside of Mathematica, rather than use ImageCrop, for the same reason I couldn’t use ImageRotate: ImageCrop doesn’t support HSB images. Please let me know if there’s a way to get around these limitations.

Comparison with Another Image Processing Approach

Here’s work done on the same image, that was posted to the Juno site by David Marriott (https://www.missionjuno.swri.edu/Vault/VaultOutput?VaultID=13520&t=1528468966). I don’t know what image-processing software he used. Note that his image has better fine detail than mine, yet is smoother and has significantly less grain:

Here's a detail shot comparing Marriott's work with mine, updated with the results from Carl Lange and halirutan:

Answer 1

You might find the functions ColorToneMapping and BrightnessEqualize useful, as well as modifying gamma in ImageAdjust. I'm not completely delighted with how my attempt has come out, but I suppose it's something. The "smoothness" you mention from the image you linked might be done using something like a WienerFilter.

attempt 1

From this code:

ImageAdjust[
 ColorToneMapping[
  BrightnessEqualize@
   ImagePad[ImageCrop[ImageRotate[img, -.23 Pi]], 20], 0.3], {.5, .1, 
  5}]

attempt 2

This has a more natural feel to it.

Rotate the image first:

juno = ImagePad[ImageCrop[ImageRotate[juno, -.22 Pi]], 20]

Now we're going to cut off the edges of the planet, since it seems like there are some weird artefacts there:

mask = Erosion[MorphologicalBinarize@juno, 10]
ja = ReplaceImageValue[juno, ColorNegate@mask -> 0]

Now, we're going to use ImageAdjust to bring the Luminosity into a nice place but leave the colours alone. This is a bit nicer than sharpening.

ja = ColorCombine[
  Flatten[
   {ImageAdjust[First@ColorSeparate[ja, "LAB"]],
    Rest@ColorSeparate[ja, "LAB"]}
   ],
  "LAB"]

Now we're going to get the HSB channels (I've fiddled with the contrast, brightness, and gamma to get strong results).

{h, s, b} = ColorSeparate[ImageAdjust[ja, {0.2, 0.1, 1.5}], "HSB"]

Now, we're going to modify the saturation channel. Since what we're really looking for are areas of interest, I figured you could use ImageSaliencyFilter to find those.

jae = ColorCombine[{h,ImageAdjust[ImageAdjust@ImageSaliencyFilter[s], {0.5, -0.1, 0.5}] - 0.3, b}, "HSB"]

We're basically recombining the HSB channels here, with a different saturation. I'd suggest playing around with the parameters here.

Answer 2

Let me give this a try and complement Carl's solution by using the single r,g,b-images. With the downloaded image-set, we first load the single channels, combine them, rotate them, and crop them correctly, and finally extract the HSB channels:

files = FileNames["*.png", {"/path/to/ImageSet/"}];
{hue, sat, lum} = 
  ColorSeparate[ImageCrop[ImageRotate[
     ColorCombine[Import /@ files[[{-1, 2, 1}]]], -.23 Pi]], "HSB"];

My goal is to show you how to enhance the planet's structure, re-map the colors and get rid of color-noise. I'm giving some tips for all three HSB channels, but nothing I will say is special knowledge or magic. So be warned.

Before we start, I'm creating a smooth mask for the planet, because I don't like artifacts that were introduce by the stitching.

mask = ImagePad[GaussianFilter[
  Erosion[Binarize[ImagePad[lum, 1], .1], 10], 15], -1]

Luminance channel

This is the place where you can do crazy stuff. In the luminance channel you should work out the details of the structure, enhance them, or even add details that you extracted by e.g. filters. The grayscale result must look awesome because this is what your eye will mainly recognize. Colors are only a nice add-on. Side note: The importance of the luminance channel is the reason why it is mostly unaffected by standard jpeg-compression. You can almost destroy the hue channel, but touch the luminance one bit too hard and you will see the artifacts in your image.

My suggestion here is to sharpen the image to enhance structures and adjust the image so that you get a good contrast and lighting. You can use this

Manipulate[
 lumCorr = ImageAdjust[Sharpen[lum, sharp], {c, b, g}],
 {{sharp, 30}, 0, 100},
 {{c, 0.725}, 0, 3},
 {{b, 0}, -1, 3},
 {{g, .9}, 0, 3}
]

I won't smooth this because it looks fine as it is. However, if you want to smooth it, you should rather play with TotalVariationFilter and friends instead of a GaussianFilter because you want to preserve edges and details.

Just to give you an idea what I mean by "adding details", let's add a bit of embossing to go really crazy. If you don't already know this, note that we can multiply and add images without the ugly wrapper functions. I'm using an embossing angle of 90 (from above) to not further enhance the stitching lines:

lumCorr = ImageEffect[lumCorr, {"Embossing", 3, 90}] + lumCorr - 1/2

Hue channel

The hue is a bit tricky, because it might be hard to estimate the output for the unaccustomed user. What we see in the original is the following

ImageHistogram[hue]

The colors include only a very narrow band. What we would like to do is to spread them apart so that we see shades of blue at the bottom and red in the upper part.

Just to show you what is possible, I'm going to use HistogramTransform instead of ImageAdjust. You could get similar results with the latter.

Manipulate[
 ImageHistogram@HistogramTransform[hue, NormalDistribution[mu, sig]],
 {{mu, .2}, 0, 1},
 {{sig, .1}, 0, 1}
]

However, this is not helpful and it should only show you what happens to the histogram when you move the slider. Far better is to look at the coloring

Manipulate[
 ColorCombine[{HistogramTransform[hue, NormalDistribution[mu, sig]], 
   sat, lumCorr}, "HSB"],
 {{mu, .5}, 0, 1},
 {{sig, .2}, 0, 1}
]

Don't pay attention to color-noise; just adjust the colors so that you like it. You should inspect if the swirls at the top get a nice different color and that the bottom contains a good mix of green an blue. I'm using

hMu = 0.102;
hSig = 0.286;

Optimally, you find values that don't expose the different colors for stitching stripes, but it goes a bit against the goal to spread the colors.

The final step is to smooth the hue channel enough to get rid of any noise and too much color detail. I'm simply using the following but feel free to try different filters:

hueCorr = 
 GaussianFilter[
  HistogramTransform[hue, NormalDistribution[hMu, hSig]], 5]

Saturation channel

I'm leaving the saturation channel untouched. However, I thought it might be nice to use less saturation in the outer regions. This will make these parts a bit grayish which (together with our smooth mask) will give kind of an atmosphere. Here is what we use:

satCorr = sat*mask + .2 GaussianFilter[Erosion[mask, 50], 200] - .2

Result

The result can be obtained by simply combining all channels. Note that we use the mask in the luminance as well:

ColorCombine[{hueCorr, satCorr, lumCorr*mask}, "HSB"]