# 3D reconstruction of a 2D SEM image

I am curious if Mathematica could do reconstruction of a 2D SEM image into a 3D. It is the same what can be done by MountainsMap SEM software (http://www.digitalsurf.fr/en/mntsem.html)

Visually it looks like this:

I would appreciate any advice on this.

__

Update 1:

I am sorry that I wasn't specific with the procedure. The way I see implementation for such reconstruction of an SEM image is through the following steps:

• Turn the SEM image into a matrix with values from 0 to 255 (my image has 8-bit depth).
• Make a 3D plot where vertical axis corresponds to the values in the matrix.

__

Update 2:

I tried to follow Eisbär and MarcoB's suggestion with ListPlot3D. I cropped a part of the SEM image from above and applied ListPlot3D. Here is what I got:

ListPlot3D[ImageData@ColorConvert[image1, "Grayscale"],
ColorFunction -> "Rainbow", Mesh -> None, Boxed -> False,
BoxRatios -> {1, 1, 0.05}, PlotRange -> All]


It looks like a nice illusion of 3D reconstruction, but as MarcoB pointed out, this approach doesn't work. If anyone has suggestion for a better approach, please let me know.

In addition, here is the reconstruction result for the SEM image that is used as example by MountainsMap.

• This looks like an interesting question, but we need a lot more information to tackle it. At the very minimum, you should provide a 2D data set to start from. Commented Jan 26, 2017 at 22:09
• Isn't the right Hand side Image what you are looking for? It has the third Dimension Color coded. I would expect that you can Export the x, y, z data by the Software and then use e.g. ListPlot3D. Commented Jan 27, 2017 at 7:49
• Slava, in addition to what MarcoB asked you about, it would be helpful to provide a principle of how the image height is coded by SEM. Probably by a scale of the gray color, is it right? Then what scale of gray corresponds to what height? Commented Jan 27, 2017 at 7:58
• Usually, you get a Matrix in a text file of e.g. 256 columns and 256 rows. The value at each Position corresponds to the height at this Pixel. One only has to scale the Pixel pitch to distances. Commented Jan 27, 2017 at 9:18
• @ViacheslavPlotnikov I am not sure that the approach you propose would work as such: note that in your image lighter color does not always mean taller feature, or viceversa. You could do what you asked as follows: ListPlot3D[ImageData@ColorConvert[img, "Grayscale"], ColorFunction -> "SouthwestColors", Mesh -> None, Boxed -> False, Axes -> False], but that produces a "spiky" surface that may not be what you want yet. Commented Jan 27, 2017 at 18:48

So my core suggestion is that if you know someone with that software, you should have them churn out lots of these transformations for you and feed all that into a neural-net. At its heart, this is exactly the sort of problem a neural net should do well.

But in the meantime I'm gonna put in my few cents here (few though they are). I think much of your problem will be removing lighting artifacts. For instance, we have something like the "gray-level" drift mentioned in your link. We can correct that like so:

baseImg =
ImageTake[
Import[
"http://www.digitalsurf.fr/img/3D_enhancement_of_single_SEM_\
image.jpg"],
All,
{1, 360}
] // ImageCrop;

justSEM =
First@ColorSeparate@
ColorConvert[ImageTake[baseImg, 245, {10, -10}], "Grayscale"];

With[{wh = ImageDimensions[justSEM]},
LinearGradientImage[{{0, .3}*wh, {.6, .5}*wh} -> {GrayLevel[.45],
GrayLevel[0]},
wh
]
];

semPhaseCorrected =


Note the somewhat improved balance between the left and right halves of the image.

The next issue is to remove the lighting effects from the right-hand light source (or whatever it is in an SEM).

Hopefully someone else can chime in on that.

About your question,I think that is possible to made a reconstruction. But I am not sure if mountain map do acquisition of several images taken at different angles. In this case you can get a nice example in this link.

However, this example has an approximation that is not valid, it assumes that displacement is proportional to the Z high. The model must be improved, but in principle it is possible.

• I think the OP wanted to have something like the example here digitalsurf.fr/en/mntsem.html where it works from a single SEM. It'll have to be a relative thing, of course. And some corrections will be necessary. Commented Sep 13, 2017 at 6:27

Do nice but a method

imgdata = ImageData[ColorConvert[img, "Grayscale"]];
data = Rescale[imgdata, MinMax[imgdata], {1, 20}];
dim = Dimensions[data];

Graphics3D[
BSplineSurface[
Table[{i, j, data[[i, j]]}, {i, First[dim]}, {j, Last[dim]}]],Boxed->False]