# Image Processing: Finding Orientation and Position of Symmetry Axes

Background

I'm currently toying with Principles of Design by Charlotte Jirousek in order to learn some basics of image processing. As main principals she suggests Balance, Proportion, Rhythm, Emphasis and Unity. I'm particularly interested in Balance.

According to her definition: Balance is the concept of visual equilibrium, and relates to our physical sense of balance. It is a reconciliation of opposing forces in a composition that results in visual stability. Most successful compositions achieve balance in one of two ways: symmetrically or asymmetrically. Balance in a three dimensional object is easy to understand; if balance isn't achieved, the object tips over. To understand balance in a two dimensional composition, we must use our imaginations to carry this three dimensional analogy forward to the flat surface.

Symmetrical balance can be described as having equal "weight" on equal sides of a centrally placed fulcrum. It may also be referred to as formal balance. When the elements are arranged equally on either side of a central axis, the result is Bilateral symmetry. This axis may be horizontal or vertical. It is also possible to build formal balance by arranging elements equally around a central point , resulting in radial symmetry.

There is a variant of symmetrical balance called approximate symmetry in which equivalent but not identical forms are arranged around the fulcrum line.

Question

How can I find Orientation and Position of Symmetry Axes of the given figures?

Solution Directions

In fact this question meant to be a toy example for image processing. However, after some investigation this turns out to be a challenging subject in it self.See the link for related papers submitted and published in both computer vision and computer graphics as well as changing test images.

In this approach Gradient Orientation Histogram is used.

o = Flatten@ImageData@GradientOrientationFilter[imag, 3];
\[ScriptCapitalA] = WeightedData[o, w];
\[ScriptCapitalD] = HistogramDistribution[\[ScriptCapitalA]];
t = Table[{x, PDF[\[ScriptCapitalD], x]}, {x, -\[Pi]/2, \[Pi]/2, 0.05}];
Max[t\[Transpose][[2]]]


As explained in the paper this gives the axes. Also available in Mathematica GradientOrientationFilter examples.

• Another approach could be based on component in Mathematica which may require additional image processing.
• This question feels underspecified. – Rahul Dec 29 '12 at 5:24
• In fact this question meant to be a toy example for image processing but turns out to be a drop in the ocean. – s.s.o Dec 29 '12 at 23:01
• Without looking into this in detail, I can't see how the HistogramDistribution of orientations will be able to locate the center of symmetry. Doesn't this just find the orientations without placing them at the center of symmetry? Maybe I'm overlooking something... for the orientations, it looks like a clever idea though. – Jens Dec 29 '12 at 23:43
• The code above was just a small bit of the total process. 1. Perform a gradient operation on the image; 2. Obtain the gradient orientation histogram; 3. Search for the convolution peaks of the gradient orientation histogram by using the Fourier transform to obtain the orientations of symmetry axes; 4. Obtain sub-angular positional accuracy by fitting a parabola function, and finding the peak position analytically; 5. Perform symmetry check or evaluation about the obtained symmetry axes; 6. Obtain the position of this symmetry axis by using the center of mass of the object; and draw axes. – s.s.o Dec 29 '12 at 23:56
• @Jens just corrected the link. Above is the over all process explained as in the paper. – s.s.o Dec 29 '12 at 23:58

This could be understood as a physics problem. From mechanics we know that the center of mass is at a point of high symmetry, and if there are symmetry axes (under rotation), then the inertia tensor with respect to the center of mass will have symmetry axes (maybe not all of them - there may be more than there are dimensions) among its principal axes (i.e. eigenvectors).

So given an image, we first convert it to a 2d grid in which each pixel corresponds to a Cartesian coordinate location, and the pixel value is the equivalent of the mass. You have to decide how to translate the color values of the image into a mass distribution. The easiest is to use the brightness.

Given this matrix, the center of mass coordinate com can be obtained by forming the mass-weighted average of the pixel positions. With that, we can calculate the two-by-two matrix inertia using the standard formula for the inertia tensor.

The principal axes are obtained from Eigenvectors[inertia]. Since inertia is a real symmetric matrix by definition, the eigenvectors are orthogonal. One of the will be the symmetry axis, if there is one.

I'll program the moment of inertia calculation in the most straightforward way without optimizing it (e.g., I don't use the symmetry of inertia because it's not worth the time). Here is the function that takes a raster image and plots it together with the principal axes relative to the center of mass that were found:

findSym[img_] :=
Module[{imd, nx, ny, points, weights, mass, com, pointsCom, diag,
inertia, v1, v2},
imd = Transpose@
ImageData[RemoveAlphaChannel@ColorConvert[img, "Grayscale"]];
{nx, ny} = Dimensions[imd];
points = Tuples[{Range[nx], Reverse@Range[ny]}];
weights = Flatten[imd];
mass = Total[weights];
com = Total[points weights]/mass // N;
pointsCom = (Subtract[#, com] & /@ points) weights;
diag = Total[(#.#) & /@ pointsCom ];
inertia =
Table[KroneckerDelta[i, j] diag -
Total[(#[[i]] #[[j]]) & /@ pointsCom], {i, 1, 2}, {j, 1, 2}];
{v1, v2} = Eigenvectors[inertia];
Graphics[{Inset[Image[Transpose@imd], Automatic, Automatic,
Scaled[1]], Red, Point[com], Blue,
Line[{{com, nx/3 v1 + com}, {com, nx/3 v2 + com}}]},
PlotRange -> {{1, nx}, {1, ny}}]]


The given input is converted to a grayscale image without alpha channel, so we can extract from it a two-dimensional array of scalar (non-negative) values imd that represent our mass distribution.

In the output, I reconstruct the image from imd and add a red dot for the center of mass, and two blue lines showing the principal axes.

Now to test it, I'll import one of your images and rotate it arbitrarily:

g = Import["http://i.stack.imgur.com/lSEfx.gif"];
im = ImageResize[ImageRotate[g, -Pi/3], 150]


Here is the calculation:

findSym[im]


In more complicated pictures, you'd want to pre-process your image by filtering maybe a certain color or selecting only certain morphological components, but that is too wide a field as to be easily covered here.

It would probably be more elegant to reformulate the inertia definition in terms of Moment, but I'd do that only if you want to apply this idea to larger images. In general, you probably won't want very high resolution for the analysis anyway because the visual symmetries in art are determined by the coarse features.

Edit

After writing this, I also remembered PrincipalComponents which finds symmetry axes of an image as well. There is an example in the section "Neat Examples" that does an alignment with the symmetry axes very similar to what the home-made approach above achieves.

The reason the inertia tensor and PrincipalComponent approaches are related is that moments of inertia are basically a kind of mass-weighted "variance" of the position relative to an axis. However, if we put a smooth mass-weighting into the input of PrincipalComponents, we'll get an output that's distorted in shape according to this weighting. Therefore, the example in the documentation uses a binarized input image.

Here is that procedure, applied to the above example:

shape = Position[ImageData[Binarize[im]], 1, {2}];
ListPlot[PrincipalComponents[N@shape], AspectRatio -> Automatic,
PlotRange -> All]


So my inertia tensor approach above has some advantages because it can work more easily with a smooth "mass" distribution.

Edit 2

As mentioned in the comment, findSym relies on you to define a mass measure first, and it interprets the image that you pass to it by considering the brighter regions high in mass and the black regions to have zero mass. The latter means the region isn't part of the object for the purposes of the moment of inertia.

To show that this method does not rely on the object's shape as such (i.e., being a rectangle, e.g.), here I mutilate the test image, crop it and throw it onto a black background that bleeds into it, preserving only its internal reflection symmetry along the chandelier:

im = Rasterize[
Graphics[Inset[
ImageRotate[
ImageCrop[
SetAlphaChannel[g,
ColorNegate@
Binarize@
Erosion[Colorize[ClusteringComponents[Binarize@g]],
2]], {226, 270}, Bottom], Pi/3], {Right, Top}, {Right,
Top}], Background -> Black
], "Image"]


findSym[im]


This shows that the symmetry has been identified correctly even though the overall image doesn't have symmetry.

• +1 Very nice answer. Perhaps the equivalent code using PrincipalComponents[] could be a useful final touch. – Dr. belisarius Dec 29 '12 at 15:30
• @belisarius I added some more comments on that approach, but concluded that the direct inertia tensor approach is better, after all. – Jens Dec 29 '12 at 18:07
• Yep, I that was the root of my request. I also couldn't find an equivalent way using PrincipalComponents[] – Dr. belisarius Dec 29 '12 at 18:58
• I combined im with a image of a random field of similar size ranging between 0 and 0.5. The results are here !Mathematica graphics. I wonder if this method relies on the qualities of the uniform black ground on which the image of interest is placed ? – image_doctor Dec 29 '12 at 20:22
• @image_doctor Absolutely. The method depends on what you consider part of the object whose symmetry axis you're trying to find. That's why I said you have to process the image first, by defining the "mass" measure. In your case, use a threshold, or a filter, to define the object clearly. Then the mass distribution within the object remains as the only thing determining the axes. – Jens Dec 29 '12 at 20:51