# How to detect crosses and circles in 60x60 raster images?

I have hand-written 60 pixel times 60 pixel squares. I need to detect whether they are empty, x or circle. TextRecognize function fails. Is there some other function to process this kind of raster images with text?

Harder example

Circles: (0,0..9), (0..5,0), (0..5,9), (5,0..9)

Crosses: (2,3..6), (4,4..5)

Empty: (1,1..8), (2,1..2), (3,1..8), (4,1..3), (4,6..8)

I try to summarize and help people to solve the harder puzzle. Work in progress. Have fun!

I. Preprocessing (example)

1.3. rotation

II. Testing

2.1. Further info about mathematical morphology and Mathematica's intro.

-
Can you add a few sample images, so potential answerers can test their answers? –  nikie Apr 18 '13 at 19:08
In general, ImageCorrelate and ComponentMeasurements might be worth a try. –  nikie Apr 18 '13 at 19:10
As to the failure of TextRecognize: I seem to remember having read somewhere it uses a dictionary approach and it is therefore better in recognizing words than separate characters. –  Sjoerd C. de Vries Apr 18 '13 at 20:47
@SjoerdC.deVries here –  The Toad Apr 19 '13 at 0:10
many of the approaches here will fail if the input is even a little sloppy, not closing the "O". –  george2079 May 15 '13 at 18:14

For robustness I think it would be best to apply multiple tests and check the results for agreement. Here is one simple test you could include in that suite:

f = MorphologicalEulerNumber[Blur @ #, 0.8] &;


The function should return 0 for X's and 1 for O's. Quoting the documentation:

MorphologicalEulerNumber[image] by default gives the total number of connected white regions in image, minus the number of black holes that occur inside those regions.

You can tune the parameter 0.8 according to the density of your images and the noise level.
You may also try other pre-filters besides Blur, such as MedianFilter.

Example:

imgs = Import /@
{"http://i.stack.imgur.com/gDzKy.png",
"http://i.stack.imgur.com/LMPQq.png"}

f /@ imgs


{0, 1}


f2 =
Switch[
{
MorphologicalEulerNumber @ #,
500 < Total[1 - ImageData[#], 2]
} & @ Binarize[# ~Blur~ 4, 0.8],
{1, True}, "O",
{0, True}, "X",
{_, False}, "",
_, "Err"
] &;

img = Import["http://i.stack.imgur.com/NbTGY.jpg"];
grid = ImagePartition[
ImageCrop[ImageRotate[img, 0.7 \[Degree]], {1180, 720}, {-0.15, 0.2}], {118, 119}];

Map[
ImageCompose[#, Rasterize[Style[f2@#, Red], Background -> None]] &,
grid, {2}
] // GraphicsGrid


-
coming up with MorphologicalEulerNumber to solve this particular problem is a stroke of genius... –  Stefan Apr 18 '13 at 20:59
Circles (0,2) and (5,5) returns me errors, requiring apparently better preprocessing: argument change in Binarize from 0.8 to 1 gives totally different results. I cannot understand why it worked you like that, I just copied your code -- why the difference? –  hhh Apr 18 '13 at 21:25
You could also use Euler's characteristic to differentiate different shapes: X has 1 and circle has 0. I am trying to think how it would work with harder shapes -- no Euler characteristic command in Mathematica, odd. –  hhh Apr 18 '13 at 21:36
@hhh Since I'm not getting the errors ("Err") from my code it is hard for me to debug. I'm surprised as I wouldn't expect any of these functions to change between versions. There are three parameters to the second method you might experiment with: the prefilter type and magnitude (here Blur and 4), the Binarize threshold, and the number of black pixels required to consider the square non-empty (here 500). –  Mr.Wizard Apr 19 '13 at 2:09
@Stefan Thanks! I don't get many of those. :^) –  Mr.Wizard Apr 19 '13 at 2:09

As I said in the comment, ComponentMeasurements is a easy and robust way to differentiate simple shapes.

Using your image and binarizing it:

img = ColorConvert[Import["http://i.stack.imgur.com/NbTGY.jpg"],
"Grayscale"];


ComponentMeasurements calculates a list of measurements for each connected component.

components =
ComponentMeasurements[
bin, {"Centroid", "Area", "FilledCircularity",


In this case, it calculates the centroid, area and the filled area/perimeter ratio compared to a circle. That's a very good measure to recognize circles. The EquivalentDiskRadius is just there for the display:

Show[img,
Graphics[
{
Thick,
{If[#[[3]] > 0.5, Red, Blue], Circle[#[[1]], #[[4]]]} & /@
components[[All, 2]]
}]]


Objects with FilledCircularity < 0.5 are displayed as blue circles, FilledCircularity > 0.5 red:

-
Too bad ComponentMeasurements isn't in version 7. It looks spiffy. :-) –  Mr.Wizard Apr 18 '13 at 20:28
Suppose differentiation btw o and #. The morphological euler number would fail. Define perimeter for # as total length of lines in it. Now the area/perimeter_# < area/perimeter_o so this measure probably better for differentation. Differentiating chars used in ASCII games such as @,#,u,&,x,o -- can be pretty tricky. Have to consider new methods for differentiation or more distinct chars, someone must have considered this earlier... –  hhh Apr 18 '13 at 21:55

This trick sometimes works:

i1 = Import["http://i.stack.imgur.com/gDzKy.png"];
i2 = Import["http://i.stack.imgur.com/LMPQq.png"];
TextRecognize@ImageAssemble[{i1, i2, i1, i2, i1, i2}]
(*
--> "XOXOXO"
*)


Use it by appending the unknown square to a bunch of known ones, and detecting the last character.

-
Ok, Ok ... and the string length in case the new char is blank. –  belisarius Apr 18 '13 at 19:12
I cannot understand why it does not work with one image like "TextRecognize@i1" or "TextRecognize@{i1}" but "TextRecognize@ImageAssemble[{i1,i2}]" works. –  hhh Apr 18 '13 at 19:31
@hhh TextRecognize is designed to work on words, not individual characters. See this, also answered by belisarius. –  Mr.Wizard Apr 18 '13 at 19:39
This method results into non-robust implementation at least in my Mathematica (one lower than newest), mathematica.stackexchange.com/questions/25250/…. Why? –  hhh May 15 '13 at 18:00
@hhh probably you're trying to match blank spaces. It is robust as long as you manage blank spaces by "hand" –  belisarius May 15 '13 at 18:02