33
$\begingroup$

I have hand-written 60 pixel times 60 pixel squares. I need to detect whether they are empty, x or circle. TextRecognize function fails. Is there some other function to process this kind of raster images with text?

enter image description here

Harder example

Circles: (0,0..9), (0..5,0), (0..5,9), (5,0..9)

Crosses: (2,3..6), (4,4..5)

Empty: (1,1..8), (2,1..2), (3,1..8), (4,1..3), (4,6..8)

I try to summarize and help people to solve the harder puzzle. Work in progress. Have fun!

I. Preprocessing (example)

1.1. thread about getting grid from raster image

1.2. convexity fix

1.3. rotation

II. Testing

2.1. Further info about mathematical morphology and Mathematica's intro.

Improve this question
$\endgroup$
10
  • 2
    $\begingroup$ Can you add a few sample images, so potential answerers can test their answers? $\endgroup$
    – Niki Estner
    Commented Apr 18, 2013 at 19:08
  • 1
    $\begingroup$ In general, ImageCorrelate and ComponentMeasurements might be worth a try. $\endgroup$
    – Niki Estner
    Commented Apr 18, 2013 at 19:10
  • $\begingroup$ As to the failure of TextRecognize: I seem to remember having read somewhere it uses a dictionary approach and it is therefore better in recognizing words than separate characters. $\endgroup$
    – Sjoerd C. de Vries
    Commented Apr 18, 2013 at 20:47
  • $\begingroup$ @SjoerdC.deVries here $\endgroup$
    – rm -rf ♦
    Commented Apr 19, 2013 at 0:10
  • 1
    $\begingroup$ many of the approaches here will fail if the input is even a little sloppy, not closing the "O". $\endgroup$
    – george2079
    Commented May 15, 2013 at 18:14

3 Answers 3

Reset to default
33
$\begingroup$

For robustness I think it would be best to apply multiple tests and check the results for agreement. Here is one simple test you could include in that suite:

f = MorphologicalEulerNumber[Blur @ #, 0.8] &;

The function should return 0 for X's and 1 for O's. Quoting the documentation:

MorphologicalEulerNumber[image] by default gives the total number of connected white regions in image, minus the number of black holes that occur inside those regions.

You can tune the parameter 0.8 according to the density of your images and the noise level.
You may also try other pre-filters besides Blur, such as MedianFilter.

Example:

imgs = Import /@
  {"https://i.sstatic.net/gDzKy.png", 
   "https://i.sstatic.net/LMPQq.png"}

f /@ imgs

enter image description here

{0, 1}

The whole enchilada

f2 =
 Switch[
    {
      MorphologicalEulerNumber @ #,
      500 < Total[1 - ImageData[#], 2]
    } & @ Binarize[# ~Blur~ 4, 0.8],
    {1, True}, "O",
    {0, True}, "X",
    {_, False}, "",
    _, "Err"
 ] &;

img = Import["https://i.sstatic.net/NbTGY.jpg"];
grid = ImagePartition[
   ImageCrop[ImageRotate[img, 0.7 \[Degree]], {1180, 720}, {-0.15, 0.2}], {118, 119}];

Map[
 ImageCompose[#, Rasterize[Style[f2@#, Red], Background -> None]] &,
 grid, {2}
] // GraphicsGrid

enter image description here

Improve this answer
$\endgroup$
8
  • 4
    $\begingroup$ coming up with MorphologicalEulerNumber to solve this particular problem is a stroke of genius... $\endgroup$
    – Stefan
    Commented Apr 18, 2013 at 20:59
  • $\begingroup$ Circles (0,2) and (5,5) returns me errors, requiring apparently better preprocessing: argument change in Binarize from 0.8 to 1 gives totally different results. I cannot understand why it worked you like that, I just copied your code -- why the difference? $\endgroup$
    – hhh
    Commented Apr 18, 2013 at 21:25
  • $\begingroup$ You could also use Euler's characteristic to differentiate different shapes: X has 1 and circle has 0. I am trying to think how it would work with harder shapes -- no Euler characteristic command in Mathematica, odd. $\endgroup$
    – hhh
    Commented Apr 18, 2013 at 21:36
  • $\begingroup$ @hhh Since I'm not getting the errors ("Err") from my code it is hard for me to debug. I'm surprised as I wouldn't expect any of these functions to change between versions. There are three parameters to the second method you might experiment with: the prefilter type and magnitude (here Blur and 4), the Binarize threshold, and the number of black pixels required to consider the square non-empty (here 500). $\endgroup$
    – Mr.Wizard
    Commented Apr 19, 2013 at 2:09
  • 2
    $\begingroup$ @Stefan Thanks! I don't get many of those. :^) $\endgroup$
    – Mr.Wizard
    Commented Apr 19, 2013 at 2:09
19
$\begingroup$

As I said in the comment, ComponentMeasurements is a easy and robust way to differentiate simple shapes.

Using your image and binarizing it:

img = ColorConvert[Import["https://i.sstatic.net/NbTGY.jpg"], 
  "Grayscale"];
bin = ColorNegate@Binarize[ImageAdjust[GaussianFilter[img, 5]]]

ComponentMeasurements calculates a list of measurements for each connected component.

components = 
  ComponentMeasurements[
   bin, {"Centroid", "Area", "FilledCircularity", 
    "EquivalentDiskRadius"}, #2 > 100 &];

In this case, it calculates the centroid, area and the filled area/perimeter ratio compared to a circle. That's a very good measure to recognize circles. The EquivalentDiskRadius is just there for the display:

Show[img,
 Graphics[
  {
   Thick,
   {If[#[[3]] > 0.5, Red, Blue], Circle[#[[1]], #[[4]]]} & /@ 
    components[[All, 2]]
   }]]

Objects with FilledCircularity < 0.5 are displayed as blue circles, FilledCircularity > 0.5 red:

result

Improve this answer
$\endgroup$
2
  • $\begingroup$ Too bad ComponentMeasurements isn't in version 7. It looks spiffy. :-) $\endgroup$
    – Mr.Wizard
    Commented Apr 18, 2013 at 20:28
  • $\begingroup$ Suppose differentiation btw o and #. The morphological euler number would fail. Define perimeter for # as total length of lines in it. Now the area/perimeter_# < area/perimeter_o so this measure probably better for differentation. Differentiating chars used in ASCII games such as @,#,u,&,x,o -- can be pretty tricky. Have to consider new methods for differentiation or more distinct chars, someone must have considered this earlier... $\endgroup$
    – hhh
    Commented Apr 18, 2013 at 21:55
10
$\begingroup$

This trick sometimes works:

i1 = Import["https://i.sstatic.net/gDzKy.png"];
i2 = Import["https://i.sstatic.net/LMPQq.png"];
TextRecognize@ImageAssemble[{i1, i2, i1, i2, i1, i2}]
(*
--> "XOXOXO"
*)

Use it by appending the unknown square to a bunch of known ones, and detecting the last character.

Improve this answer
$\endgroup$
6
  • $\begingroup$ Ok, Ok ... and the string length in case the new char is blank. $\endgroup$
    – Dr. belisarius
    Commented Apr 18, 2013 at 19:12
  • $\begingroup$ I cannot understand why it does not work with one image like "TextRecognize@i1" or "TextRecognize@{i1}" but "TextRecognize@ImageAssemble[{i1,i2}]" works. $\endgroup$
    – hhh
    Commented Apr 18, 2013 at 19:31
  • 1
    $\begingroup$ @hhh TextRecognize is designed to work on words, not individual characters. See this, also answered by belisarius. $\endgroup$
    – Mr.Wizard
    Commented Apr 18, 2013 at 19:39
  • $\begingroup$ This method results into non-robust implementation at least in my Mathematica (one lower than newest), mathematica.stackexchange.com/questions/25250/…. Why? $\endgroup$
    – hhh
    Commented May 15, 2013 at 18:00
  • $\begingroup$ @hhh probably you're trying to match blank spaces. It is robust as long as you manage blank spaces by "hand" $\endgroup$
    – Dr. belisarius
    Commented May 15, 2013 at 18:02

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.