Applying TextRecognize on alpha-numerical table

Question

From time to time I have to extract data from scanned papers or rasterized PDFs. Imagine a table of the following kind (but much longer):

A naive application of TextRecognize on the image fails spectacularly:

"OOE-
20B-
70E~
20E-
90E-
70E-
70B-
30B-
30B-
ZOE
.50E+O2
.50E+02
.0OE+02
.O0E+O2
.00E+O2
.OOE+02
18E+O4
.30E+O3
.70E+03
10E+03

Did anybody use TextRecognize successfully on similar images?

Answer 1

Here is a way of extracting the positions of the various characters in your image by using ImageCorrelate.

Define the image to be worked on.

image = <your image goes here>

Define a set of kernel images to correlate with the image. I use {0,1,2,3,4,5,6,7,8,9,+,-,E,.} scaled to an appropriate size and then rasterised.

kernels =
  Map[
    (Style[#, FontSize -> 28, FontWeight -> Bold] // Rasterize // 
    ColorNegate // Binarize) &,
    Join[Range[0, 9], {"+", "-", "E", "."}]
  ]

Prepare the image for use in ImageCorrelate. Adding 0.01 to the 0 background avoids problems with it being “detected” - try omitting this step to see what happens.

imageneg = image // ColorNegate // ImageAdd[#, 0.01] &

Compute the CosineDistance image correlation for each kernel image. Threshold the results to detect instances of each kernel image, and Dilate the resulting small bright detections so you can easily see them.

imagecorrels = 
  Map[
    (ImageCorrelate[imageneg, #, CosineDistance] // ColorNegate //
     Binarize[#, 0.85] &) &,
    kernels
  ];

MapThread[Column[{#1, #2}] &, {Dilation[#, 3] & /@ imagecorrels, kernels}]

The results are very good for this set of kernel images. However, some of the “+” and “-“ are missed, which is probably because their kernel images differ too much from what appears in the image to be processed, and the “.” at the bottom of the image are missed, but this looks like an “edge effect” that would go away if you didn’t crop the image so tightly. Also, the “I” of “SI” is mistaken for a “1”.

The image processing could be tweaked to improve this performance. For instance, you could make the correlation results "compete" with each other to determine the "detection" at each image location - i.e. identify before detect - rather than simply threshold the correlations independently.

The locations of the detected image kernels could then be post-processed to create a representation of your image in terms of kernel images.

Answer 2

I get perfect reading using Tesseract using pagesegmode value 6.

I did this test after install Tesseract

SetDirectory@NotebookDirectory[];
Run["/usr/local/bin/tesseract hex.png hex -psm 6 "]

on as hex.png

and get

with no need to restrict character set (you can see how to do it here)

The only mistake are between 0 and O that can be easily corrected as @belisarius shows here.

Update

Now let's do it inside Mathematica. In this post from Wolfram Community, Ilian Gachevski shows me how to control page segmentation from inside Mathematica. It's as simple as:

i=Import[hex.png]
TextRecognize[i, "SegmentationMode" -> 6]

You can check that options are equivalent executing:

Map[Row[#,"  "]&,Image`ExternalOCRDump`$TextRecognizeSegmentationModes,{-2}]//TableForm

With all the comfort of doing all inside Mathematica. Just don't forget that this is an undocumented option.

Answer 3

As I've shown here, TextRecognize[] is a very basic tool and you usually need to tweak it to fit your needs.

In your case, you could do something like:

i = Binarize@ColorNegate@Import["https://i.sstatic.net/lP6Yt.jpg"];

(* Group the chars line by line *)
ifc = ImageForestingComponents[ColorNegate@i, Automatic, {First@ImageDimensions@i, 1}];

ClearAll[im];
(* A function for dropping blank spaces between letters *)
rule = {h__, {0 ..}, w : {0 ..}, {0 ..}, t__} -> {h, w, w, t};
im[x_] := Transpose[ Transpose[Replace[ifc, Except[x] -> 0, {2}] //. rule] //. rule];

(* French periodic tables are better understood :) *)
tr = TextRecognize[Image[im[#]], Language -> "French"] & /@  Range[2, Max@ifc];

(* Split column blocks *)
strs = {StringTake[#, {1, -17}], StringTake[#, {-16, -9}], StringTake[#, -8]} & /@ tr

$\left( \begin{array}{ccc} \text{H} & \text{6.00E-O9} & \text{3.5OE+O2} \\ \text{H2} & \text{l.20E-09} & \text{4.50E+02} \\ \text{I-lE} & \text{5.70E-08} & \text{l.00E+02} \\ \text{C} & \text{4.2OE-l2} & \text{8.00E+O2} \\ \text{N} & \text{3.90E-l2} & \text{8.00E+02} \\ 0 & \text{3.70E-12} & \text{B.00E+02} \\ \text{NA} & \text{6.70E-80} & \text{1.18E+04} \\ \text{MG} & \text{9.3OE-40} & \text{5.30E+03} \\ \text{SI} & \text{8.3OE-24} & \text{2.70E+03} \\ \text{S} & \text{4.20E-l4} & \text{1.10E+O3} \end{array} \right)$

You may see that there are problems (easy to fix knowing the context) with the number zero and the letter "o", and with the number eight and the letter "B". Also, one letter "H" got read as "I-1"

Now, Here you have a bunch of transformation rules designed to get the correct answer based upon the errors observed in TextRecognize[]

base = "(\\+|[0-9]|E|\\.|\-)";
MatrixForm[FixedPoint[
  StringReplace[#, {"\[Euro]" -> "E", "I-l" -> "H",
       WordBoundary ~~ "0" ~~ WordBoundary -> "O",
       RegularExpression[base ~~ "O"] :> "$1" ~~ "0",
   RegularExpression[base ~~ "B"] :> "$1" ~~ "8",
       RegularExpression["B" ~~ base] :> "8" ~~ "$1"}] & /@ # &, strs]] 

• $\left( \begin{array}{ccc} \text{H} & \text{6.00E-09} & \text{3.50E+02} \\ \text{H2} & \text{l.20E-09} & \text{4.50E+02} \\ \text{HE} & \text{5.70E-08} & \text{l.00E+02} \\ \text{C} & \text{4.20E-l2} & \text{8.00E+02} \\ \text{N} & \text{3.90E-l2} & \text{8.00E+02} \\ \text{O} & \text{3.70E-12} & \text{8.00E+02} \\ \text{NA} & \text{6.70E-80} & \text{1.18E+04} \\ \text{MG} & \text{9.30E-40} & \text{5.30E+03} \\ \text{SI} & \text{8.30E-24} & \text{2.70E+03} \\ \text{S} & \text{4.20E-l4} & \text{1.10E+03} \end{array} \right)$

For performance issues, please see the companion question Performance: Collapsing repeated contiguous rows & cols from a matrix