22
$\begingroup$

I'm trying to digitize some documents, and I came across a very cool app called camscanner app which performs parallax transform and ocr very nicely, now I'm implementing it in mathematica...

enter image description here

Given a picture of a business card (taken perhaps at an angle) I'd like to read off the information. I'm trying to solve it in two steps:

  1. Calculate the parallax using locators and PerspectiveTransformation[]
  2. Clean up the image and OCR with TextRecognize[]

Here are sample images to work with:

enter image description here enter image description here enter image description here enter image description here enter image description here

Improve this question
$\endgroup$
6
  • $\begingroup$ See this answer and the link immediately referenced: mathematica.stackexchange.com/a/13922/363 $\endgroup$
    – Chris Degnen
    Commented Jan 21, 2013 at 15:50
  • 1
    $\begingroup$ Ah right, but I want to solve the general problem :) $\endgroup$
    – M.R.
    Commented Jan 21, 2013 at 15:58
  • $\begingroup$ thx @ChrisDegnen I did see that before $\endgroup$
    – M.R.
    Commented Jan 21, 2013 at 16:00
  • 1
    $\begingroup$ @M.R. How would one extract the info separately? TextRecognize seems simplistic and not useful. I've often wondered if there are any undocumented options for it... $\endgroup$
    – Tom Wellington
    Commented Jan 21, 2013 at 16:09
  • 3
    $\begingroup$ also not to be missed: How to peel the labels from marmalade jars using Mathematica? $\endgroup$
    – Yves Klett
    Commented Jan 21, 2013 at 19:05

1 Answer 1

Reset to default
21
$\begingroup$

Played with some image processing functions, get some rough procedure.

Import the test image:

img = Import["https://i.sstatic.net/H2Ksg.jpg"];

Do some gamma adjust to emphasize the edge:

img // ImageAdjust[#, {0, 0, 5}] &;

Draw rough edges:

GradientFilter[%, 2, "NonMaxSuppression" -> True] // ImageAdjust

Binarize and dilate it to form connected edges:

% // MorphologicalBinarize[#, {.1, .1}] & // Dilation[#, 1] &

Draw edges which are straight and long enough:

% // DeleteSmallComponents[#, 3200] &
EdgeDetect[%, 1, .1, "StraightEdges" -> 0.2] // DeleteSmallComponents[#, 300] &

Detect lines:

lines = ImageLines[%];
Show[img, Graphics[{Thick, Orange, Line /@ lines}]]

Mathematica graphics

Extract corners of the card:

lineEqs = Cross[Append[{x, y} - #1, 0], Append[#2 - #1, 0]][[3]] & @@ # & /@ lines
corners = Select[
  {x, y} /. Solve[Thread[# == 0], {x, y}][[1]] & /@
   Subsets[lineEqs, {2}],
  Norm[#] < 2000 &]

{{258.935, 624.228}, {904.807, 376.208}, {75.9044, 279.788}, {739.114, 5.80901}}

Extract the information piece:

correctedimg = With[{w = 900, h = 500},
  transfunc = 
   FindGeometricTransform[{{0, h}, {w, h}, {0, 0}, {w, 0}}, corners][[
    2]];
  ImageCrop[
   ImagePerspectiveTransformation[img, transfunc, 
    DataRange -> Full], {w, h}, Top]
  ]

Mathematica graphics

infoPiece = 
 ImageAdjust[
  ImageCrop[
   ImageCrop[correctedimg, {420, 260}, {Left, Center}], {350, Full}, 
   Right], {5, .1, 1.2}]

Mathematica graphics

Finally, do some OCR:

TextRecognize[infoPiece]

"TRAVIS HOWELL

Graphic + Web Designer

Q 1 23 456 7890

Q trvshowe!|@gmail.com

? www.TravisHD.com"

Conclusion

Though the image processing procedure is very rough, the outcome image could be thought as fair good (at least true for specialised OCR software). So the left work, like Tom said in comment, seems to be about how to make TextRecognize working better.

Improve this answer
$\endgroup$
19
  • 1
    $\begingroup$ FWIW the original target used a OCR-friendly typeface that would make the final step significantly more accurate. $\endgroup$
    – Mr.Wizard
    Commented Jan 21, 2013 at 20:05
  • $\begingroup$ @Mr.Wizard IMO it's not that OCR-friendly. An OCR-friendly font (such as this and this) should have significant difference from one glyph to another. The fonts in the original target is hard to tell capital "I" from lowercase "l". $\endgroup$
    – Silvia
    Commented Jan 22, 2013 at 13:58
  • $\begingroup$ Nice work Silvia! Does anyone know of any free OCR libraries that one could call from mathematica? $\endgroup$
    – M.R.
    Commented Jan 23, 2013 at 0:24
  • $\begingroup$ @M.R. Thanks M.R. I don't know any free OCR libs, but I think there should be some open-sourced ones. $\endgroup$
    – Silvia
    Commented Jan 23, 2013 at 0:32
  • 1
    $\begingroup$ @M.R. I'm thinking about a solution based on geometric properties (such as a constraint on aspect-ratio of the extracted rectangle) and image segmentation on textures formed by characters. OCR softwares usually use these tech to automatically compose the layout. I'll try and see if this is a right way. $\endgroup$
    – Silvia
    Commented Jan 30, 2013 at 10:35

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.