# How to extract metadata from an image of a business card?

I'm trying to digitize some documents, and I came across a very cool app called camscanner app which performs parallax transform and ocr very nicely, now I'm implementing it in mathematica...

Given a picture of a business card (taken perhaps at an angle) I'd like to read off the information. I'm trying to solve it in two steps:

1. Calculate the parallax using locators and PerspectiveTransformation[]
2. Clean up the image and OCR with TextRecognize[]

Here are sample images to work with:

-
See this answer and the link immediately referenced: mathematica.stackexchange.com/a/13922/363 – Chris Degnen Jan 21 '13 at 15:50
Ah right, but I want to solve the general problem :) – M.R. Jan 21 '13 at 15:58
thx @ChrisDegnen I did see that before – M.R. Jan 21 '13 at 16:00
@M.R. How would one extract the info separately? TextRecognize seems simplistic and not useful. I've often wondered if there are any undocumented options for it... – Tom Wellington Jan 21 '13 at 16:09
also not to be missed: How to peel the labels from marmalade jars using Mathematica? – Yves Klett Jan 21 '13 at 19:05

Played with some image processing functions, get some rough procedure.

Import the test image:

img = Import["http://i.stack.imgur.com/H2Ksg.jpg"];


Do some gamma adjust to emphasize the edge:

img // ImageAdjust[#, {0, 0, 5}] &;


Draw rough edges:

GradientFilter[%, 2, "NonMaxSuppression" -> True] // ImageAdjust


Binarize and dilate it to form connected edges:

% // MorphologicalBinarize[#, {.1, .1}] & // Dilation[#, 1] &


Draw edges which are straight and long enough:

% // DeleteSmallComponents[#, 3200] &
EdgeDetect[%, 1, .1, "StraightEdges" -> 0.2] // DeleteSmallComponents[#, 300] &


Detect lines:

lines = ImageLines[%];
Show[img, Graphics[{Thick, Orange, Line /@ lines}]]


Extract corners of the card:

lineEqs = Cross[Append[{x, y} - #1, 0], Append[#2 - #1, 0]][[3]] & @@ # & /@ lines
corners = Select[
{x, y} /. Solve[Thread[# == 0], {x, y}][[1]] & /@
Subsets[lineEqs, {2}],
Norm[#] < 2000 &]

{{258.935, 624.228}, {904.807, 376.208}, {75.9044, 279.788}, {739.114, 5.80901}}


Extract the information piece:

correctedimg = With[{w = 900, h = 500},
transfunc =
FindGeometricTransform[{{0, h}, {w, h}, {0, 0}, {w, 0}}, corners][[
2]];
ImageCrop[
ImagePerspectiveTransformation[img, transfunc,
DataRange -> Full], {w, h}, Top]
]


infoPiece =
ImageCrop[
ImageCrop[correctedimg, {420, 260}, {Left, Center}], {350, Full},
Right], {5, .1, 1.2}]


Finally, do some OCR:

TextRecognize[infoPiece]


"TRAVIS HOWELL

Graphic + Web Designer

Q 1 23 456 7890

Q trvshowe!|@gmail.com

? www.TravisHD.com"

# Conclusion

Though the image processing procedure is very rough, the outcome image could be thought as fair good (at least true for specialised OCR software). So the left work, like Tom said in comment, seems to be about how to make TextRecognize working better.

-
FWIW the original target used a OCR-friendly typeface that would make the final step significantly more accurate. – Mr.Wizard Jan 21 '13 at 20:05
@Mr.Wizard IMO it's not that OCR-friendly. An OCR-friendly font (such as this and this) should have significant difference from one glyph to another. The fonts in the original target is hard to tell capital "I" from lowercase "l". – Silvia Jan 22 '13 at 13:58
Nice work Silvia! Does anyone know of any free OCR libraries that one could call from mathematica? – M.R. Jan 23 '13 at 0:24
@M.R. Thanks M.R. I don't know any free OCR libs, but I think there should be some open-sourced ones. – Silvia Jan 23 '13 at 0:32
@M.R. I'm thinking about a solution based on geometric properties (such as a constraint on aspect-ratio of the extracted rectangle) and image segmentation on textures formed by characters. OCR softwares usually use these tech to automatically compose the layout. I'll try and see if this is a right way. – Silvia Jan 30 '13 at 10:35