# How to extract metadata from an image of a business card?

I'm trying to digitize some documents, and I came across a very cool app called camscanner app which performs parallax transform and ocr very nicely, now I'm implementing it in mathematica...

Given a picture of a business card (taken perhaps at an angle) I'd like to read off the information. I'm trying to solve it in two steps:

1. Calculate the parallax using locators and PerspectiveTransformation[]
2. Clean up the image and OCR with TextRecognize[]

Here are sample images to work with:

• See this answer and the link immediately referenced: mathematica.stackexchange.com/a/13922/363 Jan 21, 2013 at 15:50
• Ah right, but I want to solve the general problem :)
– M.R.
Jan 21, 2013 at 15:58
• thx @ChrisDegnen I did see that before
– M.R.
Jan 21, 2013 at 16:00
• @M.R. How would one extract the info separately? TextRecognize seems simplistic and not useful. I've often wondered if there are any undocumented options for it... Jan 21, 2013 at 16:09
• also not to be missed: How to peel the labels from marmalade jars using Mathematica? Jan 21, 2013 at 19:05

Played with some image processing functions, get some rough procedure.

Import the test image:

img = Import["https://i.stack.imgur.com/H2Ksg.jpg"];


Do some gamma adjust to emphasize the edge:

img // ImageAdjust[#, {0, 0, 5}] &;


Draw rough edges:

GradientFilter[%, 2, "NonMaxSuppression" -> True] // ImageAdjust


Binarize and dilate it to form connected edges:

% // MorphologicalBinarize[#, {.1, .1}] & // Dilation[#, 1] &


Draw edges which are straight and long enough:

% // DeleteSmallComponents[#, 3200] &
EdgeDetect[%, 1, .1, "StraightEdges" -> 0.2] // DeleteSmallComponents[#, 300] &


Detect lines:

lines = ImageLines[%];
Show[img, Graphics[{Thick, Orange, Line /@ lines}]]


Extract corners of the card:

lineEqs = Cross[Append[{x, y} - #1, 0], Append[#2 - #1, 0]][[3]] & @@ # & /@ lines
corners = Select[
{x, y} /. Solve[Thread[# == 0], {x, y}][[1]] & /@
Subsets[lineEqs, {2}],
Norm[#] < 2000 &]

{{258.935, 624.228}, {904.807, 376.208}, {75.9044, 279.788}, {739.114, 5.80901}}


Extract the information piece:

correctedimg = With[{w = 900, h = 500},
transfunc =
FindGeometricTransform[{{0, h}, {w, h}, {0, 0}, {w, 0}}, corners][[
2]];
ImageCrop[
ImagePerspectiveTransformation[img, transfunc,
DataRange -> Full], {w, h}, Top]
]


infoPiece =
ImageCrop[
ImageCrop[correctedimg, {420, 260}, {Left, Center}], {350, Full},
Right], {5, .1, 1.2}]


Finally, do some OCR:

TextRecognize[infoPiece]


"TRAVIS HOWELL

Graphic + Web Designer

Q 1 23 456 7890

Q trvshowe!|@gmail.com

? www.TravisHD.com"

# Conclusion

Though the image processing procedure is very rough, the outcome image could be thought as fair good (at least true for specialised OCR software). So the left work, like Tom said in comment, seems to be about how to make TextRecognize working better.

• FWIW the original target used a OCR-friendly typeface that would make the final step significantly more accurate. Jan 21, 2013 at 20:05
• @Mr.Wizard IMO it's not that OCR-friendly. An OCR-friendly font (such as this and this) should have significant difference from one glyph to another. The fonts in the original target is hard to tell capital "I" from lowercase "l". Jan 22, 2013 at 13:58
• Nice work Silvia! Does anyone know of any free OCR libraries that one could call from mathematica?
– M.R.
Jan 23, 2013 at 0:24
• @M.R. Thanks M.R. I don't know any free OCR libs, but I think there should be some open-sourced ones. Jan 23, 2013 at 0:32
• @M.R. I'm thinking about a solution based on geometric properties (such as a constraint on aspect-ratio of the extracted rectangle) and image segmentation on textures formed by characters. OCR softwares usually use these tech to automatically compose the layout. I'll try and see if this is a right way. Jan 30, 2013 at 10:35