# Handwriting recognition with Mathematica

I am trying to teach Mathematica to read my handwriting. Since I don't want to ruin my reputation by showing my own handwriting, I am going to use a font called blackjack.

Lets say this is a paragraph I have written.

para = StringTake[ExampleData[{"Text", "OriginOfSpecies"}, "FormattedText"], 401]


INTRODUCTION.

When on board H.M.S. 'Beagle,' as naturalist, I was much struck with certain facts in the distribution of the inhabitants of South America, and in the geological relations of the present to the past inhabitants of that continent. These facts seemed to me to throw some light on the origin of species--that mystery of mysteries, as it has been called by one of our greatest philosophers.

First I am going to recognise an alphabet say $h$. (At some point in distant future I am thinking about keeping a sample file of actual handwritten alphabets.)

font = "blackjack";
text = Binarize@Rasterize@Style[para, Bold, 30, FontFamily -> font];
w = Binarize@Rasterize@Style["h", Bold, 30, FontFamily -> font];

x = ImageCorrelate[text, w, NormalizedSquaredEuclideanDistance];
w1 = ColorNegate[Binarize[x, 0.12]];
pos = loc[[All, 2, 1]]; Length[pos1]
Length[pos]
Show[text, Graphics[{Opacity[0.5], Red, Disk[#, 10] & /@ pos}], ImageSize -> 500]


Then I iterate over all alphabets, signs and digits.

alph = Join[ToUpperCase[#] & /@ Alphabet[], Alphabet[],
{".", ",", ";", ":", "-", "?"}, {"1", "2", "3", "4", "5", "6", "7", "8", "9", "0"}]

wlist = {};
Do[
w = Binarize@Rasterize@Style[abc, Bold, 30, FontFamily -> font];
x = ImageCorrelate[text, w, NormalizedSquaredEuclideanDistance];
w1 = ColorNegate[Binarize[x, 0.11]];
pos = loc[[All, 2, 1]];
If[Length[pos] > 0, AppendTo[wlist, {abc, pos}]],
{abc, alph}]


Then convert it to a machine readable font and use TextRecognize

newtext = Graphics[Block[{w = #[[1]], pos = #[[2]]},
Text[Style[w, 18], #] & /@ pos] & /@ wlist, ImageSize -> 700]


TextRecognize[newtext]


INTRQDUCTIQN

When on board H M S 'B glef 5 naiurallsl, I W5 mu h stru k wlth ceftaln fa ts ln the d|str|but|0n of the lnhabltants of South Amenca, and |n the geologncat r Iahons of the prsem lo the pal lnhdiltants of thal tnnllnem ThSe fa ts seemed to me to throw some ||ght on the orlgln of pec|s-- that myslety of mysletls, as || ha been called by ore ofour grealsl ph||os phefs

Now the question - How to improve this?

The major challenge is to identify all the alphabets. Some alphabets are missing. In some cases $c$ looks like $e$ etc. I was thinking about using different font families and create a list with Classify for better comparison - still not sure how good that would be.

The last part concerning TextRecognize probably can be improved by rearranging the positions of individual alphabets to avoid any overlapping.

• may be using the code in this place can help you make your final result more reliable? maybe this can solve the "c" "e" problem. – Wjx Jul 20 '16 at 8:51
• @Wjx, I don't think so. Text typed by a student from dictation is checked against the original - there is no original in my case to compare with. – Sumit Jul 20 '16 at 9:18
• Oh, yeah. I misunderstood that, then maybe we can uae some dictionary function to do so? – Wjx Jul 20 '16 at 9:58
• So we do know the size of the text in the image and the image is perfect? It seems that your code work poor when dealing with even slightly tilted image or slightly different sizes. – Wjx Jul 22 '16 at 8:04
• My initial idea was to make it more general. For starter, we can assume a perfect version. I had the problem with size and orientation in the beginning, and I was trying to handle that with slight variations of each alphabet combined with Classify. – Sumit Jul 22 '16 at 11:22