20
$\begingroup$

I have a set of scanned documents with handwritten text which by default is printed with very low contrast. Here is an example (a part of a document):

img = Import["https://i.sstatic.net/sgRUc.jpg"]

image

The entire text is written in one pen. But when printing directly to a black&white laser printer with a grayscale conversion, I get a hardly readable result. I tried to improve the printing quality using ColorReplace, but it introduces artifacts:

{img2 = ImageTake[img, {158, 510}, {677, 1105}], 
 ColorReplace[img2, RGBColor[{169, 147, 209}/255.] -> Black, .15]}

output

Is there a better way to improve the quality of handwritten text without introducing artifacts?

Thinking: given that the problem is very common, maybe there are ready-made neural networks for this purpose?

$\endgroup$
6
  • $\begingroup$ Perhaps a combination of contrast correction, speckle removal and smoothing? Binarize[Blur[ColorNegate@DeleteSmallComponents[ColorNegate@Binarize[ImageAdjust[img, .2], .99], 100], 3], .8] You can play with the parameters to achieve a more desirable result ... $\endgroup$
    – Domen
    Commented Jul 27, 2022 at 11:54
  • $\begingroup$ @Domen The result isn't satisfactory, and I expect that the whole approach based on simple Binarize cann't give a good result. I would expect that a solution going through a LAB colorspace for separating the handwritten text would give something better. Also, I think there should be a neural net based approach for this purpose (since the problem is very common). $\endgroup$ Commented Jul 27, 2022 at 12:05
  • $\begingroup$ This is the result I get with my approach. Which features of the result do you still consider to be unsatisfactory? I can't read cyrillic, but compared to the original images (and to your approach), the result is much better. $\endgroup$
    – Domen
    Commented Jul 27, 2022 at 12:08
  • $\begingroup$ @Domen The result contains large gaps inside of the letters of the handwritten text. This is expected when using a Binarize-based approach. Actually, these gaps are absent in the original image: they just correspond to lesser-saturated color of the pen. $\endgroup$ Commented Jul 27, 2022 at 12:12
  • $\begingroup$ A great follow up question is why TextRecognize remains so bad after all these years…? $\endgroup$
    – M.R.
    Commented Jul 29, 2022 at 21:38

4 Answers 4

17
$\begingroup$

First attempt with acceptable result

Here is an approach which gives a satisfactory result. ColorsNear by default uses the "CIE2000" ColorDistance metric which works well in this case. The obtained image is readable and doesn't have too many artifacts. However, the result critically depends on a good choice for the penColor value, and requires manual adjusting the distance parameter.

penColor = {169, 147, 209}/255.;
(* This value is found by trial and error *)
distance = .16;
img2 = ColorDetect[img, ColorsNear[RGBColor[penColor], distance]]
mask = MorphologicalBinarize[ColorNegate[ImageApply[Min[#/penColor] &, img]]]
final = img*ColorNegate[img2] - mask

img2

mask

final

The inscription looks contrasting and, which is very important, at the same time naturally continuous, the noise level is acceptable.


Improved approach based on DominantColors

Here is an attempt to improve the algorithm.

The main disadvantage of the previous approach is that it cruicially depends on manually choosen value for the pen color. Let's employ DominantColors for finding this value automatically:

dc = DominantColors[img, 3]

dominant colors

The first color is background, the second is print color and the third is writing color.

Second, we can directly isolate the writing, the print and the background with ColorDetect. The "CIE76" ColorDistance metric in this case allows to achieve the same results as the "CIE2000", but is much faster:

(* This value is found by trial and error *)
dist = .219;
{back, print, writing} = ColorDetect[img, ColorsNear[#, dist, "CIE76"]] & /@ dc;

Surprisingly, some places where the writing overlaps itself and becomes most saturated are not recognized by ColorDetect as part of the writing, but fortunately are not recognized as part of the background either (same problem with the "CIE2000" metric):

ImageTake[#, {697, 754}, {923, 951}] & /@ {img, writing, back}

list

Due to this feature we should use back at the final stage for filling such holes in the writing as follows:

final = ColorNegate[writing + print + ColorNegate[back]]

final

The result is very similar to what we got with the previous method. The significant advantage of this approach is that it has only one parameter, which must be adjusted by trial and error.

However, the result can be improved further by clipping the grayscale values from the top:

Manipulate[ImageClip[
  ImageTake[final, {215, 520}, {850, 1232}], {0, thr2}, {0, 1}], 
   {{thr2, .82}, .9, .6, -.01}]

animation

The value thr2 = .82 seems optimal:

final2 = ImageClip[final, {0, .82}, {0, 1}]

final2

$\endgroup$
13
$\begingroup$

High contrast by "PrincipalComponentsAnalysis"

I use DimensionReduce[data,n] which projects the data into another space of dimension n. The base of that space is defined by Method -> "PrincipalComponentsAnalysis", orthogonal vectors that best fit the data.

I use Blur[img,5] to smooth the data, and noisy data is harder to classify. One could play with Dilation, FillingTransform and other transformations that fill space between data.

Because I'm choosing a space of dimension 3, the partitioned data is interpreted by Image as an RGB image.

The main component is assigned red, and the second component is green. The order is not necessarily consistent.

Because writing, print and background are predominantly similar colours, each of these should be given a primary colour. However, the print is black, which is a zero magnitude for any vector base, so it didn't get its own vector.

img  = Import["https://i.sstatic.net/sgRUc.jpg"]
img2 = Image@Partition[DimensionReduce[
    Flatten[ImageData[Blur[img,5]],1]
    , 3
    , PerformanceGoal -> "Quality"
    , Method -> "PrincipalComponentsAnalysis"
    
],3974]

enter image description here

$\endgroup$
8
  • $\begingroup$ I get a different result with your code $\endgroup$
    – yode
    Commented Jul 27, 2022 at 14:19
  • 1
    $\begingroup$ Could you provide some clarification on what Method -> "PrincipalComponentsAnalysis" actually produces? It's not clear why we have so many colors in the final image. $\endgroup$ Commented Jul 27, 2022 at 15:04
  • 1
    $\begingroup$ Thank you for the clarification. I've added another answer with new findings regarding the DimensionReduce approach. $\endgroup$ Commented Jul 27, 2022 at 19:09
  • $\begingroup$ Could you please show how to properly get the desired grayscale image from the output of DimensionReduce? $\endgroup$ Commented Aug 4, 2022 at 9:18
  • 1
    $\begingroup$ It certainly has value and I already upvoted it. I still think that there should be a way to get a proper grayscale image from the output of DimensionReduce, but I feel like it will end up being more complicated and less reliable than the approach I posted in my first answer. $\endgroup$ Commented Aug 5, 2022 at 8:21
6
$\begingroup$

When thinking on the DimensionReduce approach suggested by rhermans I realized that it would probably be better to work in a linear colorspace when performing dimension reduction, and also that we can directly request reduction to dimension 1 (what is formally our goal, since we are going to produce a grayscale image as a result). Then I looked at the obtained distribution of intensity values:

data = Rescale@Flatten@DimensionReduce[Flatten[ImageData[ColorConvert[img, "XYZ"]], 1], 1];
Histogram[Flatten[data], Automatic, "Probability", PlotRange -> {All, {0, .01}}]

histogram

From the histogram it is clear that we have a sum of three distributions with vastly different scales. Playing with clipping thresholds allows to produce quite interesting results which I still don't fully understand:

lowerThr = .16;
upperThr = .19;
dataClipped = Clip[data, {lowerThr, upperThr}, {0, 1}];
ColorNegate@Image@ArrayReshape[dataClipped, Reverse@ImageDimensions[img]]

final

The image obtained contains more artifacts, than I got with the previous method. However, I suspect that this approach can be improved for providing even better results.

$\endgroup$
2
$\begingroup$

Fully automatic quick and dirty approach:

background = First@DominantColors[img, ColorCoverage -> {.6, 1}];
cd = ColorDetect[img, ColorsNear[background, 0, "CIE76"]]

out

final = HistogramTransform[img, cd]

out

The amount of noise is high, but contrast and readability have increased dramatically.

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.