Improve scanned image with handwritten text for high-contrast printing: how it can be done with Mathematica?

Question

I have a set of scanned documents with handwritten text which by default is printed with very low contrast. Here is an example (a part of a document):

img = Import["https://i.stack.imgur.com/sgRUc.jpg"]

The entire text is written in one pen. But when printing directly to a black&white laser printer with a grayscale conversion, I get a hardly readable result. I tried to improve the printing quality using ColorReplace, but it introduces artifacts:

{img2 = ImageTake[img, {158, 510}, {677, 1105}], 
 ColorReplace[img2, RGBColor[{169, 147, 209}/255.] -> Black, .15]}

Is there a better way to improve the quality of handwritten text without introducing artifacts?

Thinking: given that the problem is very common, maybe there are ready-made neural networks for this purpose?

Answer 1

First attempt with acceptable result

Here is an approach which gives a satisfactory result. ColorsNear by default uses the "CIE2000" ColorDistance metric which works well in this case. The obtained image is readable and doesn't have too many artifacts. However, the result critically depends on a good choice for the penColor value, and requires manual adjusting the distance parameter.

penColor = {169, 147, 209}/255.;
(* This value is found by trial and error *)
distance = .16;
img2 = ColorDetect[img, ColorsNear[RGBColor[penColor], distance]]
mask = MorphologicalBinarize[ColorNegate[ImageApply[Min[#/penColor] &, img]]]
final = img*ColorNegate[img2] - mask

The inscription looks contrasting and, which is very important, at the same time naturally continuous, the noise level is acceptable.

---

Improved approach based on DominantColors

Here is an attempt to improve the algorithm.

The main disadvantage of the previous approach is that it cruicially depends on manually choosen value for the pen color. Let's employ DominantColors for finding this value automatically:

dc = DominantColors[img, 3]

The first color is background, the second is print color and the third is writing color.

Second, we can directly isolate the writing, the print and the background with ColorDetect. The "CIE76" ColorDistance metric in this case allows to achieve the same results as the "CIE2000", but is much faster:

(* This value is found by trial and error *)
dist = .219;
{back, print, writing} = ColorDetect[img, ColorsNear[#, dist, "CIE76"]] & /@ dc;

Surprisingly, some places where the writing overlaps itself and becomes most saturated are not recognized by ColorDetect as part of the writing, but fortunately are not recognized as part of the background either (same problem with the "CIE2000" metric):

ImageTake[#, {697, 754}, {923, 951}] & /@ {img, writing, back}

Due to this feature we should use back at the final stage for filling such holes in the writing as follows:

final = ColorNegate[writing + print + ColorNegate[back]]

The result is very similar to what we got with the previous method. The significant advantage of this approach is that it has only one parameter, which must be adjusted by trial and error.

However, the result can be improved further by clipping the grayscale values from the top:

Manipulate[ImageClip[
  ImageTake[final, {215, 520}, {850, 1232}], {0, thr2}, {0, 1}], 
   {{thr2, .82}, .9, .6, -.01}]

The value thr2 = .82 seems optimal:

final2 = ImageClip[final, {0, .82}, {0, 1}]

Answer 2

High contrast by "PrincipalComponentsAnalysis"

I use DimensionReduce[data,n] which projects the data into another space of dimension n. The base of that space is defined by Method -> "PrincipalComponentsAnalysis", orthogonal vectors that best fit the data.

I use Blur[img,5] to smooth the data, and noisy data is harder to classify. One could play with Dilation, FillingTransform and other transformations that fill space between data.

Because I'm choosing a space of dimension 3, the partitioned data is interpreted by Image as an RGB image.

The main component is assigned red, and the second component is green. The order is not necessarily consistent.

Because writing, print and background are predominantly similar colours, each of these should be given a primary colour. However, the print is black, which is a zero magnitude for any vector base, so it didn't get its own vector.

img  = Import["https://i.stack.imgur.com/sgRUc.jpg"]
img2 = Image@Partition[DimensionReduce[
    Flatten[ImageData[Blur[img,5]],1]
    , 3
    , PerformanceGoal -> "Quality"
    , Method -> "PrincipalComponentsAnalysis"
    
],3974]

Answer 3

When thinking on the DimensionReduce approach suggested by rhermans I realized that it would probably be better to work in a linear colorspace when performing dimension reduction, and also that we can directly request reduction to dimension 1 (what is formally our goal, since we are going to produce a grayscale image as a result). Then I looked at the obtained distribution of intensity values:

data = Rescale@Flatten@DimensionReduce[Flatten[ImageData[ColorConvert[img, "XYZ"]], 1], 1];
Histogram[Flatten[data], Automatic, "Probability", PlotRange -> {All, {0, .01}}]

From the histogram it is clear that we have a sum of three distributions with vastly different scales. Playing with clipping thresholds allows to produce quite interesting results which I still don't fully understand:

lowerThr = .16;
upperThr = .19;
dataClipped = Clip[data, {lowerThr, upperThr}, {0, 1}];
ColorNegate@Image@ArrayReshape[dataClipped, Reverse@ImageDimensions[img]]

The image obtained contains more artifacts, than I got with the previous method. However, I suspect that this approach can be improved for providing even better results.

Answer 4

Fully automatic quick and dirty approach:

background = First@DominantColors[img, ColorCoverage -> {.6, 1}];
cd = ColorDetect[img, ColorsNear[background, 0, "CIE76"]]

final = HistogramTransform[img, cd]

The amount of noise is high, but contrast and readability have increased dramatically.