# How to straighten a image with some text

Of course I know there are some related post about this topic,such as How to align rotated images vertically.And I have answered that post with two different method here and here.

But in this post,the case will be more precise,which have this features. The image always have a little lean on horizontal direction.it is about ±10°.

This is current method based on GradientOrientationFilter

img=Import["https://i.sstatic.net/cCjyn.png"];
Binarize[Dilation[ColorNegate[Binarize[img]], {Array[1 &, 15]}]],1]]


Caculate the angle

angle = -Mean[Select[Catenate[ImageData[gradImg]], Abs[#] < 0.8 &]]


Get the result

ImageRotate[img, angle]


Of course this code can serve another image.

# Complaint about the current method:

1. I don't know why I should select those value with a threshold 0.8.Actually N[20 °]==0.35,but if I use 0.35,I will get a bad result
2. This is not a efficient method when the processed image have a big size
3. I don't like the Dilation,because which mean I should adjust the threshold 15 for different image.

Actually I allways really don't hope any threshold value(or as little as possible) apprear in my solution,which will reduce the universality of the method. Sincely to request a better method here.

• Maybe overkill, but you could try building a neural net that returns the rotation angle since it's so easy to make training data. Commented Jul 28, 2017 at 18:17
• @ChipHurst So easy????
– yode
Commented Jul 29, 2017 at 1:47
• Can't you create the training data by doing ImageRotate[im_, RandomReal[{0,2Pi}] to a bunch of images of text? Commented Jul 29, 2017 at 1:52
• @ChipHurst Maybe it can treat different angle for a same image,but I don't sure it can treat different image.
– yode
Commented Jul 29, 2017 at 1:58
• @yode with enough data I figure you probably can get a good net for that sort of thing. Maybe you could make images of the entire contents of this or take pngs of stuff here (chopped into, say, 500 pixel columns) rotate them by arbitrary angles, and pass 80% of that into a net with some convolution laters as your training set, 20% as your test set, and see how it goes. Commented Jul 30, 2017 at 20:48

Expanding on Chip Hurst's comment I built out a sample net to try this out:

net =
NetInitialize@
NetChain[{
ConvolutionLayer[
32,
3,
"Dilation" -> 3
],
AggregationLayer[Mean],
LinearLayer[1]
},
"Input" -> NetEncoder[{"Image", {750, 750}, "ColorSpace" -> "RGB"}]
];


And I made my training data as such:

imgs =
Map[
ImagePartition[
Import[#],
{10000, 100}
] &,
FileNames["*.png", "~/Desktop/blee"]
] // Flatten;

rots =
ImageRotate[#, #2, Masking -> Full] -> {#2} &, {
Flatten@ConstantArray[imgs, 3],
RandomReal[2 \[Pi], 3*Length@imgs]
}];


where ~/Desktop/blee contains the first three notebooks from here exported as PNG.

Then I let it train for all of three minutes because I didn't really want to sink the time to train a high quality net and because I was sure my net was crap.

Then I found a better net (probably, haven't trained it) here from this post on community: http://community.wolfram.com/groups/-/m/t/1136388

net2 =
NetChain[
Append[
Normal[
NetReplacePart[
Drop[
NetModel[
"SqueezeNet V1.1 Trained on ImageNet Competition Data"], -2],
"Input" -> {3, 750, 750}]
],
"classifier" -> LinearLayer[1]
],
"Input" -> NetEncoder[{"Image", {750, 750}, "ColorSpace" -> "RGB"}]
]


Which is probably a decent net. I don't really know. It's got lots of layers at least, which is enough to impress me. On the other hand the training time they estimate for it an hour, since I have to train on the CPU, as TargetDevice->"GPU" failed and what you really want is for it to train on GPU.

So maybe some time when I have time to kill I'll try that.

If someone wants to train that net here's the training code (pulled straight from that guy's report):

rots2 =
ImageRotate[#, #2, Masking -> Full] -> {#2} &, {
imgs,
RandomReal[2 \[Pi], Length@imgs]
}];

NetTrain[net2,
LearningRateMultipliers -> {"classifier" -> 1, "conv10" -> 1,
"fire9" -> 1, _ -> None}
]


And if anyone has a GPU to train that (i.e. isn't using an old Mac) it'd probably be pretty fast to train.

• I'm trying this on a Linux machine with a gpu... are the images you had really 750x750?
– M.R.
Commented Aug 9, 2017 at 22:13
• Also I think the validation set should consist of different images not just different rotations, don't you think? Also why did you choose 10000x100 sized images?
– M.R.
Commented Aug 9, 2017 at 22:32
• @M.R. I can't remember why 10000. I think I chose a size that was larger than whatever size they were so I'd have strips, rather than a grid. I don't think it pads but maybe it does. The 750x750 was to give something squarish for the 90 degree case. Commented Aug 9, 2017 at 22:54

Using ImageLines seems to work in this case

ImageRotate[img, -ArcTan[(#[[2, 2]] - #[[1, 2]])/(#[[2, 1]] - #[[1,
1]])] &@(ImageLines[GradientFilter[img, 1], Method -> "RANSAC"] //
Total)]


• or to filter for max 20 degree angle: ImageRotate[img, Median[Select[-ArcTan[(#[[2, 2]] - #[[1, 2]])/(#[[2, 1]] - #[[1, 1]])] & /@ ImageLines[ GradientFilter[img, 1]], # < 20/360 2 \[Pi] && # > -20/360 2 \[Pi] &]]] Commented Jul 31, 2017 at 14:40
• I have to say,the ImageLine is not a efficient solution..
– yode
Commented Aug 1, 2017 at 9:45
• you could use , Method -> "RANSAC", this takes AbsoluteTiming from 0.41 for your code to 0.27 on my machine. The real benefit is not having to specify parameters as in your code as you indicate limits the use Commented Aug 1, 2017 at 14:49

Here is a different strategy. If you know, you are working with pages of text or line-content like in your second example, you can use this knowledge. A text is almost always arranged in long lines. Additionally, the contrast between the text and the background is often fairly high. That being said, Binarize should often work without parameters and give you a good separation between text (or lines) and background.

img1 = Import["https://i.sstatic.net/cCjyn.png"];
img2 = Import["https://i.sstatic.net/reBC0.png"];
Row[Binarize /@ {img1, img2}]


When you are using something like a gradient-orientation-filter, you end up with all the edges around single letters. Even if you, like in your example dilate the image first, the letters will make small bumps that all contribute to your list of orientations that you need to filter out.

When I understood your intention correctly, this is not what you want. How do you see that the image is slightly rotated? You see it because the lines of text are not quite horizontal.

A better approach would, therefore, be to work on objects that are a line of text. We know that letters in your example are really close together. So Closing will surely help to get rid of most of the gaps.

Let's see how far this brings us. The only parameter I'm using is the value for the morphological closing.

HighlightImage[
#,
Polygon /@
Values[ComponentMeasurements[Closing[Binarize[ColorNegate[#]], 3],
"MinimalBoundingBox"]]
] & /@ {img1, img2}


Some of the small objects don't represent the image orientation, but the larger objects are pretty good indicators for the rotation. Now, we can use the "Orientation"s of the objects with ComponentMeasurements but we have one final thing to deal with: There might be objects that are vertical. They too indicate the rotation, but we have to use their orientation with an offset of 90 Degree.

Let us write a small function f, that maps an input orientation to the correct orientation we want to use for rotating.

f[phi_] := With[{a = Mod[phi + Pi, Pi]},
Piecewise[{{a - Pi/2, Pi/4 < a < 3 Pi/4}, {a - Pi, 3 Pi/4 <= a}},
a]
]


and test it. Blue is the input orientation and red is what we will finally use to rotate the image.

Manipulate[
Graphics[{
Circle[],
Blue, Arrow[{{0, 0}, {Cos[phi], Sin[phi]}}],
Red, Arrow[{{0, 0}, {Cos[f[phi]], Sin[f[phi]]}}]
}], {phi, -Pi, Pi}
]


Putting all together gives us the following function

rotateBack[img_, closing_: 3, minLength_: 50] := Module[{
bin = Closing[Binarize[ColorNegate[img]], closing],
orientations
},
orientations =
Values[ComponentMeasurements[bin, "Orientation", #Length > minLength &]];
ImageRotate[img, -Median[f /@ orientations], Background -> White]
]


To get an idea how good this small function works, let us write a short highlighting code that puts some horizontal lines in the results:

Module[{nx, ny},
{nx, ny} = ImageDimensions[#];
HighlightImage[
#,
Table[Line[{{1, y}, {nx, y}}], {y, 1, ny, 30}]
]
] & /@ {rotateBack[img1], rotateBack[img2]}


Et voila

• Do you mean Closing will better Dilation here?
– yode
Commented Aug 12, 2017 at 5:57
• @yode Dilation expands objects morphologically. We only want to close gaps between the letters. In my opinion Closing is the better alternative here. Commented Aug 12, 2017 at 13:21