7
$\begingroup$

Of course I know there are some related post about this topic,such as How to align rotated images vertically.And I have answered that post with two different method here and here.

But in this post,the case will be more precise,which have this features. The image always have a little lean on horizontal direction.it is about ±10°.

This is current method based on GradientOrientationFilter

img=Import["https://i.stack.imgur.com/cCjyn.png"];
ImageAdjust[
 gradImg = GradientOrientationFilter[
   Binarize[Dilation[ColorNegate[Binarize[img]], {Array[1 &, 15]}]],1]]

Mathematica graphics

Caculate the angle

angle = -Mean[Select[Catenate[ImageData[gradImg]], Abs[#] < 0.8 &]]

Get the result

ImageRotate[img, angle]

Mathematica graphics

Of course this code can serve another image.

Complaint about the current method:

  1. I don't know why I should select those value with a threshold 0.8.Actually N[20 °]==0.35,but if I use 0.35,I will get a bad result
  2. This is not a efficient method when the processed image have a big size
  3. I don't like the Dilation,because which mean I should adjust the threshold 15 for different image.

Actually I allways really don't hope any threshold value(or as little as possible) apprear in my solution,which will reduce the universality of the method. Sincely to request a better method here.

$\endgroup$
  • 1
    $\begingroup$ Maybe overkill, but you could try building a neural net that returns the rotation angle since it's so easy to make training data. $\endgroup$ – Chip Hurst Jul 28 '17 at 18:17
  • $\begingroup$ @ChipHurst So easy???? $\endgroup$ – yode Jul 29 '17 at 1:47
  • $\begingroup$ Can't you create the training data by doing ImageRotate[im_, RandomReal[{0,2Pi}] to a bunch of images of text? $\endgroup$ – Chip Hurst Jul 29 '17 at 1:52
  • $\begingroup$ @ChipHurst Maybe it can treat different angle for a same image,but I don't sure it can treat different image. $\endgroup$ – yode Jul 29 '17 at 1:58
  • 2
    $\begingroup$ @yode with enough data I figure you probably can get a good net for that sort of thing. Maybe you could make images of the entire contents of this or take pngs of stuff here (chopped into, say, 500 pixel columns) rotate them by arbitrary angles, and pass 80% of that into a net with some convolution laters as your training set, 20% as your test set, and see how it goes. $\endgroup$ – b3m2a1 Jul 30 '17 at 20:48
6
+50
$\begingroup$

Expanding on Chip Hurst's comment I built out a sample net to try this out:

net =
  NetInitialize@
   NetChain[{
     ConvolutionLayer[
      32,
      3,
      "Dilation" -> 3
      ],
     AggregationLayer[Mean],
     LinearLayer[1]
     },
    "Input" -> NetEncoder[{"Image", {750, 750}, "ColorSpace" -> "RGB"}]
    ];

And I made my training data as such:

imgs =
  Map[
    ImagePartition[
      Import[#],
      {10000, 100}
      ] &,
    FileNames["*.png", "~/Desktop/blee"]
    ] // Flatten;

rots =
  MapThread[
   ImageRotate[#, #2, Masking -> Full] -> {#2} &, {
    Flatten@ConstantArray[imgs, 3],
    RandomReal[2 \[Pi], 3*Length@imgs]
    }];

where ~/Desktop/blee contains the first three notebooks from here exported as PNG.

Then I let it train for all of three minutes because I didn't really want to sink the time to train a high quality net and because I was sure my net was crap.

Then I found a better net (probably, haven't trained it) here from this post on community: http://community.wolfram.com/groups/-/m/t/1136388

Minimally adapting the net to our task we have:

net2 =
 NetChain[
  Append[
   Normal[
    NetReplacePart[
     Drop[
      NetModel[
       "SqueezeNet V1.1 Trained on ImageNet Competition Data"], -2], 
     "Input" -> {3, 750, 750}]
    ],
   "classifier" -> LinearLayer[1]
   ],
  "Input" -> NetEncoder[{"Image", {750, 750}, "ColorSpace" -> "RGB"}]
  ]

Which is probably a decent net. I don't really know. It's got lots of layers at least, which is enough to impress me. On the other hand the training time they estimate for it an hour, since I have to train on the CPU, as TargetDevice->"GPU" failed and what you really want is for it to train on GPU.

So maybe some time when I have time to kill I'll try that.

If someone wants to train that net here's the training code (pulled straight from that guy's report):

rots2 =
  MapThread[
   ImageRotate[#, #2, Masking -> Full] -> {#2} &, {
    imgs,
    RandomReal[2 \[Pi], Length@imgs]
    }];

NetTrain[net2,
 Thread[rots, Rule],
 ValidationSet -> Thread[rots2, Rule], 
 LearningRateMultipliers -> {"classifier" -> 1, "conv10" -> 1, 
   "fire9" -> 1, _ -> None}
 ]

And if anyone has a GPU to train that (i.e. isn't using an old Mac) it'd probably be pretty fast to train.

$\endgroup$
  • $\begingroup$ I'm trying this on a Linux machine with a gpu... are the images you had really 750x750? $\endgroup$ – M.R. Aug 9 '17 at 22:13
  • $\begingroup$ Also I think the validation set should consist of different images not just different rotations, don't you think? Also why did you choose 10000x100 sized images? $\endgroup$ – M.R. Aug 9 '17 at 22:32
  • $\begingroup$ @M.R. I can't remember why 10000. I think I chose a size that was larger than whatever size they were so I'd have strips, rather than a grid. I don't think it pads but maybe it does. The 750x750 was to give something squarish for the 90 degree case. $\endgroup$ – b3m2a1 Aug 9 '17 at 22:54
4
$\begingroup$

Using ImageLines seems to work in this case

ImageRotate[img, -ArcTan[(#[[2, 2]] - #[[1, 2]])/(#[[2, 1]] - #[[1, 
      1]])] &@(ImageLines[GradientFilter[img, 1], Method -> "RANSAC"] // 
Total)]

enter image description here

$\endgroup$
  • $\begingroup$ or to filter for max 20 degree angle: ImageRotate[img, Median[Select[-ArcTan[(#[[2, 2]] - #[[1, 2]])/(#[[2, 1]] - #[[1, 1]])] & /@ ImageLines[ GradientFilter[img, 1]], # < 20/360 2 \[Pi] && # > -20/360 2 \[Pi] &]]] $\endgroup$ – Ruud3.1415 Jul 31 '17 at 14:40
  • $\begingroup$ I have to say,the ImageLine is not a efficient solution.. $\endgroup$ – yode Aug 1 '17 at 9:45
  • $\begingroup$ you could use , Method -> "RANSAC", this takes AbsoluteTiming from 0.41 for your code to 0.27 on my machine. The real benefit is not having to specify parameters as in your code as you indicate limits the use $\endgroup$ – Ruud3.1415 Aug 1 '17 at 14:49
4
+500
$\begingroup$

Here is a different strategy. If you know, you are working with pages of text or line-content like in your second example, you can use this knowledge. A text is almost always arranged in long lines. Additionally, the contrast between the text and the background is often fairly high. That being said, Binarize should often work without parameters and give you a good separation between text (or lines) and background.

Lets try your two examples:

img1 = Import["https://i.stack.imgur.com/cCjyn.png"];
img2 = Import["https://i.stack.imgur.com/reBC0.png"];
Row[Binarize /@ {img1, img2}]

Mathematica graphics Mathematica graphics

When you are using something like a gradient-orientation-filter, you end up with all the edges around single letters. Even if you, like in your example dilate the image first, the letters will make small bumps that all contribute to your list of orientations that you need to filter out.

When I understood your intention correctly, this is not what you want. How do you see that the image is slightly rotated? You see it because the lines of text are not quite horizontal.

A better approach would, therefore, be to work on objects that are a line of text. We know that letters in your example are really close together. So Closing will surely help to get rid of most of the gaps.

Let's see how far this brings us. The only parameter I'm using is the value for the morphological closing.

HighlightImage[
   #,
   Polygon /@ 
    Values[ComponentMeasurements[Closing[Binarize[ColorNegate[#]], 3],
       "MinimalBoundingBox"]]
   ] & /@ {img1, img2}

Mathematica graphics Mathematica graphics

Some of the small objects don't represent the image orientation, but the larger objects are pretty good indicators for the rotation. Now, we can use the "Orientation"s of the objects with ComponentMeasurements but we have one final thing to deal with: There might be objects that are vertical. They too indicate the rotation, but we have to use their orientation with an offset of 90 Degree.

Let us write a small function f, that maps an input orientation to the correct orientation we want to use for rotating.

f[phi_] := With[{a = Mod[phi + Pi, Pi]},
  Piecewise[{{a - Pi/2, Pi/4 < a < 3 Pi/4}, {a - Pi, 3 Pi/4 <= a}}, 
   a]
]

and test it. Blue is the input orientation and red is what we will finally use to rotate the image.

Manipulate[
 Graphics[{
   Circle[],
   Blue, Arrow[{{0, 0}, {Cos[phi], Sin[phi]}}],
   Red, Arrow[{{0, 0}, {Cos[f[phi]], Sin[f[phi]]}}]
   }], {phi, -Pi, Pi}
]

enter image description here

Putting all together gives us the following function

rotateBack[img_, closing_: 3, minLength_: 50] := Module[{
   bin = Closing[Binarize[ColorNegate[img]], closing],
   orientations
   },
  orientations = 
   Values[ComponentMeasurements[bin, "Orientation", #Length > minLength &]];
  ImageRotate[img, -Median[f /@ orientations], Background -> White]
  ]

To get an idea how good this small function works, let us write a short highlighting code that puts some horizontal lines in the results:

Module[{nx, ny},
   {nx, ny} = ImageDimensions[#];
   HighlightImage[
    #,
    Table[Line[{{1, y}, {nx, y}}], {y, 1, ny, 30}]
    ]
   ] & /@ {rotateBack[img1], rotateBack[img2]}

Et voila

Mathematica graphics Mathematica graphics

$\endgroup$
  • $\begingroup$ Do you mean Closing will better Dilation here? $\endgroup$ – yode Aug 12 '17 at 5:57
  • $\begingroup$ @yode Dilation expands objects morphologically. We only want to close gaps between the letters. In my opinion Closing is the better alternative here. $\endgroup$ – halirutan Aug 12 '17 at 13:21

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.