17
$\begingroup$

The task is to recognize the tubes on the ceiling in the following image

original image

Get the binary image

binimg = ColorSeparate[img][[2]] // Binarize[#, 0.25] & // Thinning

binarized image

Using ImageLines as in:

lines = ImageLines[binimg, 0.074];
Show[img, Graphics[{Thick, Orange, Line /@ lines}]]

yields:

original image and detected lines

The number of detected lines is

In[82]:= Length@lines

Out[82]= 30

Problems:

  1. Not all tubes are found (see the image below; missing tubes indicated by arrows)
  2. Some objects other than the tubes are recognized as lines as well

missing tubes indicated by arrows

How can one improve the accuracy of the method?

$\endgroup$
18
$\begingroup$

I have a method that's more accurate, but I'm not sure how robust it is in the end. But maybe some of my tricks are useful for you.

My first step to make the problem easier is to try to remove the perspective. If the pipes are all (more or less) vertical lines in the image, I can use image processing filters with anisotropic filter sizes, i.e. filters that are taller than wide.

To do that, I need to find the vanishing point, i.e. the point where parallel lines in the world intersect in the image. So, let's find a few lines:

img = Import["http://i.stack.imgur.com/JygpT.jpg"];    
{w, h} = ImageDimensions[img];    
l = ImageLines[EdgeDetect[img], MaxFeatures -> 40];    
Show[img, Graphics[{Orange, Line /@ l}]]

enter image description here

Not all of those are edges of pipes, but if we filter out the "more or less vertical" lines, like this:

isVertical[l_] := Module[{dx, dy},
  {dx, dy} = Abs[Subtract @@ l];
  dx < 2 dy]

then most of them go through the vanishing point we're looking for:

Show[img, Graphics[{Orange, Line /@ Select[l, isVertical]}]]

enter image description here

Now we're looking for a point, that is as close as possible to each of these lines. So I'll need a function that calculates the distance between a line and a point:

lineDist[{p1_, p2_}, q_] := Module[{dir, norm},
  dir = Normalize[p2 - p1];
  norm = {{0, 1}, {-1, 0}}.dir;

  Abs[(q - p1).norm]]

then the distances of some point {vx,vy} to all of these lines is:

vanishingPointDists = lineDist[#, {vx, vy}] & /@ Select[l, isVertical];

And we simply minimize that:

{err, solution} = FindMinimum[Total[vanishingPointDists], {vx, vy}];    
vanishingPoint = {vx, vy} /. solution;

Note that I'm using the total absolute distance as my optimization objective. (AKA L1 error norm). This has two consequences:

  • The solution is quite robust to outliers, so the non-pipe lines don't change the result too much
  • The problem is a linear programming problem, which means FindMinimum is fast and finds the optimal solution

To visualize the result, I'll draw a few lines going through the found vanishing point:

Show[img, 
 Graphics[{Red, 
   Table[Line[{{x, 1000}, vanishingPoint}], {x, -500, 2000, 50}]}]]

enter image description here

Now, to transform the image. My first attempt was to use a projective transform, but in the resulting image the lower half of the source image gets magnified to a large part of the output image. That's of course geometrically correct, but from an image processing view, most of the image would be unusable image interpolation artifacts, and the information of the closer image parts is mostly lost. So I've used an ad-hoc transform that simply "spreads" the x-coordinates and leaves the y-coordinates unchanged:

verticalized = ImageTransformation[img, Module[{x, y},
    {x, y} = # - vanishingPoint;
    {x*y/h, y} + vanishingPoint
    ] &, PlotRange -> Full]

enter image description here

(a disadvantage of the transform is that it doesn't transform straight lines to straight lines - so any step working with verticalized will have to take that into account.)

A little image processing to binarize the pipes:

gauss = GaussianFilter[verticalized, {{10, 1}}];
binary = MorphologicalBinarize[gauss, {.1, .5}];
HighlightImage[verticalized, binary]

enter image description here

And finally, the part I'm not really happy with: The actual counting. I simply take each row in the binary image and count connected white "runs". Ideally, all rows would agree on the same count (the number of pipes), so in practice I'll use the "majority vote" among all rows:

(* count black -> white edges *)
countBrightRuns = Count[Differences[#], 1] &;

(* count "pipes" in each image row and take the majority vote *)
SortBy[Tally[countBrightRuns /@ ImageData[binary]], Last][[-3 ;;]]

{{17, 70}, {11, 99}, {16, 105}}

Now 16 seems the right result (at least in the image region I've cut out with ImageTransformation - you'll have to play with PlotRange above to get the pipes on the right.) But 105 : 99 is a very close vote.

$\endgroup$
  • 1
    $\begingroup$ Simply amazing. $\endgroup$ – Yves Klett Jan 31 '16 at 14:06
  • $\begingroup$ I have read it some times.There are some place can be simplified I think.I will post what I have thought these days.And the answer give me a deeply moving.Thank you very much. $\endgroup$ – yode Feb 6 '16 at 16:31
  • $\begingroup$ I have posts a commentary answer just now in the following.If you are available,you can take a look. $\endgroup$ – yode Feb 13 '16 at 10:37
1
$\begingroup$

Just to be clear this is not a answer,but a long comment by a illustrative example. This is a pic to be used.

enter image description here

This place can be simplified

v = {vx, vy} = 
  Values@Last@
    FindMinimum[
     Total[RegionDistance[InfiniteLine@#, {mx, my}] & /@ 
       ImageLines@EdgeDetect[pic]], {mx, my}]

{838.911, 8.29829}

Get the picture's heigh and width.

{w, h} = ImageDimensions[pic];

This place is a confusion haunt me some time

As what you have posts

ImageTransformation[pic, 
 Module[{x, y}, {x, y} = # - v; {x*y/h, y} + v] &, PlotRange -> Full]

enter image description here This is my code,and I think should have a same effect as yours.

ImageTransformation[pic, {Times @@ #/h, #[[2]]} &, 
 DataRange -> {{-vx, w - vx}, {0, h}}]

enter image description here

But what's problem about this two method to cause this?

enter image description here

$\endgroup$
  • 3
    $\begingroup$ This should be posted as a new question linking back to this one. $\endgroup$ – Mr.Wizard Feb 13 '16 at 10:40
  • $\begingroup$ I think I just have changed the picture for better explain my viewpoint and advice about this question.Orz... $\endgroup$ – yode Feb 13 '16 at 10:48
  • $\begingroup$ Your ImageTransformation isn't using vy at all? That can't be right... $\endgroup$ – Niki Estner Feb 13 '16 at 11:34
  • $\begingroup$ @nikie But I think this transform have nothing to do with vy.I just wanna change the x-coordinate.Note the DataRange -> {..., {0, h}} $\endgroup$ – yode Feb 13 '16 at 11:46
  • $\begingroup$ Yes, but you want to change the x-coordinate in relation to the y-coordinate, and you want the singularity to be at y==vy, not y==0. So I think you need DataRange -> {{-vx, w - vx}, {0, h}-vy} (not tested) $\endgroup$ – Niki Estner Feb 13 '16 at 12:01

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.