18
$\begingroup$

A crucial step in self-driving car is to detect lane lines. I'm wondering whether this can be achieved robustly using Mathematica's rich image processing tools.

For example how can I take camera image of this (taken from here)

enter image description here

and detect the two lane lines like this

enter image description here

I tried to use the ImageLines to find the lines, but its seems difficult to pick up the left lane line:

img = Import[
  "https://github.com/ralphie9224/CarND-LaneLines-P1/raw/master/test_\
images/solidWhiteRight.jpg"];

lines = ImageLines[img, 0.4, 0.8, 
   Method -> {"Hough", "Segmented" -> True}, MaxFeatures -> 2];

HighlightImage[img, {Pink, Thickness[0.005], Line /@ lines}]

enter image description here

I also tried to apply a mask before running through ImageLines,

polygon = {{510, 92}, {337, 404}, {337, 581}, {510, 960}};
pixelIndex = 
  Table[{i, j}, {i, 1, ImageDimensions[img][[2]]}, {j, 1, 
    ImageDimensions[img][[1]]}];
mask = 
 Image[Replace[
   RegionMember[Polygon@polygon, pixelIndex], {True -> 1, 
    False -> 0}, -1]];
masked = ImageMultiply[ColorConvert[img, "Grayscale"], mask];

But it is still difficult to pick up the left lane line

lines = ImageLines[masked, Method -> {"Hough", "Segmented" -> True}, 
   MaxFeatures -> 5];
HighlightImage[masked, {Orange, Line /@ lines}]

enter image description here

$\endgroup$
  • $\begingroup$ In the real world, lane lines are often poorly maintained to the point of being worn away completely in some jurisdictions (like mine in Massachusetts). Or they are obscured by fallen leaves, snow, road repairs, or rain at night (i.e., by lights reflecting off the wet sheen) or heavy traffic. So really, self driving cars need to function without relying on or expecting to find lane lines even on roads where they are assumed to be at least theoretically present. $\endgroup$ – user316117 Nov 30 '16 at 18:50
16
$\begingroup$

ImageLines expects an input image where the "line" pixels are white and the other pixels are dark, so unless you're looking at bubble chamber images, you're expected to do some preprocessing, like this:

img = Import["https://i.stack.imgur.com/SJBKi.jpg"];
binary = MorphologicalBinarize[img, .9];
HighlightImage[img, binary]

enter image description here

With this image, we can find the lane lines:

lines = ImageLines[Thinning@binary];
HighlightImage[img, {Pink, Thickness[0.005], Line /@ lines}]

enter image description here

$\endgroup$
6
$\begingroup$

I played around a bit and the main functions I think help are RemoveBackground, Binarize and GradientFilter. Finally, using the right arguments to ImageLines help you decide how distinct the lines need to be. The code I ended up with is this:

Note img is your original image

(*DeleteSmallComponents removes the small artifacts from binarizing*)
filtered = DeleteSmallComponents[
   (*Brightens the processed image*)
   ImageAdjust[

    (*Takes the change in brightness, can be used as an edge filter*)
    GradientFilter[

     Binarize[

      (*Removes the sky as its brightness is confusing ImageLines*)
      RemoveBackground[
       img,
       Lighter@Blue
       ],

       (*Binarizing threshold*)
      .84
      ],
      (*GradientFilter pixel radius*)
     1
     ]
    ],

   (*Number of small components to delete*)
   20
   ];


lines = ImageLines[
  (*Another gradient filter to define a bit more clearly*)
  GradientFilter[filtered, 1], 
  (*Threshold for lines*)
  0.03, 
  (*How distinct/different the lines should be*)
  0.1
];

HighlightImage[img, {Pink, Thickness[0.005], Line /@ lines}]

RoadLines

$\endgroup$
  • $\begingroup$ Thanks for the answer. I'm wondering why did you delete your answer in the first place. I think the answer is great and I and many others may found it very useful. I cannot ping you within the chat so I asked several people with access to moderator tools for help to bring back your answer. $\endgroup$ – xslittlegrass Dec 2 '16 at 14:56
  • $\begingroup$ Oh, thank you :) I tested it and it seemed pretty inefficient, especially compared to the other answer so I thought it would be better to keep the answer list clear. I'm glad you found it useful. $\endgroup$ – lowriniak Dec 2 '16 at 15:15
6
$\begingroup$

The existing answers are based on the ImageLines function, here I'm providing an answer based on the geometric transformations, which can work in curved roads as well.

We take an image of a straight road first

i = ImageTrim[Import["https://github.com/udacity/CarND-Advanced-Lane-Lines/raw/master/test_images/straight_lines2.jpg"], {{0, 50}, {1280, 718}}];
{width, height} = ImageDimensions[i];

Then use the same method as in here to remove the perspective distortion from the image. We define a perspective distorted rectangle in the original image and then find a geometric transformation that transforms this distorted rectangle to an undistorted one. The same transformation will take us from the original image to the undistorted image.

pts1 = {{289, 3}, {431, 106}, {869, 106}, {1034, 3}};
wdWidth = 200;
aspect = 5;
wdHeight = 80;
pts2 = {{width/2 - wdWidth/2, 0}, {width/2 - wdWidth/2, 
    wdHeight}, {width/2 + wdWidth/2, wdHeight}, {width/2 + wdWidth/2, 
    0}};
t = FindGeometricTransform[pts2, pts1][[2]]

This compares the original and the undistorted image

Row@{Show[i, 
   Graphics[{PointSize[Large], Red, Point[pts1], Transparent, 
     EdgeForm[Red], Polygon[pts1]}], ImageSize -> Large], 
  Show[ImagePerspectiveTransformation[i, t, PlotRange -> Full], 
   Graphics[{PointSize[Large], Green, Point[pts2], Transparent, 
     EdgeForm[Green], Polygon[pts2]}], ImageSize -> Large]}

enter image description here

Now we can binarize the image and get the lane pixels

iwrap = ImageTrim[
  ImagePerspectiveTransformation[i, t, 
   PlotRange -> Full], {{width/2 - wdWidth/2 - wdWidth/4, 
    0}, {width/2 + wdWidth/2 + wdWidth/4, height}}];
ibinary = Binarize[iwrap];
{newWidth, newHeight} = Reverse[ImageData[ibinary] // Dimensions];
idx = Position[ImageData[ibinary], 1];
leftIdx = Select[idx, #[[2]] < 150 &];
rightIdx = Complement[idx, leftIdx];

and fit the pixel positions using quadratic form.

leftModel = Fit[leftIdx, {x^2, x, 1}, x]
rightModel = Fit[rightIdx, {x^2, x, 1}, x]

This shows the lane lines in the undistorted image

Show[ImagePerspectiveTransformation[i, t, PlotRange -> Full], 
 Plot[{leftModel, rightModel}, {x, 1, 670}, PlotStyle -> Thick] /. 
  Line[pts_] :> 
   Line[pts /. {x_, y_} -> {y + (width - newWidth)/2, height - x}], 
 ImageSize -> Large]

enter image description here

Now we can change back to the perspective image using the inverse transform and plot the lane

plane = Cases[
   Plot[{leftModel, rightModel}, {x, 1, 670}, PlotStyle -> Thick] /. 
    Line[pts_] :> 
     Line[pts /. {x_, y_} -> {y + (width - newWidth)/2, height - x}], 
   Line[pt_] :> Line[pt], ∞];
t2 = FindGeometricTransform[pts1, pts2][[2]];

Show[i, Graphics[{Green, Opacity[0.25], 
   Polygon[Join[Reverse[#[[1]]], #[[2]]] &@(plane /. 
       Line[pts_] :> t2 /@ pts)], Blue, Opacity[0.5], 
   Thickness[0.003], plane /. Line[pts_] :> Line[t2 /@ pts]}]]

enter image description here

Since this method doesn't depend on the ImageLines, it can also work for a curved road:

enter image description here

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.