Find lane lines

Question

A crucial step in self-driving car is to detect lane lines. I'm wondering whether this can be achieved robustly using Mathematica's rich image processing tools.

For example how can I take camera image of this (taken from here)

and detect the two lane lines like this

I tried to use the ImageLines to find the lines, but its seems difficult to pick up the left lane line:

img = Import[
  "https://github.com/ralphie9224/CarND-LaneLines-P1/raw/master/test_\
images/solidWhiteRight.jpg"];

lines = ImageLines[img, 0.4, 0.8, 
   Method -> {"Hough", "Segmented" -> True}, MaxFeatures -> 2];

HighlightImage[img, {Pink, Thickness[0.005], Line /@ lines}]

I also tried to apply a mask before running through ImageLines,

polygon = {{510, 92}, {337, 404}, {337, 581}, {510, 960}};
pixelIndex = 
  Table[{i, j}, {i, 1, ImageDimensions[img][[2]]}, {j, 1, 
    ImageDimensions[img][[1]]}];
mask = 
 Image[Replace[
   RegionMember[Polygon@polygon, pixelIndex], {True -> 1, 
    False -> 0}, -1]];
masked = ImageMultiply[ColorConvert[img, "Grayscale"], mask];

But it is still difficult to pick up the left lane line

lines = ImageLines[masked, Method -> {"Hough", "Segmented" -> True}, 
   MaxFeatures -> 5];
HighlightImage[masked, {Orange, Line /@ lines}]

Answer 1

ImageLines expects an input image where the "line" pixels are white and the other pixels are dark, so unless you're looking at bubble chamber images, you're expected to do some preprocessing, like this:

img = Import["https://i.stack.imgur.com/SJBKi.jpg"];
binary = MorphologicalBinarize[img, .9];
HighlightImage[img, binary]

With this image, we can find the lane lines:

lines = ImageLines[Thinning@binary];
HighlightImage[img, {Pink, Thickness[0.005], Line /@ lines}]

Answer 2

I played around a bit and the main functions I think help are RemoveBackground, Binarize and GradientFilter. Finally, using the right arguments to ImageLines help you decide how distinct the lines need to be. The code I ended up with is this:

Note img is your original image

(*DeleteSmallComponents removes the small artifacts from binarizing*)
filtered = DeleteSmallComponents[
   (*Brightens the processed image*)
   ImageAdjust[

    (*Takes the change in brightness, can be used as an edge filter*)
    GradientFilter[

     Binarize[

      (*Removes the sky as its brightness is confusing ImageLines*)
      RemoveBackground[
       img,
       Lighter@Blue
       ],

       (*Binarizing threshold*)
      .84
      ],
      (*GradientFilter pixel radius*)
     1
     ]
    ],

   (*Number of small components to delete*)
   20
   ];


lines = ImageLines[
  (*Another gradient filter to define a bit more clearly*)
  GradientFilter[filtered, 1], 
  (*Threshold for lines*)
  0.03, 
  (*How distinct/different the lines should be*)
  0.1
];

HighlightImage[img, {Pink, Thickness[0.005], Line /@ lines}]

Answer 3

The existing answers are based on the ImageLines function, here I'm providing an answer based on the geometric transformations, which can work in curved roads as well.

We take an image of a straight road first

i = ImageTrim[Import["https://github.com/udacity/CarND-Advanced-Lane-Lines/raw/master/test_images/straight_lines2.jpg"], {{0, 50}, {1280, 718}}];
{width, height} = ImageDimensions[i];

Then use the same method as in here to remove the perspective distortion from the image. We define a perspective distorted rectangle in the original image and then find a geometric transformation that transforms this distorted rectangle to an undistorted one. The same transformation will take us from the original image to the undistorted image.

pts1 = {{289, 3}, {431, 106}, {869, 106}, {1034, 3}};
wdWidth = 200;
aspect = 5;
wdHeight = 80;
pts2 = {{width/2 - wdWidth/2, 0}, {width/2 - wdWidth/2, 
    wdHeight}, {width/2 + wdWidth/2, wdHeight}, {width/2 + wdWidth/2, 
    0}};
t = FindGeometricTransform[pts2, pts1][[2]]

This compares the original and the undistorted image

Row@{Show[i, 
   Graphics[{PointSize[Large], Red, Point[pts1], Transparent, 
     EdgeForm[Red], Polygon[pts1]}], ImageSize -> Large], 
  Show[ImagePerspectiveTransformation[i, t, PlotRange -> Full], 
   Graphics[{PointSize[Large], Green, Point[pts2], Transparent, 
     EdgeForm[Green], Polygon[pts2]}], ImageSize -> Large]}

Now we can binarize the image and get the lane pixels

iwrap = ImageTrim[
  ImagePerspectiveTransformation[i, t, 
   PlotRange -> Full], {{width/2 - wdWidth/2 - wdWidth/4, 
    0}, {width/2 + wdWidth/2 + wdWidth/4, height}}];
ibinary = Binarize[iwrap];
{newWidth, newHeight} = Reverse[ImageData[ibinary] // Dimensions];
idx = Position[ImageData[ibinary], 1];
leftIdx = Select[idx, #[[2]] < 150 &];
rightIdx = Complement[idx, leftIdx];

and fit the pixel positions using quadratic form.

leftModel = Fit[leftIdx, {x^2, x, 1}, x]
rightModel = Fit[rightIdx, {x^2, x, 1}, x]

This shows the lane lines in the undistorted image

Show[ImagePerspectiveTransformation[i, t, PlotRange -> Full], 
 Plot[{leftModel, rightModel}, {x, 1, 670}, PlotStyle -> Thick] /. 
  Line[pts_] :> 
   Line[pts /. {x_, y_} -> {y + (width - newWidth)/2, height - x}], 
 ImageSize -> Large]

Now we can change back to the perspective image using the inverse transform and plot the lane

plane = Cases[
   Plot[{leftModel, rightModel}, {x, 1, 670}, PlotStyle -> Thick] /. 
    Line[pts_] :> 
     Line[pts /. {x_, y_} -> {y + (width - newWidth)/2, height - x}], 
   Line[pt_] :> Line[pt], ∞];
t2 = FindGeometricTransform[pts1, pts2][[2]];

Show[i, Graphics[{Green, Opacity[0.25], 
   Polygon[Join[Reverse[#[[1]]], #[[2]]] &@(plane /. 
       Line[pts_] :> t2 /@ pts)], Blue, Opacity[0.5], 
   Thickness[0.003], plane /. Line[pts_] :> Line[t2 /@ pts]}]]

Since this method doesn't depend on the ImageLines, it can also work for a curved road: