6
$\begingroup$

I am playing around with Mathematica's new Neural Network toolbox and am trying to teach a network to detect lines in quite noisy images.

An example Input image would look like the following:

Input

The corresponding (desired) output is an image like this one (currently I am drawing those lines by hand...):

Output

How would one approach this? In detail:

  1. How many Input->Output pairs would I need? Currently I have about 100 but I guess this is not enough.

  2. What Network architecture would you recommend? I tried a couple of combinations of Convolution and Ramp/Tanh layers but without amazing results.

  3. Is there a more suited loss layer than the standard MeanSquare one for my purpose?

  4. Would you even recommend using a neural network for this task?

Thanks, Max

Improve this question
$\endgroup$
10
  • 4
    $\begingroup$ Why not use techniques from computer vision such as Hough transform? $\endgroup$
    – David G. Stork
    Commented Apr 17, 2017 at 17:56
  • 1
    $\begingroup$ @user3683367 Why are you only paying attention to the diagonal lines? Is that a requirement? You seem to be throwing away vertical lines (from noise) which have about same luminosity as the diagonal ones. So how do you know what's noise and what's the real deal? Run these on your source gif and see what you get: Binarize@ImageAdjust[ImageConvolve[ImageAdjust[ImageSubtract[gif,.15],1.05], {{-1,0,1},{-2,0,2},{-1,0,1}}],1.1] or ImageAdjust@GradientFilter[ImageAdjust[ImageSubtract[gif,.1],1.2],{2,.5}] $\endgroup$
    – Gregory Klopper
    Commented Apr 17, 2017 at 19:44
  • $\begingroup$ @DavidG.Stork The lines are not necessary straight but can become quite curved. Also I only want clear diagonal lines and discard noisy areas/lines. Thats why I initially thought the network approach might be best. As far as I understood the Hough Transform is mostly useful to extract broken lines or? $\endgroup$
    – MaxJ
    Commented Apr 17, 2017 at 19:52
  • 3
    $\begingroup$ This is a semantic segmentation type problem. The desired output of the network is a heat map (or threshholded heatmap) of whether or not a given pixel belongs to one of your desired pixels. There is an example in the 11.1 docs on semantic segmentation that is a pretty good place to start $\endgroup$
    – Yss
    Commented Apr 17, 2017 at 23:11
  • 1
    $\begingroup$ In documentation for NetTrain under Applications> Computer Vision. $\endgroup$
    – Gregory Klopper
    Commented Apr 18, 2017 at 21:32

2 Answers 2

Reset to default
4
$\begingroup$

I would recommend trying a U-Net style network (Link to the original U-Net article) for segmentation of such features. It has fared well with a small number of inputs for training. I don't use Mathematica but there seems to be at least one implementation out there. If you're willing to switch to Python, this Keras implementation has proven sufficient.

Improve this answer
$\endgroup$
1
2
$\begingroup$

In the end I implemented a U-Net style network with some further tweaks to improve the detection of lines that reverse direction.It was a long journey with a lot of fun and frustration along the way.

You can read about it here: https://elifesciences.org/articles/42288

Improve this answer
$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.