4
$\begingroup$

I am trying to segment some very irregular cells out of images generated by timelapse microscopy. These cells don't fit any simple shape (not circular or elliptical) and they grow and change shape over time. I am trying to segment them by creating a manually generated mask for the first timepoint and then using that mask to find the cell in later timepoints and updating the mask to fit the new shape of the cell (since the cell will have changed shape due to growth and possibly changed position slightly over time). In other words, the mask will need to be dynamically modified over time.

Here's an image of a field of the crazy looking cells I am trying to segment:

Crazy cells I am trying to segment

Here's a manually generated mask for cell in the lower right I would like to get data on:

manually generated mask for cell I want to get data on and track/segment over time

Here are the same cells slightly later (cells are a little bigger and have shifted a bit). I would like to use my mask above to find the same cell in this later image and then modify the original mask to fit the cell here (so I can get data on its growth without having to manually draw a new mask):

same cells a little later on after having grown a bit

I realize this is not a simple problem but was hoping to at least be able to locate the same cell at the later timepoint using ImageCorrelate or some shape detecting function. However, I am not having much luck with this. Any suggestions are welcome.

Improve this question
$\endgroup$
0

2 Answers 2

Reset to default
2
$\begingroup$

Not perfect, but a start. 

(*Load the Images*)

i1 = ImageResize[Import["https://i.stack.imgur.com/rcKOy.jpg"], 600];
im = ImageResize[Import["https://i.stack.imgur.com/2keNS.jpg"], 600];
i2 = ImageResize[Import["https://i.stack.imgur.com/IuYU8.jpg"], 600];

(* get the corresponding points*)

ivp = ImageCorrespondingPoints[i1, i2];
ivpt = Round@Transpose@ivp;

(* Select the corresponding points inside the mask *)

ivps = Select[ivpt, MemberQ[PixelValuePositions[Binarize@im, 1], #[[1]]] &];

Show[i1, Graphics[{Red, Point[Transpose[ivps][[1]]]}]]
Show[i2, Graphics[{Red, Point[Transpose[ivps][[2]]]}]]

Mathematica graphics

Now we will try to find a pure quadratic (for simplicity) transformation between the points.
I'll follow @Jens 

dataXY = First@Transpose@ivps;
{dataFx1, dataFy1} = Map[List, Transpose@Last@Transpose@ivps, {2}];
ones = ConstantArray[{1}, Length@dataFx1];
d1 = Flatten /@ Transpose[{dataXY, ones, dataFy1}];
d0 = Flatten /@ Transpose[{dataXY, 0 ones, dataFx1}];
data = Join[d0, d1];

modelx[x_, y_] := ax x^2 + bx x y + cx y^2 + ix
modely[x_, y_] := ay x^2 + by x y + cy y^2 + iy
model[x_, y_, s_] := modely[x, y]*s + modelx[x, y]*(1 - s)

ff = FindFit[data, model[x, y, s], {ax, ay, bx, by, cx, cy, ix, iy}, {x, y, s}];

The transformed mask is:

it = ImageForwardTransformation[EdgeDetect[im, 2], 
                                ({modelx @@ ##, modely @@ ##} /. ff) &, 
                                DataRange -> Full]

Mathematica graphics

And superimposed with the second image we get:

ImageMultiply[ i2, ColorReplace[ColorNegate@it, Black -> Red]]

Mathematica graphics

Improve this answer
$\endgroup$
1
  • $\begingroup$ Thanks for giving this a try. Took me a bit to get a sense of what's going on in the code. Couple questions: i.) Is it possible to get a dense or possibly uniform sampling of points within the mask to then use ImageCorrespondingPoints on in the later image? It's unclear to me how the ImageCorrespondingPoints algorithm finds the corresponding points it does. ii.) Do you think fitting a global set of corresponding points from the whole of both images would give a better fit? iii.) What other models (besides quadratic) do you think are worth a shot trying out here? $\endgroup$
    – user13999
    Mar 11, 2016 at 20:38
2
$\begingroup$

It looks as if the cells are dark and have a bright border. If that is always the case, and if you don't mind a little manual adjustment, you can put a marker on each cell and use watershed segmentation to find the bright borders in between.

I've manually put points in each of the cells using a LocatorPane:

img = Import["https://i.stack.imgur.com/rcKOy.jpg"];    
pts = {{399, 496}, {497, 537}, {346, 507}, {349, 434}, {434, 
    269}, {337, 385}, {213, 485}, {231, 443}, {196, 536}, {79, 
    571}, {98, 505}, {150, 434}, {128, 397}, {576, 558}, {536, 
    581}, {198, 581}, {24, 540}, {62, 525}, {92, 87}};

LocatorPane[Dynamic[pts], img, 
 Appearance -> {Graphics[{Red, Point[{0, 0}]}]}, 
 LocatorAutoCreate -> True]

enter image description here

And these are the resulting watershed boundaries:

HighlightImage[img, 
 ColorNegate@Binarize@Image[WatershedComponents[img, pts]]]

enter image description here

Since you want to track these cells in a series of frames, you could then find a new marker point for each cell, and use that marker point in the next frame. The marker should be as far away as possible from the border, so I would use ComponentMeasurements to get a Mask of each component, then use DistanceTransform on the mask and find the point with the highest value, i.e. the point that's farthest from the boundary.

Improve this answer
$\endgroup$
6
  • $\begingroup$ This looks pretty good. I have actually tried watershedding before but you seem to be getting better results than I did. Couple follow-ups: $\endgroup$
    – user13999
    Mar 11, 2016 at 15:08
  • $\begingroup$ Couple follow-ups: i.) how can I split a watershed contour when the watershed results in undersegmentation? Ideally I would like to split the contour manually by drawing a line across a contour or at worst split the contour by adding another marker and rerunning the watershed. ii.) how did you find the markers you used for this position? Trial and error or did you do the distance transform? $\endgroup$
    – user13999
    Mar 11, 2016 at 15:16
  • $\begingroup$ @user13999: The only way to split watersheds is to use two markers instead of one. It shouldn't matter too much where in the two cells you place the markers. I placed the markers manually using LocatorPane. I can't tell if it worked immediately because it's so robust or because I got lucky. But as long as the border around/between the cells is brighter than anything in the cells, it should be robust. $\endgroup$
    – Niki Estner
    Mar 11, 2016 at 15:29
  • $\begingroup$ PS: I tried the distance transform to find improved markers for the next frame, but wasn't overwhelmed by the results. I'll see if I can think of something else when I have a little more time... $\endgroup$
    – Niki Estner
    Mar 11, 2016 at 15:30
  • $\begingroup$ Yes, when I've tried to work with watershedding in the past the problem is that it needs a lot of manual intervention to get the markers just right (especially for a crowded field of cells). If you have a lot of time and patience you can usually get a pretty good segmentation result but ideally the choice of marker and position could be automated and informed by the previous timepoint segmentation. $\endgroup$
    – user13999
    Mar 11, 2016 at 19:08

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge that you have read and understand our privacy policy and code of conduct.

Not the answer you're looking for? Browse other questions tagged or ask your own question.