6
$\begingroup$

I have a video of nanotubes in solution taken with a confocal laser microscope. I typically track these nanotubes with Fiesta or Fiji, but in this case, there is variation in brightness that is confounding my tracking programs.

enter image description here

Here’s a link to the full video (.tif file)

What I tried

If you can't download the above link, here are 2001 frames in TIF file, it might take a few minutes to download:

frames = CloudGet @ CloudObject @ "https://www.wolframcloud.com/obj/b4056a15-97e5-464a-879e-89852b85ebfd";

enter image description here

Using the image tool-ribbon to get points in the target blob:

enter image description here

So then extract the region of the nanotube as a mask:

pts = {{104.125`, 63.28125`}, {89.1328125`, 66.29296875`}};
RegionBinarize[frames[[1]], pts, .6]

enter image description here

But then using ImageCorrespondingPoints and ImageFeatureTrack the results were pretty bad:

pos = ImageValuePositions[RegionBinarize[frames[[1]], c, .6], 1];    
res = ImageFeatureTrack[frames, pos]
Graphics[{If[FreeQ[#, _Missing], {RandomColor[], Line[{#}]}] & /@ Transpose[res]}]

enter image description here

g = If[FreeQ[#, _Missing], Graphics@Point@#, Nothing] & /@ res;
ListAnimate@g

Treating everything as points doesn't really help in recovering the perimeter:

enter image description here

I was hoping some cv experts could help out with tracking the deforming shape of the nanotube over all the frames.

$\endgroup$
11
$\begingroup$

Since the object you want to track is moving slowly, you can use ImageDisplacements to calculate a per-pixel optical flow:

f = frames[[200 ;; ;; 3]];
flow = Image /@ ImageDisplacements[f];

Now flow contains a dx/dy displacement vector for each pixel for each pair of frames, that tells you the relative displacement between two frames. $*$

So the idea would be to pick a starting point in the first frame:

pt = {83., 48.}
HighlightImage[f[[1]], pt]

enter image description here

and then track that point through from frame to frame by accumulating the relative displacement vectors:

pts = FoldList[
   Function[{p, flow}, p + Mean[ImageTrim[flow, p, 10]]], 
   pt, flow];

Manipulate[HighlightImage[f[[i]], {pts[[i]]}], {i, 1, Length[f], 1}]

enter image description here

$*$ Note 1: I've skipped the first 200 frames, because there's an abrupt jump around frame 100. It's much larger than 1 pixel, so the dense flow "looses" the object here. You would have to use e.g. image alignment whole frames to remove this "jump" first.

Note 2: I'm only using every third frame to make processing faster.

$\endgroup$
  • $\begingroup$ I'm trying to find the full segmentation boundary of the string shape though (not just a point) $\endgroup$ – M.R. Oct 18 '19 at 20:16
  • $\begingroup$ Because what I really need to compute how it "wiggles" you see... $\endgroup$ – M.R. Oct 18 '19 at 20:19
  • $\begingroup$ @M.R. Can you define "how it wiggles" a bit more? I suspect it would be more robust to estimate "wiggliness" from the optical flow directly, instead of tracking points and then calculating something from these points. Tracking usually means that errors accumulate from one frame to the next. When you calculate a mean over some function of the flow vectors, errors (might hopefully) average out. $\endgroup$ – Niki Estner Oct 19 '19 at 5:39
  • $\begingroup$ I need to model it with the difeq for rod deformation and look at the normal modes. $\endgroup$ – M.R. Oct 20 '19 at 21:27
  • $\begingroup$ @M.R. Why not, then, track multiple points? Maybe one could do something like the perimeter of a region that is the same shape as the blob? $\endgroup$ – CA Trevillian Oct 21 '19 at 1:54

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.