How to track a deformable rod in video?

Question

I have a video of nanotubes in solution taken with a confocal laser microscope. I typically track these nanotubes with Fiesta or Fiji, but in this case, there is variation in brightness that is confounding my tracking programs.

Here’s a link to the full video (.tif file)

What I tried

If you can't download the above link, here are 2001 frames in TIF file, it might take a few minutes to download:

frames = CloudGet @ CloudObject @ "https://www.wolframcloud.com/obj/b4056a15-97e5-464a-879e-89852b85ebfd";

Using the image tool-ribbon to get points in the target blob:

So then extract the region of the nanotube as a mask:

pts = {{104.125`, 63.28125`}, {89.1328125`, 66.29296875`}};
RegionBinarize[frames[[1]], pts, .6]

But then using ImageCorrespondingPoints and ImageFeatureTrack the results were pretty bad:

pos = ImageValuePositions[RegionBinarize[frames[[1]], c, .6], 1];    
res = ImageFeatureTrack[frames, pos]
Graphics[{If[FreeQ[#, _Missing], {RandomColor[], Line[{#}]}] & /@ Transpose[res]}]

g = If[FreeQ[#, _Missing], Graphics@Point@#, Nothing] & /@ res;
ListAnimate@g

Treating everything as points doesn't really help in recovering the perimeter:

I was hoping some cv experts could help out with tracking the deforming shape of the nanotube over all the frames.

Answer 1

Since the object you want to track is moving slowly, you can use ImageDisplacements to calculate a per-pixel optical flow:

f = frames[[200 ;; ;; 3]];
flow = Image /@ ImageDisplacements[f];

Now flow contains a dx/dy displacement vector for each pixel for each pair of frames, that tells you the relative displacement between two frames. $*$

So the idea would be to pick a starting point in the first frame:

pt = {83., 48.}
HighlightImage[f[[1]], pt]

and then track that point through from frame to frame by accumulating the relative displacement vectors:

pts = FoldList[
   Function[{p, flow}, p + Mean[ImageTrim[flow, p, 10]]], 
   pt, flow];

Manipulate[HighlightImage[f[[i]], {pts[[i]]}], {i, 1, Length[f], 1}]

$*$ Note 1: I've skipped the first 200 frames, because there's an abrupt jump around frame 100. It's much larger than 1 pixel, so the dense flow "looses" the object here. You would have to use e.g. image alignment whole frames to remove this "jump" first.

Note 2: I'm only using every third frame to make processing faster.