After experimenting with image alignment aprroaches presented here and here and some other approaches involving ImageDistance
and NMinimize
, I realized that the most robust approach is the direct search, and it is probably can be implemented effficiently using Compile
or FunctionCompile
(but my experience with them is still very limited).
Suppose we have two overlapping screenshots with equal dimensions (and equal number of channels), but originally saved in a lossy format like JPG:
imgs = Import /@ {"https://i.imgur.com/sXrjTom.jpg", "https://i.imgur.com/gWvFAod.jpg"};
{i1, i2} = ImageTake[#, {230, 2210}] & /@ imgs
We need to find the correct vertical alignment for them. Unfortunately, ImageAlign
is not only slow, but also gives unstable results, as I describe in this answer. So the best what we can do is to implement our own alignment algorithm.
The basic algorithm is fairly simple. Supposing that the images are successive, and the second image continuates the first:
Get
ImageDimensions
and convert our images intoImageData
:{width, height} = ImageDimensions[i1]; id1 = ImageData[i1, "Byte", Interleaving -> True]; id2 = ImageData[i2, "Byte", Interleaving -> True];
Define a distance finction depending on the alignment:
distance[align_] := Total@TakeLargest[Abs[Flatten[id1[[-align ;;]] - id2[[;; align]]]], 100]
Calculate the distances for all alignments starting from some minimal value:
distances = Monitor[Table[{align, distance[align]}, {align, 20, height}], align];
Take the alignment corresponding to the minimal distance as the correct alignment:
MinimalBy[distances, Last]
{{613, 2653}}
ListLinePlot[distances, PlotRange -> All]
Another obvious variation would be calculating the distances until we find the distance lesser than some predefined threshold:
thr = 5*width;
dist = Infinity;
align = 19;
While[dist > thr && align <= height - 1,
align += 1;
dist = distance[align]];
{align, dist}
{613, 2653}
What would be the most efficient way to implement both these approaches as compiled functions?
ParallelTable
. Another strategy might be to first bluri1
andi2
, e.g. withGaussianFilter
. The would smear out the peak in the distance plot such that it might become detectable byTable[{align, distance[align]}, {align, 20, height,step}]
with larger stepsizestep
, e.g.step = 10
. If you have found the bestalign
for the blurred data, you can refine it by search betweenalign-step
andalign+step
. It is also possible to do more than one level of refinement... $\endgroup$Downsample
orArrayResample
the images, derive a first guess foralign
from the downsamples, and then do a similar refinement. $\endgroup$