Imagine that I have a real life video, with all the artefacts coming from real life, and also from video compression, etc, and that I can edit this video, with filters and cropping, in such a way that I can improve the success of ImageAlign
as an image stabilizer. How can I align/stabilize the original video, using the transformation rules identified for the edited one?
(I think that) This is not a question on how to recreate the ImageAlign function, from other functions belonging to the same family (FindGeometricTransform
, etc).
(I'm sure that) this is not a question on what Options would make it work better on the specific example I putted together (unless there's an option that gets these transformation rules out..., and that I completely missed)
Let's start by creating a shaky film:
film = Rasterize /@ Table[
Graphics3D[{EdgeForm[None],
Blue,
GeometricTransformation[Cuboid[{-4, -4, 0}],
Table[RotationTransform[
i*Pi/2 + angle, {0, 0, 1}, {0, 0, 0}], {i, 4}]],
Red,
GeometricTransformation[Cuboid[{-4, -4, 0}],
Table[RotationTransform[i*Pi/2, {0, 0, 1}, {0, 0, 0}], {i,
4}]]},
Boxed -> False,
ViewVector -> {5 {Pi, Pi/2, 2}, {0 + RandomReal[],
0 + RandomReal[], 0}}, ViewAngle -> Pi/5],
{angle, 0, 2 Pi/4 - Pi/32, Pi/32}
];
ListAnimate[film]
And now, let's try to unshake it:
filmSteady1 = ImageAlign[film[[1]], #, TransformationClass -> "Rigid"] & /@ film;
(First side note: if we do not Rasterize
it before, interestingly, it still works, but there's probably a bug in the internals of the ImageAlign function, or in compile, since I keep receiving the message: "CompiledFunction::cfn: Numerical error encountered at instruction 8; proceeding with uncompiled evaluation. >>"). I'm not expert enough to understand its origins. Can someone try to catch this one, so that it gets reported?
ListAnimate[filmSteady1]
Very bad...
Now, lets try ImageAlign
with a very simple filter added (again, this is an example; one can think of masks, two completely different videos, coming from two different recording technologies, etc):
infoFilm = ColorReplace[#, Blue -> White, 0.3] & /@ film;
filmSteady2 = ImageAlign[infoFilm[[1]], #, TransformationClass -> "Rigid"] & /@ infoFilm;
ListAnimate[filmSteady2]
Much better! How can I then apply to the target/original video, the identified transformation rules?
Second side note: if instead of using "Blue->White", you just write "Blue", the results from ImageAlign
are not the same... Shouldn't there be a way for ImageAlign
to keep the transparent information? (I imagine this is coming from the background option, but it makes more sense on image borders, than in the middle of the image).
Third side note: Once the transformation rules are listed, it is possible to imagine a multitude of techniques that would probably be more adapted to a video: comparing each image/frame to the previous one (in time), and then, pre-combining the transformation rules all together, before processing the image; analyze just every n frames, and analytically determine halfway transformations to apply to the non analyzed frames (so to go faster); determine a smooth transformation "path", obeying to some spline rule or similar; etc...
ImageAlign
usesImageKeypoints
andFindGeometricTransform
internally. Can't you do the same, in order to get access to the transformation? $\endgroup$ – Oleksandr R. May 2 '15 at 13:25ImageAlign
options as undocumented possibilities forFindGeometricTransform
, perhaps. $\endgroup$ – Oleksandr R. May 2 '15 at 13:32