# How to merge several partially overlapping screenshots into one image?

I have two screenshots:

For a long screenshot picture I usually use PhotoShop to combine both images:

How do I use Mathematica to make an automated program to do this? This is my current semi-automatic solution to get the long screenshot:

list = {pic1, pic2} =
Import /@ {"http://i.stack.imgur.com/e57Tv.png",
"http://i.stack.imgur.com/jUC5F.png"};
newpic2 =
ImageTake[
pic2, -800](*This step is very empirical, I am not satisfied with it.*);
align = Binarize[ImageAlign[pic1, newpic2, Background -> Black], 0];
crop = ImageTake[list[[1]],
Last@ImageDimensions[list[[1]]] -
Abs[Subtract @@@
Values[ComponentMeasurements[align, "BoundingBox"]]][[-1, -1]]];
ImageAssemble[{{crop}, {newpic2}}]


Is there a smarter method?The ImageCorrespondingPoints should be helpful,but I I don't know how to realize it.

Update:

The @Arnoud Buzing's answer almost solve this problem,but there are TWO difficulties still to overcome

1. First difficulty:

As the answer we can build a function to do this:

MergeImage[image1_, image2_] :=
Module[{i1 = ImageData[image1], i2 = ImageData[image2]},
{s1, s2} = LongestCommonSubsequencePositions[i1, i2];
Join[Take[i1, Last@s1], Take[i2, {Last@s2 + 1, -1}]] // Image]


But when the

image1 is image2 is

Due to the common subsequence is not the Longest,the long-screenshots will be wrongly merge.

2. Second difficulty:

The MergeImage we built cannot distinguish this ordring of these screenshots automatically.Such as we can run it MergeImage[image1, image2],But not MergeImage[image2, image1].There is a image3:

If our last MergeImage can run it like this to get the right long-screenshots,the problem have been solved.

MergeImage[RandomSample[{image1, image2, image3}]]

-
Life would be easier if you first crop out the top of image2, image3 etc before attempting anything automatic. – Rahul Mar 6 at 15:19
Possibly related: mathematica.stackexchange.com/q/32612/484 – Rahul Mar 6 at 15:22
@Rahul Because the image2image3 can be different picture completely.I just use SE's screenshots to express this kind of question in here.So If you wanna crop it manually,It is a very dull works.And your case of that link have no part which is juxtaposition each other. – yode Mar 6 at 15:44
You don't need to crop it exactly, you just need to get rid of the top bar assuming you know roughly how big it is. – Rahul Mar 6 at 16:56

This solves your "entangled images" case.

ClearAll[getMinRowFromImage, getBestMatch, findAllMatches, joinImages,
findImagesSequence, getIntensity];

getIntensity[i_Image, pos_List] :=
(* Calculates the intensity of a pixel for any image color space.*)
First@ColorConvert[Flatten@{PixelValue[i, pos]}, ImageColorSpace[i] -> "Grayscale"]

getMinRowFromImage[i_Image] :=
(* returns the min intensity value for an image and the row where it occurs*)
{getIntensity[i, #[[1]]], #[[1, 2]]} &@PixelValuePositions[i, "Min"]

getBestMatch[i_List, {m_Integer, n_Integer}, lines_Integer] :=
(* Finds the best match between i[[n]] and the last lines of i[[m]].
Note that "Padding -> None" is what allows this to run within
reasonable time, calculating the correlation only by rows and
thus returning single column image*)
getMinRowFromImage@ImageCorrelate[i[[n]], ImageTake[i[[m]], -lines],

findAllMatches[i_List, lines_Integer, sens_Real] :=
(*forms all permutations {m,n} from the list of images
calculates the best match between all pairs and
select those below the sensitivity parameter
Could be done more efficiently, since we expect only one
valid continuation foreach image, so we don't really need to
calculate them all*)
With[{permutations = Position[IdentityMatrix[Length@i], 0]},
Select[{#, getBestMatch[i, #, lines]} & /@ permutations, #[[2, 1]] < sens &]
]


.

findImagesSequence[g_Graph] :=
(* g is  a directed PathGraph whose wheights are the best match lines
for each pair. So, the Max of the distance matrix is at the
{head, tail} pair. We  find the (only) path that goes from head
to tail, hence finding the right image sequence. There is a function
in Mma help that does the same for undirected Graphs, but I think
this one is better for our purpose *)
FindShortestPath[g, Sequence @@ VertexList[g][[First@Position[#, Max@#] &
[GraphDistanceMatrix@g /. ∞ -> 0]]]]

joinImages[i_List, linesToMatch_Integer, sens_Real] :=
(* Ensambles a full image from parts consisting in same column width
images matching from some row onwards for a minumum of "linesToMatch".
The sensitivity parameter seeme not really needed but perhaps useful
for very similar images
*)
Module[{matches, g, seqimg, fromline},
(* First find the pairs matching *)
matches = findAllMatches[i, linesToMatch, sens];

(*Now we build up the whole sequence, from head to tail.
Graph features are great for that*)
g = Graph[Rule @@@ #[[All, 1]], EdgeWeight -> Flatten@#[[All, 2, 2]]] &@matches;
seqimg = findImagesSequence[g];
fromline = PropertyValue[{g, DirectedEdge @@ #}, EdgeWeight] & /@
Partition[seqimg, 2, 1];

(*Finally assemble the whole thing*)
ImageAssemble@Join[{{i[[First@seqimg]]}},
MapThread[{ImageTrim[#1, {{0, #2}, {ImageDimensions[#1][[2]], 0}}]} &,
{i[[Rest@seqimg]], fromline}]]
]


Usage:

l = {"http://i.stack.imgur.com/IXFEq.png",
"http://i.stack.imgur.com/FMtjm.png",
"http://i.stack.imgur.com/aj8a1.png"};
(*Resize and GrayScale for speed*)
i = ImageResize[#, 500] & /@ (ColorConvert[#, "Grayscale"] & /@ Import /@ l);

sensitivity = .1;
lnsTomatch = 150;
joinImages[i, lnsTomatch, sensitivity]


Coloring not included in this code, used here to show how the image was composed from the three snapshots. The last step (the joining of images, excluding the import part) is instantaneous in my machine.

-
"<del>Perhaps</del> I'll add some explanations" – Aisamu Mar 14 at 22:21
It seem that you can reach the top of another mountain just cost my a nap time. – yode Mar 14 at 23:19
The bounty expiring soon.I must choose a answer.But some bugs are exist still in this code.It'll give some error information when I run it like this.Look forward to your explanations and consummating it. – yode Mar 15 at 14:13
Of course I have run that sentence of ImageResize.But the error imformations is still.And after this post,I'll take a good study of Graph Theory to follow your steps. – yode Mar 15 at 15:41
@Aisamu Done :) – Dr. belisarius Mar 17 at 16:31

Very roughly (using image1 and image2):

i1 = ImageData[image1];
i2 = ImageData[image2];
css = LongestCommonSubsequence[i1, i2];
s1 = SequencePosition[i1, css];
s2 = SequencePosition[i2, css];
Join[Take[i1, s1[[1,2]] ], Take[i2, {s2[[1,2]] + 1, -1}]] // Image


That seems to work for me and gives the images concatenated together.

-
i1 = ImageData[image1]; i2 = ImageData[image2]; {s3, s4} = LongestCommonSubsequencePositions[i1, i2]; Join[Take[i1, Last@s3], Take[i2, {Last@s4 + 1, -1}]] // Image – yode Mar 6 at 8:47
I have make this problem more clear just now.If you are available,you can take a try please. – yode Mar 6 at 9:40

Since this is an iphone screenshot, we know the exact pixel dimensions of the toolbar and so no fancy sequence matching need be done:

MergeImage[image1_, image2_] := Module[{i1, i2, i3},
i1 = ImageData[image1][[;; 228]];
i2 = ImageData[image1][[228 ;;]];
i3 = ImageData[image2][[228 ;;]];
Join[i1, i2, i3] // Image]


-
Thanks a lot for your answer,but maybe you misunderstand this topic.1.The screenshot from XiaoMi cell-phone(base in Andriod).2.Maybe ImageTake+ImageAssemble Can do what you have done.3.We have image1image2 and image3.And we want get a long screenshot by they,but they have some repeated part each other(you can look these picture carefully). – yode Mar 14 at 7:53