Skip to main content
3 of 4
deleted 234 characters in body
Kagaratsch
  • 12.1k
  • 4
  • 25
  • 75

Efficiently merge lists chronologically without duplicates?

Prepare some dummy data to investigate:

alist = RandomSample[Table[ToExpression["a" <> ToString[i]], {i, 1, 10000}]];
range = Range[10000];
Do[
 len = RandomInteger[{3000, 10000}];
 select = Sort[RandomSample[range, len]];
 sublists[j] = alist[[select]];
 , {j, 1, 5000}]

we have 5000 sublists of length between 3000 and 10000 elements "ai" in a particular order. I would like to have a function that merges all 5000 sublists such that all duplicates are discarded and any "ai" appear to the left of "aj" if they do so in any of the sublists. How can one do this in Mathematica most efficiently?

A tiny example of the above would be:

alist = {a1,a3,a2,a5,a4};
sublists[1] = {a1,a5};
sublists[2] = {a3,a2,a5};
sublists[3] = {a1,a2};

so that the function merge returns:

merge[Table[sublists[i],{i,1,3}]]

{a1,a3,a2,a5}

Note that merge was not given the actual alist.

Kagaratsch
  • 12.1k
  • 4
  • 25
  • 75