2
$\begingroup$

I have a list of integer 2-tuples s = { {{3,4},{11,5}} , {{11,5},{3,4}}, {{5,5},{3,2}} } (in reality this set is much larger).

I am trying to delete duplicates from this list. Specifically, I want to treat lists within s which, up to a reordering, contain the same lists (2-tuples) as the same, and keep only one.

For instance, if s were my actual list, I would want to convert it into the following list: { {{3,4},{11,5}} , {{5,5},{3,2}} }.

I can't figure out an efficient way of doing this; in reality, my list s is quite large, and its sublists also consist of many elements.

$\endgroup$
1
  • 4
    $\begingroup$ DeleteDuplicatesBy[s, Sort] $\endgroup$
    – wuyudi
    Oct 5 '20 at 4:05
5
$\begingroup$
DeleteDuplicates[Sort /@ s]
{{{3, 4}, {11, 5}}, {{3, 2}, {5, 5}}}

You can also use Gather or GatherBy:

Gather[Sort /@ s][[All, 1]]
{{{3, 4}, {11, 5}}, {{3, 2}, {5, 5}}}
GatherBy[s, Sort][[All, 1]]
{{{3, 4}, {11, 5}}, {{5, 5}, {3, 2}}}

All three above are faster than DeleteDuplicatesBy[Sort] for long lists of lists:

SeedRandom[1]
ss = RandomInteger[10, {100000, 2, 2}];

r1 = DeleteDuplicates[Sort /@ ss]; // RepeatedTiming // First
0.038
r2 = Gather[Sort /@ ss][[All, 1]]; // RepeatedTiming // First
0.12
r3 = GatherBy[ss, Sort][[All, 1]]; // RepeatedTiming // First
0.13
r4 = DeleteDuplicatesBy[Sort]@ss; // RepeatedTiming // First
0.15
Sort /@ r1 == Sort /@ r2 == Sort /@ r3 == Sort /@ r4
True
SeedRandom[1]
ss = RandomInteger[10, {100000, 50, 5}];

r1 = DeleteDuplicates[Sort /@ ss]; // RepeatedTiming // First
1.26
r2 = Gather[Sort /@ ss][[All, 1]]; // RepeatedTiming // First
1.69
r3 = GatherBy[ss, Sort][[All, 1]]; // RepeatedTiming // First
1.61
r4 = DeleteDuplicatesBy[Sort]@ss; // RepeatedTiming // First
3.27
Sort /@ r1 == Sort /@ r2 == Sort /@ r3 == Sort /@ r4
True
$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.