6
$\begingroup$

I am interested in removing successive duplicate rows from a very large list (~200k rows by 4 columns) using Mathematica.The built-in 'DeleteDuplicates' seems to remove all duplicates, I am interested in removing contiguous or successive duplicate. In other words, duplicate can still exist but not in a contiguous manner.

As an example: TCC12, TCC12, B96, TM12, TCC12, B96 should be TCC12, B96, TM12, TCC12, B96

Thanks

$\endgroup$

4 Answers 4

5
$\begingroup$
x = {TCC12, TCC12, B96, TM12, TCC12, B96};

x /. {a___, b_, b_, c___} :> {a, b, c}

{TCC12, B96, TM12, TCC12, B96}

First /@ Split[x]

Same output, but probably faster.

Update thanks to Belisarius

(m = {{1, 1, 1, 1}, {2, 2, 0, 2}, {0, 1, 0, 0}, {2, 2, 0, 0}}) // MatrixForm

enter image description here

Map[First, Split /@ m, {2}] // MatrixForm

enter image description here

$\endgroup$
4
  • $\begingroup$ @belisarius - Please see update :) $\endgroup$
    – eldo
    Sep 25, 2014 at 18:10
  • $\begingroup$ Yup, For the OP case (2D list) it should be something like Map[First, Split /@ x, {2}] $\endgroup$ Sep 25, 2014 at 18:13
  • 1
    $\begingroup$ I am not sure that's what the OP intended (that is, the way the question is phrased, he might have meant to delete duplicate ROWS, not duplicates FROM rows. $\endgroup$
    – Igor Rivin
    Sep 25, 2014 at 19:22
  • $\begingroup$ @Igor, yes you are right,I am interested in deleting duplicate ROWS (which could be reduced to a N*1 matrix, when all the 4 column are concatenated for my specific case), all the same thank you all for your response. $\endgroup$
    – sarge
    Oct 24, 2014 at 15:26
3
$\begingroup$

Another variant:

deleteConsecutive = Split[#][[All, 1]]&

deleteConsecutive@{TCC12, TCC12, B96, TM12, TCC12, B96}

(* {TCC12, B96, TM12, TCC12, B96} *)
$\endgroup$
2
$\begingroup$
f = #1 & @@@ Split @ # &;

x = {TCC12, TCC12, B96, TM12, TCC12, B96};
f@x
(* {TCC12,B96,TM12,TCC12,B96} *)

m = {{1, 1, 1, 1}, {2, 2, 0, 2}, {0, 1, 0, 0}, {2, 2, 0, 0}};
f /@ m
(* {{1},{2,0,2},{0,1,0},{2,0}} *)
$\endgroup$
2
$\begingroup$
m = {{1, 1, 1, 1}, {2, 2, 0, 2}, {0, 1, 0, 0}, {2, 2, 0, 0}};

Since V 13.1 we have

DeleteAdjacentDuplicates /@ m

{{1}, {2, 0, 2}, {0, 1, 0}, {2, 0}}

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.