Remove Successive Duplicate From A Large List (200k by 4 matrix)

I am interested in removing successive duplicate rows from a very large list (~200k rows by 4 columns) using Mathematica.The built-in 'DeleteDuplicates' seems to remove all duplicates, I am interested in removing contiguous or successive duplicate. In other words, duplicate can still exist but not in a contiguous manner.

As an example: TCC12, TCC12, B96, TM12, TCC12, B96 should be TCC12, B96, TM12, TCC12, B96

Thanks

4 Answers

x = {TCC12, TCC12, B96, TM12, TCC12, B96};

x /. {a___, b_, b_, c___} :> {a, b, c}


{TCC12, B96, TM12, TCC12, B96}

First /@ Split[x]


Same output, but probably faster.

Update thanks to Belisarius

(m = {{1, 1, 1, 1}, {2, 2, 0, 2}, {0, 1, 0, 0}, {2, 2, 0, 0}}) // MatrixForm


Map[First, Split /@ m, {2}] // MatrixForm


• @belisarius - Please see update :)
– eldo
Sep 25, 2014 at 18:10
• Yup, For the OP case (2D list) it should be something like Map[First, Split /@ x, {2}] Sep 25, 2014 at 18:13
• I am not sure that's what the OP intended (that is, the way the question is phrased, he might have meant to delete duplicate ROWS, not duplicates FROM rows. Sep 25, 2014 at 19:22
• @Igor, yes you are right,I am interested in deleting duplicate ROWS (which could be reduced to a N*1 matrix, when all the 4 column are concatenated for my specific case), all the same thank you all for your response. Oct 24, 2014 at 15:26

Another variant:

deleteConsecutive = Split[#][[All, 1]]&

deleteConsecutive@{TCC12, TCC12, B96, TM12, TCC12, B96}

(* {TCC12, B96, TM12, TCC12, B96} *)

f = #1 & @@@ Split @ # &;

x = {TCC12, TCC12, B96, TM12, TCC12, B96};
f@x
(* {TCC12,B96,TM12,TCC12,B96} *)

m = {{1, 1, 1, 1}, {2, 2, 0, 2}, {0, 1, 0, 0}, {2, 2, 0, 0}};
f /@ m
(* {{1},{2,0,2},{0,1,0},{2,0}} *)

m = {{1, 1, 1, 1}, {2, 2, 0, 2}, {0, 1, 0, 0}, {2, 2, 0, 0}};


Since V 13.1 we have

DeleteAdjacentDuplicates /@ m


{{1}, {2, 0, 2}, {0, 1, 0}, {2, 0}}