# Replacing duplicates instead of deleting them

My code replaces each repeated element by "X".

ReplaceRepeated[{1, 2, 3, 2, 5, 2, 1, 5, 5, 2, 3, 3,
7}, {a1___, y_ /; y != #, a2___, y_, a3___} :> {a1, y, a2, #,
a3}] &@"X"

(*{1, 2, 3, "X", 5, "X", "X", "X", "X", "X", "X", "X", 7}*)


What is the most elegant code to do this?

ClearAll[replaceDuplicates]
replaceDuplicates[rep_: "X"] := Module[{f}, f[y_] := (f[y] = rep; y); f /@ #] &


Examples:

replaceDuplicates[] @ {1, 2, 3, 2, 5, 2, 1, 5, 5, 2, 3, 3, 7}

{1, 2, 3, "X", 5, "X", "X", "X", "X", "X", "X", "X", 7}

replaceDuplicates[Nothing] @ {1, 2, 3, 2, 5, 2, 1, 5, 5, 2, 3, 3, 7}

{1, 2, 3, 5, 7}


Alternatively:

ClearAll[replaceDuplicates2]
replaceDuplicates2[rep_: "X"] := Module[{y = #},
y[[Join @@ (Rest /@ Values @ PositionIndex[y])]] = rep; y] &


Examples:

replaceDuplicates2[] @ {1, 2, 3, 2, 5, 2, 1, 5, 5, 2, 3, 3, 7}

{1, 2, 3, "X", 5, "X", "X", "X", "X", "X", "X", "X", 7}

replaceDuplicates2[Nothing] @ {1, 2, 3, 2, 5, 2, 1, 5, 5, 2, 3, 3, 7}

{1, 2, 3, 5, 7}

• Very clever, and ten times faster than what I suggested. Cheers! Commented Apr 24, 2021 at 17:30
• I am trying to understand what your function is really doing. Commented Apr 24, 2021 at 20:03
• @azerbajdzan the heart of it is this definition f[y_] := (f[y] = rep; y) which will execute the first time it sees a value y and first perform the assignment f[y] = rep and then return y. That is why the values do not get replaced with "X" the first time they are seen. On the next time the value y is seen, because of the assignment f[y] = rep, the function simply returns y. All of this is wrapped in Module so it's cleaned up properly and then the function is mapped over the list of arguments. Commented Apr 25, 2021 at 2:54
• +1. One nitpick is that one would need to call Clear[f] (but not ClearAll!) at the end before returning the result, otherwise f will not be garbage-collected and will hang around indefinitely after the call to replaceDuplicates - and if you call it multiple times, then just as many different fs will be left hanging around. Commented Apr 26, 2021 at 11:56
• Thank you @Leonid. Excellent point.
– kglr
Commented Apr 26, 2021 at 12:04
FoldPairList[If[MemberQ[#1, #2], {"X", #1}, {#2, Append[##]}] &,
{},
{1, 2, 3, 2, 5, 2, 1, 5, 5, 2, 3, 3, 7}]

(*    {1, 2, 3, "X", 5, "X", "X", "X", "X", "X", "X", "X", 7}    *)


FoldPairList accumulates a list of "already seen" numbers, starting with the empty list. If the new number is in the "already seen" list, we emit "X" and keep the list unchanged; otherwise, we emit the new number and add it to the "already seen" list (with Append[##] being an abbreviation for Append[#1, #2]).

You may use PositionIndex with ReplacePart.

With

x = {1, 2, 3, 2, 5, 2, 1, 5, 5, 2, 3, 3, 7};


and

replaceDups[expr_, rep_ : "X"] :=
ReplacePart[expr, List /@ Flatten[Rest /@ Values@PositionIndex@expr] -> rep]


Then

replaceDups[x]

{1, 2, 3, "X", 5, "X", "X", "X", "X", "X", "X", "X", 7}


Is also faster than OP solution.

Hope this helps.

Not that elegant but rather straight.

A = {1, 2, 3, 2, 5, 2, 1, 5, 5, 2, 3, 3, 7}

ReplacePart[A, {#} & /@
Select[Range[Length[A]], MemberQ[Take[A, # - 1], A[[#]]] &] -> "X"]


Note that

Select[Range[Length[A]], MemberQ[Take[A, # - 1], A[[#]]] &]


is a set of indices those are duplicated.

list = {1, 2, 3, 2, 5, 2, 1, 5, 5, 2, 3, 3, 7};


Positions of duplicates

p = Splice @* Rest /@ Values @ PositionIndex[list]


{7, 4, 6, 10, 11, 12, 8, 9}

Using SubsetMap to replace

SubsetMap[Array["X"&,Length @ p]&, list, p]


{1, 2, 3, "X", 5, "X", "X", "X", "X", "X", "X", "X", 7}

list = {1, 2, 3, 2, 5, 2, 1, 5, 5, 2, 3, 3, 7};


Using MapAt:

p = Outer[List, Catenate[Rest /@ Values@PositionIndex[list]]]

(*{{7}, {4}, {6}, {10}, {11}, {12}, {8}, {9}}*)

MapAt["X" &, list, p]

(*{1, 2, 3, "X", 5, "X", "X", "X", "X", "X", "X", "X", 7}*)


Just an alternative relying on Ordering :

list = {1, 2, 3, 2, 5, 2, 1, 5, 5, 2, 3, 3, 7};

Permute[
KeyValueMap[Splice@ArrayPad[{#1}, {0, #2 - 1}, "X"] &]@KeySort@Counts@#
, Ordering@#] &@list


{1, 2, 3, "X", 5, "X", "X", "X", "X", "X", "X", "X", 7}