Replace missing data

I have vectors of equal lengths where I want to replace the zeroes (missing data) with their preceding values. So far I have written:

dt = {1, 0, 2, 0, 3};

MapThread[(dt[[#1]] = dt[[#2]]) &, {#, #-1}] & [Position[dt, 0]];

dt


{1, 1, 2, 2, 3}

I have 3 questions:

(1) Are there alternatives to the above code, which I don't like aesthetically ?

(2) How can I expand it to go over matrices of equal-length-vectors ?

(3) How can I deal with a zero in the first position (replace by the following element) ?

• Should multiple zeros in a row all be replaced, e.g. {2, 0, 0, 7} -> {2, 2, 2, 7}? Commented Mar 20, 2015 at 16:17
• Yes. The problem seems familiar to me too. Wasn't able to locate it.
– eldo
Commented Mar 20, 2015 at 16:32
• Related: (26723). Somewhat related: (57526) Commented Mar 20, 2015 at 16:41
• Possible duplicate: (2952) Commented Mar 20, 2015 at 16:49
• What if the first element is zero? Commented Jan 22, 2018 at 19:00

I know this is a duplicate but since I failed to find it with ten minutes of searching:

x = {2, 0, 0, 7, 0, 3, 0, 0, 1};

FoldList[If[#2 == 0, #, #2] &, x]

{2, 2, 2, 7, 7, 3, 3, 3, 1}


x = {0, 7, 0, 3, 0, 0, 1};

FoldList[If[#2 == 0, #, #2] &, SelectFirst[x, # != 0 &], Rest @ x]

{7, 7, 7, 3, 3, 3, 1}


The case without leading zeroes can also be handled nicely using Split, borrowed from Leonid's method for Fill out blanks with a upcoming number in a list?:

x = {2, 0, 0, 7, 0, 3, 0, 0, 1};

Join @@ Accumulate /@ Split[x, #2 == 0 &]

{2, 2, 2, 7, 7, 3, 3, 3, 1}


The leading zeros can be handled by prepending the first non-zero value to the list, then applying Rest at the end.

For Q2 I think all you need is to convert either method to a function and map it:

fn[x_] := FoldList[If[#2 == 0, #, #2] &, SelectFirst[x, # != 0 &], Rest @ x]

fn /@ {{2, 0, 0, 7, 0, 3, 0, 0, 1}, {0, 1, 2, 7, 0, 3, 0, 0, 1}}

{{2, 2, 2, 7, 7, 3, 3, 3, 1}, {1, 1, 2, 7, 7, 3, 3, 3, 1}}

• Phantastic !!! How would you map it over matrices ? (Q2)
– eldo
Commented Mar 20, 2015 at 16:36
• @eldo Would you please provide an example? Commented Mar 20, 2015 at 16:41
• --------------{x,x}
– eldo
Commented Mar 20, 2015 at 16:50
• @eldo Sorry, I still don't understand. Commented Mar 20, 2015 at 16:53
• Dear Wizard, my real situation is like this: x = {{2, 0, 0, 7, 0, 3, 0, 0, 1}, {0, 1, 2, 7, 0, 3, 0, 0, 1}}
– eldo
Commented Mar 20, 2015 at 17:01

TemporalData + MissingDataMethod

ClearAll[interpolateMissing]
interpolateMissing = Quiet @ TemporalData[# /. 0 -> Missing[], Automatic,
MissingDataMethod -> {"Interpolation", InterpolationOrder -> 0}]["States"] &;


Examples:

interpolateMissing  @ {1, 0, 2, 0, 3}


{{1, 1, 2, 2, 3}}

interpolateMissing  @ {0, 7, 0, 3, 0, 0, 1}


{{7, 7, 7, 3, 3, 3, 1}}

interpolateMissing  @ {0, 0, 0, 0, 7, 0, 3, 0, 0, 1}


{{7, 7, 7, 7, 7, 7, 3, 3, 3, 1}}

At the price of extra pair braces in the output when the input is a list, we get the second requirement

How can I expand it to go over matrices of equal-length-vectors?

for free, since interpolateMissing works as is when the input is an array:

interpolateMissing  @ {{1, 0, 2, 0, 3, 0, 1}, {0, 7, 0, 3, 0, 0, 1}}


{{1, 1, 2, 2, 3, 3, 1}, {7, 7, 7, 3, 3, 3, 1}}

Note: Quiet is used to suppress the following warning:

InterpolatingFunction::dmval: Input value {0} lies outside the range of data in the interpolating function. Extrapolation will be used. >>

By the way, Extrapolation is exactly what we need to handle cases with leading zeros according to requirement (3).

• Maybe "replaceMissing" should be "interpolateMissing"? Commented Jan 22, 2018 at 16:29
• @Rabbit, yes, thank you.
– kglr
Commented Jan 22, 2018 at 18:26

You can also use repeated replacement 1,3) The first two rules handle leading zeros, the first one handles several leading zeros and the second one handles a single leading zero.:

cleanVector[v_]:=v//.{ {0, x:0___,y_,z___}:>{y,x,y,z},
{0, x:0...,y_,z___}:>{y,x,y,z},
{x___,y_,0,z___} :> {x,y,y,z} };


2) Depending on what you mean by 'go over matrices' I'd just map this over the matrix to clean each vector independently.

m={v1,v2,v3};
cleanVector/@m;