How to reconstruct a missing data point

In the output of a large computer simulation(thousands of cpus for 3 days), I found that one data point is missing in the result. Is it possible to reconstruct the missing value using the remaining data?

Assumptions: The dynamics in the data is generally smooth and there is no shape feature in it.

Here is the data:

data={-0.0949456,-0.2232,-0.15012,0.0364052,0.0841621,-0.0920731,-0.273141,-0.183935,0.0656169,0.140269,-0.0843562,-0.332979,-0.228146,0.0969235,0.208076,-0.0696226,-0.402307,-0.284266,0.128555,0.287427,-0.0454046,-0.479687,-0.353367,0.158121,0.377113,-0.00921489,-0.56245,-0.43569,0.182807,0.474626,0.0409945,-0.646659,-0.530218,0.199767,0.576109,0.106226,-0.727344,-0.634338,0.206674,0.676612,0.185864,-0.799108,-0.743778,0.202287,0.770614,0.277269,-0.856969,-0.852921,0.18688,0.85286,0.,-0.897303,-0.955487,0.162245,0.919149,0.475145,-0.91851,-1.04548,0.131507,0.967006,0.568474,-0.921269,-1.11803,0.0983087,0.995824,0.649178,-0.908188,-1.17004,0.0660878,1.00669,0.711966,-0.883046,-1.20025,0.0374795,1.00158,0.753415,-0.849936,-1.20902,0.0140558,0.983158,0.772116,-0.8123,-1.19776,-0.00394007,0.953394,0.768557,-0.772323,-1.16867,-0.0170188,0.914086,0.744676,-0.731382,-1.12393,-0.0260316,0.866382,0.703393,-0.689571,-1.06621,-0.0322879,0.811196,0.648481,-0.646799,-0.997935,-0.0368642,0.74929,0.583615,-0.602475,-0.921481,-0.0407482,0.681664,0.513284,-0.555671,-0.839684,-0.0446284,0.609182,0.440653,-0.507031,-0.755086,-0.0489018,0.533683,0.369238,-0.456793,-0.669996,-0.0537497,0.457136,0.301312,-0.405981,-0.586799,-0.0589172,0.381381,0.238713,-0.355861,-0.507526,-0.064281,0.308536,0.182517,-0.30782,-0.433796,-0.0694734,0.240373,0.133182,-0.263169,-0.366831,-0.0741541,0.178336,0.0906948,-0.222976,-0.30744,-0.0780402,0.123434,0.0547048,-0.187986,-0.25604,-0.0809698,0.0762321,0.0246802,-0.158568,-0.212644,-0.0828802,0.0368173,0.};


Also from other output information, I can determined that the problem data is at the index 51, which is set to 0 in the above.

So is it possible to reconstruct that value?

Here is my try:

I use the wavelet transform to identify the abrupt changes in the data, and manually adjust the data value, until I get a relative smooth wavelet transform.

Manipulate[
data1 = data;
data1[[51]] = data[[51]] + Abs[Mean[{data[[50]], data[[52]]}]]*x;
cwd = ContinuousWaveletTransform[data1,
GaborWavelet[10], {Automatic, 12}];
values = Abs[cwd[All, "Values"]];
ListDensityPlot[values, ColorFunction -> "TemperatureMap",
PlotRange -> {All, {0, 50}, {0, 0.1 Max[values]}},
ClippingStyle -> Automatic, InterpolationOrder -> 0]
,
{x, -10, 30, 1}
]


Form above I determined by eye that when x=17 the wavelet is smoothest. So the missing data I get is 0.377765. And here is a plot of the data and the wavelet transform after the fix

Question:

1. What are the general ways of reconstruct missing data?
2. Are there other methods to reconstruct the missing data, without looking by eye?
• – user484
Nov 23 '13 at 3:52
• Update: I tried the approach in that question, but it doesn't work too well thanks to the rapid oscillation in the data. Only with InterpolationOrder -> 100 does it get close to the expected result, and with even higher orders it becomes unstable.
– user484
Nov 23 '13 at 4:00
• Very nice idea indeed to use animation for manual determination of "the best" value! Nov 23 '14 at 13:16

I don't think there are "general methods". Normally interpolation or curve fitting can be used.

Let's see your particular problem. You have five distinct samplings:

ListLinePlot[data[[# ;; -1 ;; 5]] & /@ Range@5]


The first one shows the problem:

ListLinePlot[data[[1 ;; -1 ;; 5]]]


Let's see which point is the outlier:

First@Ordering[-Abs@Differences@data[[1 ;; -1 ;; 5]]]
(*
11
*)


Let's calculate a "right" value

Mean[{data[[1 ;; -1 ;; 5]][[10]], data[[1 ;; -1 ;; 5]][[12]]}]
(*
0.376207
*)


And now we replace it in the original sequence

ListLinePlot@ ReplacePart[data, Position[data, 0.][[1]] -> 0.37620699999999996]


Edit

If you want a slightly better approximation, you could do something like:

pata = Transpose[{Range@Length@data, data}];
Interpolation[Join[pata[[1 ;; -1 ;; 5]][[5 ;; 10]],
pata[[1 ;; -1 ;; 5]][[12 ;; 18]]]][51]

(*
0.375886
*)
`
• That's clever. How do you determine the number 5? What do you mean by "five distinct samplings"? Nov 23 '13 at 0:00
• @xslittlegrass Your sampling freq is almost five times the highest freq of the signal. You are sampling five times each period Nov 23 '13 at 0:06