# Deal with noise and missing points in data [closed]

I have run a simulation which has resulted in some bad data (see graph below). What would be the best way to detect and interpolate the missing points in Mathematica (i.e. remove the noise)?

Raw data here: array.dat or here:

Get["https://pastebin.com/raw/cRYScdfX"];
ListDensityPlot[LatticeConstantData1,
ScalingFunctions -> {"Log10", "Log10", "Linear"},
InterpolationOrder -> 0]


One idea: I was wondering if I could use something along the lines of image processing techniques, such as pixel averaging/smoothening, noise removal. Or could I even load the raw data into an external program to do this?

All ideas and suggestions greatly appreciated!

• You could try median filtering – mikado Sep 27 '18 at 20:54
• Thank you for the suggestion, I will give this a try... – Bart Sep 28 '18 at 7:37
• "Interpolate the missing points" & "remove the noise" – that's contradictory to me: if you have missing points, you can use Interpolation. Removing noise would require, e.g., applying some smoothing function. So what is it that you really want to do? – corey979 Sep 29 '18 at 15:44
• Apologies for not providing context: this is a phase diagram with three regions. Hence, there is a white bar of missing data and spurious blue points in the yellow region, and visa versa. The white bar is missing data and I was suggesting to eliminate this by interpolation. For the blue points in the yellow region, for example, I was also suggesting to eliminate these by interpolation, since I know roughly what the plot should look like. – Bart Oct 3 '18 at 15:21