3
$\begingroup$

I have some temperature decay data (2k points) that is very noisy for large time values. I don't want to do much or any smoothing to the data early on, but would like a large window to average over for the later data. Ideally, if I could do a moving average where the averaging window changes depending on where in the list you are (no averaging in the beginning and averaging over 30 or 40 values in the end)...I'm not sure how to implement this in mathematica.

Here's an example of the data (the lower graphs)

Unfiltered

I can use a Wiener Filter twice with a radius of 3 that cleans up the data fairly well, but I'd like to try a moving average to preserve earlier values and see if I can do better with the later data.

Filtered

Improve this question
$\endgroup$
1
  • $\begingroup$ Welcome to Mathematica.SE! I suggest the following: 1) As you receive help, try to give it too, by answering questions in your area of expertise. 2) Take the tour and check the faqs! 3) When you see good questions and answers, vote them up by clicking the gray triangles, because the credibility of the system is based on the reputation gained by users sharing their knowledge. Also, please remember to accept the answer, if any, that solves your problem, by clicking the checkmark sign! $\endgroup$
    – Chris K
    Oct 25, 2018 at 2:07

1 Answer 1

Reset to default
4
$\begingroup$

Since you did not supply any data, we'll start by making some up -- here's a function x[t] that gets noisier over time, and then it is sampled to give the data xsam:

x[t_] := Sin[t] + RandomReal[{-t/2, t/2}]; 
xsam = Table[x[t], {t, 0, 10, 0.01}];

A moving average can be implemented as the product of n samples of the data and a constant array of length n. Your goal is to increase n as time progresses. One way of implementing that is to have n increase linearly:

xfilt = Table[n = Ceiling[i/10]; 
              Total[xsam[[i ;; i+n]]*ConstantArray[1/(n+1), n+1]], 
               {i, 1, Length[xsam]-100}];

Here is a plot of the samples and the filtered version:

ListPlot[{Flatten[{ConstantArray[0, n], xfilt}], xsam}]

enter image description here

You can choose any rate of change for the filter length n by changing the functional relationship between n and i.

You asked for a moving average filter to do the smoothing. You might also consider an autoregressive filter for this task. Here is an analogous structure that uses alpha as a smoothing parameter -- I've started with 0.8 (little smoothing) and increased it to 0.99 (lots of smoothing). You can change that by redefining the alpha sequence.

xfilt = ConstantArray[0, Length[xsam]];
alpha = Range[0.8, 0.99, (0.99 - 0.8)/(Length[xfilt] - 1)];
Table[xfilt[[i + 1]] = 
   alpha[[i]]*xfilt[[i]] + (1-alpha[[i]])*xsam[[i]], {i, 1, Length[xsam]-1}];
ListPlot[{xsam, xfilt}]

enter image description here

One advantage of this approach is that you no longer have a varying time delay in the operation of the filter.

Improve this answer
$\endgroup$
2
  • $\begingroup$ Thanks so much for the response! I'm getting the error Table::iterb: Iterator {i,1,999-n} does not have appropriate bounds. when running the filter though...Also it seems like it shifts and changes the length of the filtered data...I'll have to spend some time looking at it unless it's something obvious I'm missing. $\endgroup$
    – 2close2see
    Oct 24, 2018 at 22:22
  • $\begingroup$ Sorry -- I had an indexing error -- it should now work without error. When you take an n-term moving average, you get a delay proportional to n, so it is necessary to put the delay somewhere. The above code places it at the beginning, where not much is happening. $\endgroup$
    – bill s
    Oct 24, 2018 at 22:47

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge that you have read and understand our privacy policy and code of conduct.

Not the answer you're looking for? Browse other questions tagged or ask your own question.