I am a time series novice and have been learning the art/trade from examples while being governed by the NIST handbook (ch.6, sec.4).
My question is regarding:
- How to improve the estimated time series process? Currently the ACF of the residuals of my estimated process is not entirely Normally-distributed noise.
- Is this a time series problem at all? Should I use a distribution fit instead?
I have a dataset (x
) fluctuating between discrete values 1 - 5. The raw data and the "differenced" data are shown. The CSV data is available via this Pastebin link. It is comma-separated numeric data.
I converted the data to TemporalData
and performed a UnitRootTest
.
xtd = TemporalData[Differences[#], {Length[Differences[#]]}] & /@ x;
UnitRootTest[xtd[[1]], 1, "HypothesisTestData",
SignificanceLevel -> .05]
(*The null hypothesis that the model of order 1 without a constant offset and no deterministic trend contains a unit root is rejected at the 5. percent level based on the Dickey-Fuller F test.*)
The differenced data is stationary, as indicated by the small p-value from the UnitRootTest and the rejection of the $H_0$.
I used the CorrelationFunction
to visualize the autocorrelation of the resulting time series. There is one spike (negative) at the first lag and then a rapid decline to within the 95% confidence interval.
conflevel = 0.95;
samplesize = xtd[[1]]["PathLengths"][[1]];
{minconf, maxconf} =
Quantile[NormalDistribution[], {(1 - conflevel)/2,
1 - (1 - conflevel)/2}]/Sqrt[samplesize]
ListPlot[N@CorrelationFunction[xtd[[1]]["PathStates"], {100}],
PlotRange -> {All, {-1, 1.1}}, Filling -> Axis,
GridLines -> {None, {-0.05, 0.05}},
PlotStyle -> {Red, PointSize[0.015]}]
Based on my limited knowledge, it would appear that I may need an MAProcess
with one parameter since I have a spike at lag 1 along with a quick diminishing of the ACF to within the confidence interval (See Box-Jenkins identification from the NIST Handbook).
I estimate the MAProcess
parameters:
est = EstimatedProcess[xtd[[1]]["PathStates"], MAProcess[{a}, v]]
(*MAProcess[{-1.}, 0.78754]*)
Then I compute the residuals of the estimated process with respect to the original data hoping that the residuals would be normally distributed noise (their ACF would be within their 95% confidence interval in their ACF plot)... but the residuals do not stay within the confidence intervals!
So I assume that my estimate is not a good fit.
residuals =
Rest[xtd["PathStates"] -
KalmanFilter[est, xtd[[1]]]["PathStates"]];
clev = 0.95;
samplesize =
TemporalData[residuals, {Length@ residuals}]["PathLengths"][[1]];
{minconf, maxconf} =
Quantile[NormalDistribution[], {(1 - clev)/2,
1 - (1 - clev)/2}]/Sqrt[samplesize]
DiscretePlot[CorrelationFunction[residuals, n], {n, 0, 10},
ExtentSize -> 1/2, PlotRange -> {All, {-1, 1.1}}, PlotStyle -> Black,
Frame -> True, FrameStyle -> Black, FrameLabel -> {"lag", "ACF"},
GridLines -> {None, {-0.05, 0.05}}]
- What parameters should I add to my
MAProcess
? I have tried an ARMAProcess as suggested byTimeSeriesModelFit
but that has a worse residual ACF. - Is this a timeseries problem?
EstimatedProcess[xtd[[1]]["PathStates"], ARMAProcess[{a},{b}, v]]
I think I did but to not much improvement. $\endgroup$