# Fitting higher order ARIMA models hangs, never completes

I'm working with some data that is 12 month seasonal. However, when I try to fit it to an ARIMA(12,1,3) for instance, my PC's CPU cranks up to 100% and the job never finishes. Here's some sample code.

monthlyObservations = TimeSeries[
WeatherData["KORD", "Temperature", {{2012}, {2014}, "Month"}],
TemporalRegularity -> True]
proc = EstimatedProcess[monthlyObservations, ARIMAProcess[12, 1, 3]]
forecast2 = TimeSeriesForecast[proc, monthlyObservations, {12}]


In theory (I hope) this should be fitting this weather data to an ARIMA(12,1,3) and estimating parameters. However, this starts running and just begings chugging and never completes. I'm not familiar with Mathematica's estimation process - is it attempting to do something computationally unreasonable?

For instance, lower order ARIMA works fine:

proc = EstimatedProcess[monthlyObservations, ARIMAProcess[6, 1, 2]]
forecast2 = TimeSeriesForecast[proc, monthlyObservations, {12}]


and gives me this response, almost instantly

ARIMAProcess[0.0435174, {0.453363,
0.366996, -0.385445, -0.0176661, -0.156695, -0.206585}, 1, \
{-0.174108, -0.149191}, 11.0761]


So, my question is - what's up with how Mathematica handles higher order ARIMA calculations? Is there something obvious that I am doing wrong (ie, is ARIMA even the right tool? Am I doing something dumb?

(Quick note): I am aware that you can use the TimeSeriesFit[] to do a similar process as above. However, this also struggles when finding higher order ARIMA(12,1,x) as the best fit and there are some genuine cases where I have specific domain knowledge about the generating Process and want to just estimate specific parameters.

For seasonal data you probably want SARIMA which is a more parsimonious way to work with high order ARIMA models. This is especially true given the small amount of data you are working with.

You can use TimeSeriesModelFit at various levels of automation. By default it will just try to pick the best model from its list of potential families.

mod = TimeSeriesModelFit[monthlyObservations];

mod["BestFit"]

(* SARIMAProcess[-6.66, {-1.}, 1, {}, {11, {}, 3, {}}, 7.4498, {}] *)

mod["AIC"]

(* 78.2947 *)


This is suggesting that there is a first order nonseasonal trend and a third order seasonal trend with seasonality 11 (which is not the expected 12).

You can always add restrictions as well. Here I force the seasonality to 12 since I know that this is the reality of the situation.

mod = TimeSeriesModelFit[monthlyObservations, {"SARIMA", 12}];

mod["BestFit"]

(* SARIMAProcess[0.325088, {0.287926}, 1, {-0.834235}, {12, {-0.181923}, 1, {}}, 9.43762] *)

mod["AIC"]

(* 90.8093 *)


Or if you really do want your ARIMA(12,1,3)

mod = TimeSeriesModelFit[monthlyObservations, {"ARIMA", {12, 1, 3}}];

mod["BestFit"]

(* ARIMAProcess[0.0631419, {0.224889, 0.090208, -0.199309,
0.120568, -0.281502, -0.269343, -0.243869, -0.215336, 0.280738,
0.0648557, -0.123057, 0.047778}, 1, {-0.160033, 0.280475,
0.0344021}, 8.6651] *)

mod["AIC"]

(* 111.735 *)


Notice that, according to AIC, your model does not fit as well and so it isn't surprising that automated routines fail to pick it. What is a little surprising is that the automated routine picks a seasonality of 11. Keep in mind that the AIC is lower for the automatic model. This is a shining example of why we still need to inject a little common sense into data analysis rather than relying on things to be fully automated.

I recommend reading the section on model specification for TimeSeriesModelFit under scope to get a better feel for they ways you can direct it toward the type of model you are interested in.

• Thanks Andy - I knew I could specify the model type (ie ARIMA, SARIMA, etc) but didn't know I could also specify likely parameters using the TimeSeriesModelFit function. Thanks for your help. Do you know if there are computational issues with high order ARIMA or if this is a bug? – Tom Hayden Dec 20 '14 at 22:48
• I believe EstimatedProcess is just using a different more general algorithm that doesn't scale particularly well but allows for the flexibility to fix some parameter values. – Andy Ross Dec 20 '14 at 23:12

It appears that the default estimation methods do not like the fact that you set TemporalRegularity->True, which essentially forces Mathematica to treat it as regularly sampled when it really isn't so. A better way might be to use TimeSeriesResample. E.g.

monthlyObservations =
TimeSeries[
WeatherData["KORD", "Temperature", {{2012}, {2014}, "Month"}]];

EstimatedProcess[TimeSeriesResample@monthlyObservations,
ARIMAProcess[12, 1, 3]] // AbsoluteTiming

(* {0.159959,
ARIMAProcess[
0.0298329, {0.20517,
0.799777, -0.154676, -0.298009, -0.0641139, -0.195247, -0.0989965, \
-0.0804434, -0.0175942, 0.375415, 0.0204712, -0.292919},
1, {0.283061, -0.487802, -0.205239}, 6.07723]} *)

• I think this might be considered a bug then. By setting TemporalRegularity to true it should completely ignore the actual time stamps in the fitting. – Andy Ross Dec 22 '14 at 22:43
• What if I don't really want to resample my data? – Tom Hayden Dec 23 '14 at 18:23
• Andy, Tom : I've alerted the developers to this issue internally. I'll let you know what they have to say. – Stefan R Dec 23 '14 at 21:26
• @TomHayden you don't want to resample but want to assume regularity then just extract the values and fit with those. – Andy Ross Dec 24 '14 at 20:55