# Estimate parameters of AR1 process

I am trying to estimate the parameters of a series $Z_t=W_t^\beta$, where W_t follows an AR1 process with mean 2, $\rho=0.9$ and $\sigma=0.1$:

$W_t = \rho W_{t-1} + (1-\rho) \bar{W} + \epsilon_t$

where $\epsilon_t\sim N(0,\sigma)$. Which can be coded in Mathematica as:

data = Normal[RandomFunction[ARProcess[2 (1 - 0.9), {.9}, .1], {1, 10^2}]];
data = Flatten[data, 1][[All, 2]];
lZ = #^3.5 & /@ data;


I want to then estimate the parameter $\beta(=3.5\; \text{above})$, from the series $lZ$. However, we should have:

$Z_t^{1/\beta} = \rho Z_{t-1}^{1/\beta} + (1-\rho) \bar{Z}^{1/\beta} + \epsilon_t$

To estimate these parameters, I maximise a log-likelihood, of the form:

$l = -(n/2)\ln(2\pi) - (n/2)\ln(\sigma^2) - (1/{2\sigma^2})\sum_{i=1}^n (Z_t^{1/\beta} - \rho Z_{t-1}^{1\beta} - (1-\rho) \bar{Z}^{1\beta})^2$

Which I can maximise in Mathematica by defining the function:

fMax[lSeries__, zbar_] :=
NMaximize[{-0.5 (Length@lSeries - 1) Log[2 Pi] -
0.5 (Length@lSeries - 1) Log[sigmaSq] - (1/(2 sigmaSq)) Total[
MovingMap[((#[])^(1/beta) -
rho (#[]^(1/beta)) - (1 - rho) zbar^(1/beta))^2 &,
lSeries, {2}]], sigmaSq > 0, beta > 0, 0 < rho < 1}, {beta,
rho, sigmaSq}]
fMax[lZ,Mean@lZ]


However, when I do this, the function does not seem to return anything sensible:

Failed to converge to a solution. The function may be unbounded.


And the resultant estimates are nowhere near the actual parameter values. Does anyone know where I am going wrong here? Is my system under-identified?

It also occurs to me that if $\beta\rightarrow \infty$, that the sum in the log-likelihood will disappear! (This makes me think that perhaps I need a sort of Jacobian term to account for the change of variable?)

Best,

Ben

• I'm not sure about your code, but you can accomplish the task using EstimatedProcess[data, ARProcess, ProcessEstimator->{"MaximumLikelihood", Method->"NMaximize"}] – Stefan R Aug 17 '15 at 21:50
• Correct me if I'm wrong, this would just estimate the parameters for the $W_t$ process? I can do this, it is not a problem. The difficulty is using the process $Z_t$ to estimate the parameters above, particularly $\beta$. – ben18785 Aug 17 '15 at 21:52
• One of the terms is: -0.5 (Length@lSeries - 1) Log[sigmaSq]. This is maximized when sigmaSq->0 and the max is infinity. A second problem is you have the term zbar^(1/beta)^2 which does not agree with your latexed version of the process (parenthesis in the wrong place). – bill s Aug 19 '15 at 18:09
• @bills - this term would be maximised at sigmaSq = 0, but the term after it would go to infinity. This is just the standard normal likelihood. – ben18785 Aug 19 '15 at 18:12
• Why not simplify the code to find out which part is giving the problem? If you fix sigmaSq, does the rest of it work? – bill s Aug 19 '15 at 18:14

I have now figured out the issue. It was because I was not including a Jacobian term in the likelihood $\frac{dW}{dZ}=\frac{Z^{1/\beta-1}}{\beta}$. After doing so, the log-likelihood becomes:

$l = -(n/2)ln(2\pi) - (n/2)ln(\sigma^2) - n ln(\beta) + (1/\beta-1)\sum ln(Z) - (1/{2\sigma^2})\sum_{i=1}^n (Z_t^{1/\beta} - \rho Z_{t-1}^{1\beta} - (1-\rho) \bar{Z}^{1\beta})^2$

This can be maximised in Mathematica using the following:

fMax2[lSeries__, zbar_] :=
NMaximize[{-0.5 (Length@lSeries - 1) Log[2 Pi] -
0.5 (Length@lSeries - 1) Log[sigmaSq] - (Length@lSeries - 1) Log[
beta] + ((1/beta) - 1) Total[
Log@lSeries] - (1/(2 sigmaSq)) Total[
MovingMap[((#[])^(1/beta) -
rho (#[]^(1/beta)) - (1 - rho) zbar^(1/beta))^2 &,
lSeries, {2}]], sigmaSq > 0, beta > 0, 0 < rho < 1}, {beta,
rho, sigmaSq}]
fMax2[lZ, Mean@lZ]


This now returns reasonable estimates of the true parameter values.

Best,

Ben