3
$\begingroup$

I am trying to estimate the parameters of a series $Z_t=W_t^\beta$, where W_t follows an AR1 process with mean 2, $\rho=0.9$ and $\sigma=0.1$:

$W_t = \rho W_{t-1} + (1-\rho) \bar{W} + \epsilon_t$

where $\epsilon_t\sim N(0,\sigma)$. Which can be coded in Mathematica as:

data = Normal[RandomFunction[ARProcess[2 (1 - 0.9), {.9}, .1], {1, 10^2}]];
data = Flatten[data, 1][[All, 2]];
lZ = #^3.5 & /@ data;

I want to then estimate the parameter $\beta(=3.5\; \text{above})$, from the series $lZ$. However, we should have:

$Z_t^{1/\beta} = \rho Z_{t-1}^{1/\beta} + (1-\rho) \bar{Z}^{1/\beta} + \epsilon_t$

To estimate these parameters, I maximise a log-likelihood, of the form:

$l = -(n/2)\ln(2\pi) - (n/2)\ln(\sigma^2) - (1/{2\sigma^2})\sum_{i=1}^n (Z_t^{1/\beta} - \rho Z_{t-1}^{1\beta} - (1-\rho) \bar{Z}^{1\beta})^2$

Which I can maximise in Mathematica by defining the function:

fMax[lSeries__, zbar_] := 
 NMaximize[{-0.5 (Length@lSeries - 1) Log[2 Pi] - 
0.5 (Length@lSeries - 1) Log[sigmaSq] - (1/(2 sigmaSq)) Total[
  MovingMap[((#[[2]])^(1/beta) - 
     rho (#[[1]]^(1/beta)) - (1 - rho) zbar^(1/beta))^2 &, 
     lSeries, {2}]], sigmaSq > 0, beta > 0, 0 < rho < 1}, {beta, 
     rho, sigmaSq}]
fMax[lZ,Mean@lZ]

However, when I do this, the function does not seem to return anything sensible:

Failed to converge to a solution. The function may be unbounded.

And the resultant estimates are nowhere near the actual parameter values. Does anyone know where I am going wrong here? Is my system under-identified?

It also occurs to me that if $\beta\rightarrow \infty$, that the sum in the log-likelihood will disappear! (This makes me think that perhaps I need a sort of Jacobian term to account for the change of variable?)

Best,

Ben

$\endgroup$
  • $\begingroup$ I'm not sure about your code, but you can accomplish the task using EstimatedProcess[data, ARProcess[1], ProcessEstimator->{"MaximumLikelihood", Method->"NMaximize"}] $\endgroup$ – Stefan R Aug 17 '15 at 21:50
  • $\begingroup$ Correct me if I'm wrong, this would just estimate the parameters for the $W_t$ process? I can do this, it is not a problem. The difficulty is using the process $Z_t$ to estimate the parameters above, particularly $\beta$. $\endgroup$ – ben18785 Aug 17 '15 at 21:52
  • $\begingroup$ One of the terms is: -0.5 (Length@lSeries - 1) Log[sigmaSq]. This is maximized when sigmaSq->0 and the max is infinity. A second problem is you have the term zbar^(1/beta)^2 which does not agree with your latexed version of the process (parenthesis in the wrong place). $\endgroup$ – bill s Aug 19 '15 at 18:09
  • $\begingroup$ @bills - this term would be maximised at sigmaSq = 0, but the term after it would go to infinity. This is just the standard normal likelihood. $\endgroup$ – ben18785 Aug 19 '15 at 18:12
  • $\begingroup$ Why not simplify the code to find out which part is giving the problem? If you fix sigmaSq, does the rest of it work? $\endgroup$ – bill s Aug 19 '15 at 18:14
1
$\begingroup$

I have now figured out the issue. It was because I was not including a Jacobian term in the likelihood $\frac{dW}{dZ}=\frac{Z^{1/\beta-1}}{\beta}$. After doing so, the log-likelihood becomes:

$l = -(n/2)ln(2\pi) - (n/2)ln(\sigma^2) - n ln(\beta) + (1/\beta-1)\sum ln(Z) - (1/{2\sigma^2})\sum_{i=1}^n (Z_t^{1/\beta} - \rho Z_{t-1}^{1\beta} - (1-\rho) \bar{Z}^{1\beta})^2$

This can be maximised in Mathematica using the following:

fMax2[lSeries__, zbar_] := 
 NMaximize[{-0.5 (Length@lSeries - 1) Log[2 Pi] - 
0.5 (Length@lSeries - 1) Log[sigmaSq] - (Length@lSeries - 1) Log[
  beta] + ((1/beta) - 1) Total[
  Log@lSeries] - (1/(2 sigmaSq)) Total[
  MovingMap[((#[[2]])^(1/beta) - 
       rho (#[[1]]^(1/beta)) - (1 - rho) zbar^(1/beta))^2 &, 
   lSeries, {2}]], sigmaSq > 0, beta > 0, 0 < rho < 1}, {beta, 
rho, sigmaSq}]
fMax2[lZ, Mean@lZ]

This now returns reasonable estimates of the true parameter values.

Best,

Ben

$\endgroup$
0
$\begingroup$

You've learned a valuable lesson... pesky things those "densities" - they aren't born in isolation but together with the "measure" they are defined with so the whole mathematical object becomes invariant.

"Moral of story: Transforming densities is always like a "change of variables" when integrating!

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.