# LogLikelihood function with Multivariate State Space Models

I load the packadge TimeSeries. Depending on the running Mathematica release, one should preliminary remove some built-in function with the same name, e.g.KalmanFilter, I used the following commands:

ClearAttributes[KalmanFilter, {Protected, ReadProtected}];
Clear[KalmanFilter];
Needs["TimeSeriesTimeSeries"]


I consider the following Bi-variate State Space Model (the symbols F, G, Q, R, c, d for the matrices of my model follow Mathematica Structural Models and the Kalman Filter (http://reference.wolfram.com/applications/timeseries/UsersGuideToTimeSeries/StructuralModelsAndTheKalmanFilter/index.en.html)

(F = {{-0.5, f12}, {0., -0.3}; Q={{1., 0.}, {0., 1.}}; G={{g11, 1.}, {0., g22}}; R={{1., 0.}, {0., 1.}}; c = {0., 0.}; d = {d1, 0.};)


I build the state covariance function

getP[X_, Y_] :=
Partition[
Inverse[IdentityMatrix[#^2] -
Flatten[Map[Flatten, Transpose /@ Outer[Times, X, X], {2}],
1]].Flatten[Y], #] &[Length[X]]
P = getP[F, Q];


Note that one could also get P as the solution of the equation P==FPTranspose[F]+Q. I use a Bi-dimensional data vector

data = RandomVariate[BinormalDistribution[1/2], 10];


I define the functions f and if as prescribed (the absence of d1 in P seems to be irrelevant)

Clear[f, if];
if = Function[Evaluate[P /. {f12 -> #1, g11 -> #2, g22 -> #3, d1 -> #4}]];
f[f12_?NumericQ, g11_?NumericQ, g22_?NumericQ, d1_?NumericQ] :=
-LogLikelihoodFunction[data, {{0, 0}, if[f12, g11, g22, d1]}, F, G, Q, R, c, d]


I define starting points

{f120 = RandomReal[{0.00, 1.00}], g110 = RandomReal[{-5.00, 5.00}],
g220 = RandomReal[{-5.00, 5.00}], d10 = RandomReal[{-5.00, 5.00}]};


I try to compute the LogLikelihoodFunction (this is the correct name of the documented LogLikelihood function in the packadge Time Series) at the given points

SP = {f12, g11, g22, d1} //.
{f12 -> f120, g11 -> g110, g22 -> g220, d1 -> d10};
f[f12, g11, g22, d1]
Thread::tdlen: Objects of unequal length in {{0.,0.},{1.53649,-0.102148},<<6>>,
{1.30567,-0.121967},{1.84233,0.269059}}+<<1>> cannot be combined. >>

{{0.320751,0.790727},<<8>>,{0.27184,0.60035}} at position {2, 1} in
MapThread[#1.Inverse[#2].#1&,<<1>>] has only 0 of required 1 dimensions. >>

Thread::tdlen: "Objects of unequal length in 35.9506 +(#1.Inverse[#2].#1&)+{-<<18>>,
<<3>>}+{<<1>>}+{{0.320751,0.790727},<<8>>,{0.27184,0.60035}} cannot be combined. >>


I get the above two types of error messages. Surprisingly, despite the red color of some arguments (too many arguments given) the KalmanFilter function seems to work!

 KalmanFilter[data, {{0, 0}, if[f12, g11, g22, d1]}, F, G, Q, R, c, d] // MatrixForm


Of course the minimum of the LogLikelihoodFunction cannot be computed. I tried with different matrices for R also with the null matrix. No differences. I've no problems with Uni-variate State State models. I hope that Somebody can help me! thank you in advance for your kind attention.

• could you make a simpler version of your question? It seems unlikely that the problem arises because of 10 dimensions? – chris Sep 5 '14 at 6:52
• I agree with @chris, despite anybody's good intentions, its most likely to get an answer from a simpler problem. – rhermans Sep 5 '14 at 7:36
• Thank you very much for your comments! Actually, the version that I posted comes from a more complicated problem concerning a five-dimensional state vector estimation by bi-dimensional observation. I already reduced the dimension of the state space vector to three. My concern about reducing further the state vector dimension is to avoid special cases in which the problem could disappear. However, the focus should be on the dimension of the observation vector that cannot be further reduced. If the observations are not a vector, but a scalar time series, the problem does not arise. – Roberto Monte Sep 5 '14 at 7:51
• Following chris and rhermans' suggestion, I reduced the problem to a bi-variate state vector and bi-variate observation vector. I keep on getting the same problem. Likelihoodfunction does not work while with the same argument (copy and paste!) KalmanFilter works. – Roberto Monte Sep 5 '14 at 8:31