32
$\begingroup$

As we know, for most Gaussian integrals, we can get the analytical result. Now I have many Gaussian integrals to treat, which have the following general form,

Integrate[
   x1^n1 x2^n2 x3^n3 ... xd^
     nd Exp[-{x1, x2, x3 ... xd}.matrix.{x1, x2, x3 ... xd} + c1 x1 + 
        c2 x2 ... cd xd], {x1, -Infinity, Infinity},
     {x2, -Infinity, Infinity} ... {xd, -Infinity, Infinity}]. 

The matrix is a symmetrical positive definite matrix.

We can get the analytical integral result for this general form. However, it is a difficult task for Mathematica to deal with the similiar expressions, which takes so much time. I have written a package for treating this problem.

But, it is still not sufficient because I need to deal with lots of these kinds of integral expressions. So the computation time is important.

So have you ever seen the package for doing this kind of Gaussian integral? Or can you give me some suggestions about this problem? Thank you very much!

Thanks all of you. For the concept MultinormalDistribution, it is very helpful for me.

By the way, How can I transform the following integral expression to the standard form of MultinormalDistribution in a simple way?

vec1 := {r1x, r1y, r1z};
vec2 := {r2x, r2y, r2z};
vect := {r1x, r1y, r1z, r2x, r2y, r2z};
vcof := {c1, c2, c3, c4, c5, c6};
Integrate[Exp[-vect.mat.vect + vcof.vect] (vec1.vec2)^n (vec1.vec1)^
l (vec2.vec2)^m, {r1x, -Infinity, Infinity}, {r1y, -Infinity, 
Infinity}, {r1z, -Infinity, Infinity}, {r2x, -Infinity, 
Infinity}, {r2y, -Infinity, Infinity}, {r2z, -Infinity, Infinity}];

The mat is a symmetrical positive definite sparse matrix. And n, l, m are integers.
Thanks!

$\endgroup$
5
  • $\begingroup$ Your SPD matrix is dense or sparse? $\endgroup$ Jun 15, 2012 at 6:11
  • $\begingroup$ Just like this one {{3, 0, 0, 1, 0, 0}, {0, 4, 0, 0, 1, 0}, {0, 0, 4, 0, 0, 1}, {2, 0, 0, 8, 0, 0}, {0, 2, 0, 0, 6, 0}, {0, 0, 2, 0, 0, 6}} // MatrixForm Maybe it can be called sparse matrix. $\endgroup$
    – simpler
    Jun 16, 2012 at 1:43
  • $\begingroup$ Yes, it's sparse. You might be interested to find that Mathematica supports SparseArray[]. $\endgroup$ Jun 16, 2012 at 1:49
  • $\begingroup$ I only noticed just now that you added a follow-up question. See my updated answer, I hope that will be enough for you to put everything together. $\endgroup$
    – Jens
    Jun 18, 2012 at 1:45
  • $\begingroup$ I like this one, but can one add an unknown variable inside the g. For example like $e^{-ax^2-bxy-cy^2}$ and it output like something divides $\sqrt{4ac-b^2}$ $\endgroup$
    – en kuo
    Jul 11, 2022 at 20:09

3 Answers 3

35
$\begingroup$

What you have is a MultinormalDistribution. The quadratic and linear forms in the exponential can be rewritten in terms of $-\frac12(\vec{x}-\vec{\mu})^\top\Sigma^{-1}(\vec{x}-\vec{\mu})$ where $\vec{\mu}$ represents the mean and $\Sigma$ the covariance matrix, see the documentation.

With this, you can do integrals of the type given in the question by invoking Expectation, as in this example:

Expectation[
 x^2 y^3, {x, y} \[Distributed] 
  MultinormalDistribution[{μ1, μ2}, 
  {{σ1^2, ρ σ1 σ2}, 
   {ρ σ1 σ2, σ2^2}}]]

The result is:

$\text{$\mu $1}^2 \text{$\mu $2}^3+\text{$\mu $2}^3 \text{$\sigma $1}^2+6 \text{$\mu $1} \text{$\mu $2}^2 \rho \text{$\sigma $1} \text{$\sigma $2}+3 \text{$\mu $1}^2 \text{$\mu $2} \text{$\sigma $2}^2+3 \text{$\mu $2} \text{$\sigma $1}^2 \text{$\sigma $2}^2+6 \text{$\mu $2} \rho ^2 \text{$\sigma $1}^2 \text{$\sigma $2}^2+6 \text{$\mu $1} \rho \text{$\sigma $1} \text{$\sigma $2}^3$

Edit

Regarding the normalization prefactor mentioned in Sjoerd's comment, you can use the fact that for any dimension $n$

$\iint\exp(-\frac{1}{2}\vec{z}^\top \Sigma^{-1}\vec{z})\mathrm dz^n = (2\pi)^{n/2}\sqrt{\det(\Sigma)}$

Hopefully these hints will be enough for you to fill in the missing linear-algebra steps to make the connection to your given matrix matrix.

Edit 2

In response to the comment by chris, for polynomials as prefactors one can also use the slightly simpler but equivalent form

Moment[
 MultinormalDistribution[{μ1, μ2}, 
  {{σ1^2, ρ σ1 σ2}, 
   {ρ σ1 σ2, σ2^2}}],
 {2, 3}]

This is the same example as above, with the powers of x and y appearing in the second argument. See the documentation for Moment.

The difference between Moment and Expectation is that Moment is restricted to the expectation values of polynomials.

Edit 3

Before going on with the symbolic manipulations that I assumed are desired here, let me also point out that you can do your integrals pretty straightforwardly if your integrand contains no symbolic parameters. Then you just need to do a numerical integral by replacing Integrate with NIntegrate.

But now back to the symbolic part:

A follow-up question arose how to complete the square in the exponential to get to the standard form of the multinormal distribution, starting from a form like this:

$$\exp(\,\vec{x}^\top A\vec{x}+\vec{v}^\top\vec{x})$$

The matrix $A$ in the exponential is symmetric, $A^\top=A$, and positive definite. Therefore $A$ is invertible, and the inverse is symmetric,

$$\left(A^{-1}\right)^\top=A^{-1}$$

With this, you can verify

$$\left(\vec{x}+\frac{1}{2}A^{-1}\vec{v}\right)^\top A\left(\vec{x}+\frac{1}{2}A^{-1}\vec{v}\right)=\vec{x}^\top A\vec{x}+\vec{v}^\top\vec{x}+\frac{1}{4}\vec{v}^\top A^{-1}\vec{v} $$

by directly multiplying out the factors on the left. Therefore,

$$\exp(\vec{x}^\top A\vec{x}+\vec{v}^\top \vec{x})=\exp(\left(\vec{x}-\vec{\mu}\right)^\top A\left(\vec{x}-\vec{\mu}\right)-\frac{1}{4}\vec{v}^\top A^{-1}\vec{v})$$

where

$$\vec{\mu}\equiv-\frac{1}{2}A^{-1}\vec{v} $$

Compare this to the standard form of the Gaussian integral, and you see that in the notation of Mathematica's documentation

$$A \equiv -\frac{1}{2} \Sigma^{-1}$$

and our integral differs from the standard Gaussian one by a factor

$$\exp(-\frac{1}{4}\vec{v}^\top A^{-1}\vec{v})$$

Now we have all the pieces that are needed, except that you still have to calculate the inverse matrix $A^{-1} \equiv -2\Sigma$, using

Inverse[mat]

if I go back to your original notation where the matrix $A$ is called mat.

Edit 4:

In view of the other answers, I put together the above steps in a module so that my approach can be compared more easily to the alternatives. The result is quite compact and is not significantly slower than the fastest alternative (by @ybeltukov):

gaussMoment[fPre_, fExp_, vars_] := 
 Module[{coeff, dist, ai, μ, norm},
  coeff = CoefficientArrays[fExp, vars, "Symmetric" -> True];
  ai = Inverse[2 coeff[[3]]];
  μ = -ai.coeff[[2]];
  dist = MultinormalDistribution[μ, -ai];
  norm = 1/PDF[dist, vars] /. Thread[vars -> μ];
  Simplify[
   norm Exp[1/2 coeff[[2]].μ + coeff[[1]]] Distribute@
     Expectation[fPre, vars \[Distributed] dist]]]

The normalization factor can be easily obtained from the PDF. I used the same approach as @ybeltukov to extract the matrix $A$ from the exponent, except that I added a factor of $2$ at that stage to prevent that factor from popping up twice at later points.

Here are some tests:

RepeatedTiming[
 gaussMoment[(x^2 + x^4 + x^6), -(x - 1)^2, {x}]]

$$\left\{0.0023,\frac{223 \sqrt{\pi }}{8}\right\}$$

RepeatedTiming[
 gaussMoment[(x^2 + x y) , -(x - a1)^2 - (y - a2)^2 - (x - y)^2, {x, y}]]

$$\left\{0.0014,\frac{\pi \left(4 \text{a1}^2+6 \text{a1} \text{a2}+2 \text{a2}^2+3\right) e^{-\frac{1}{3} (\text{a1}-\text{a2})^2}}{6 \sqrt{3}}\right\}$$

This is about four orders of magnitude faster than doing the integrals using plain Integrate.

One other big advantage (in addition to its simplicity and speed) is that it can deal with non-polynomial prefactors. Here is an example (it takes longer to run, but the other methods cannot do it at all):

gaussMoment[
 Sin[2 Pi x] Cos[Pi x] , -(x - a1)^2 - (y - a2)^2 - (x - 
     y)^2, {x, y}]

$$\frac{i \pi e^{-\text{a1}^2-\frac{\text{a2}^2}{2}} \left(e^{\frac{1}{6} (2 \text{a1}+\text{a2}-i \pi )^2}-e^{\frac{1}{6} (2 \text{a1}+\text{a2}+i \pi )^2}+e^{\frac{1}{6} (2 \text{a1}+\text{a2}-3 i \pi )^2}-e^{\frac{1}{6} (2 \text{a1}+\text{a2}+3 i \pi )^2}\right)}{4 \sqrt{3}}$$

$\endgroup$
6
  • $\begingroup$ Looks OK, but isn't there a need for normalization of the original integral? $\endgroup$ Jun 15, 2012 at 8:28
  • $\begingroup$ I would think this is in fact the n1,nd derivative w.r.t. I c1,I c2.. cn of the characteristic function of a MultinormalDistribution; so mathematica can provide you with the Characteristic function and you can differentiate it. $\endgroup$
    – chris
    Jun 15, 2012 at 20:42
  • 1
    $\begingroup$ @chris Absolutely - and you can go even further and use the built-in command Moment for this, too. But Expectation is a bit more general in that it allows arbitrary prefactors, not just polynomials. $\endgroup$
    – Jens
    Jun 15, 2012 at 20:49
  • $\begingroup$ For the part Edit 3, I got it. Thanks, Jens! $\endgroup$
    – simpler
    Jun 18, 2012 at 2:16
  • 1
    $\begingroup$ I think it's worth mentioning that the slowness of Moment that would potentially affect this answer appears to have been fixed in Mathematica version 10.3. See also How to efficiently find moments of a multinormal distribution $\endgroup$
    – Jens
    Jan 30, 2016 at 1:34
20
$\begingroup$

If there is no pre-exponential factor then integral has a simple answer $$ \int \exp({\bf x}^TA{\bf x}+{\bf b}^T{\bf x}+c)d{\bf x} = \frac{\pi^{n/2}}{\sqrt{\det(-A)}}\exp\left(c - \frac{1}{4}{\bf b}^T A^{-1}{\bf b}\right) $$

it can be implemented in Mathematica very compactly

gaussianIntegralExp[expr_, vars_] := π^(Length[vars]/2)
    Exp[#1 - LinearSolve[#3, #2].#2/4]/Sqrt[Det[-#3]] & @@ 
       CoefficientArrays[expr, vars, "Symmetric" -> True] // Simplify

gaussianIntegralExp[-(x - 1)^2 - y^2 - x y, {x, y}] // AbsoluteTiming
{0.000685, (2 E^(1/3) π)/Sqrt[3]}
Integrate[Exp[-(x - 1)^2 - y^2 - x y], {x, -∞, ∞}, {y, -∞, ∞}] // AbsoluteTiming
{0.685393, (2 E^(1/3) π)/Sqrt[3]}

Great speedup!

Pre-exponential factor can be treated as moment of multinormal distribution. One can use Moment and MultinormalDistribution as in the Jens' answer but it is slow (especially for large moments). I describe this problem as a separate question and post my own answer in which I describe the moment function

gaussianIntegral[expr_, vars_List] := 
  Module[{coeff, exp, μ, a, b, c, rul, factor, σ, moment},
     {coeff, exp} = {expr/Exp[#], #} &[Plus @@ Cases[FactorList[expr], {E^x_, p_} :> p x]];
     {c, b, a} = CoefficientArrays[exp, vars, "Symmetric" -> True];
     μ = LinearSolve[a, b]/2;
     rul = Cases[#, x_ /; EvenQ@Total@First[x]] &@
     CoefficientRules[coeff /. Thread[vars -> vars - μ], vars];
     If[rul == {}, Return[0]];
     factor = π^(Length[vars]/2) Exp[c - μ.b/2]/Sqrt@Det[-a];
     If[MatchQ[rul, {{0 ..} -> _}], Return[rul[[1, 2]] factor]];
     σ = -Inverse[a]/2;
     moment[{0 ...}] = 1;
     moment[x_List] /; Total[x] == 2 := If[Length[#] == 1, σ[[#[[1]], #[[1]]]], 
        σ[[#[[1]], #[[2]]]]] &@Position[x, y_?Positive][[All, 1]];
     moment[x_List] := moment[x] = 
        Module[{i = LengthWhile[x, # == 0 &] + 1, x2 = x}, x2[[i]]--; 
         Sum[If[x2[[j]] > 0, x2[[j]] σ[[i, j]] moment@MapAt[# - 1 &, x2, j], 0], 
          {j, i, Length[x]}]];
     Total[#2 moment[#1] & @@@ rul] factor
]

Description:

  1. Extract the pre-factor and the exponent.
  2. Shift pre-factor by μ (the mean of the distribution)
  3. If there is only odd moments then return 0.
  4. If the pre-factor is a constant then return known result.
  5. Calculate moments (see here for the description of moment).

For the nice syntax highlighting add

SyntaxInformation[gaussianIntegral] = {"ArgumentsPattern" -> {_, _}, 
  "LocalVariables" -> {"Solve", 2}};

Examples:

gaussianIntegral[(x^2 + x^4 + x^6) Exp[-(x - 1)^2], {x}] // AbsoluteTiming
{0.003386, (223 Sqrt[π])/8}
Integrate[(x^2 + x^4 + x^6) Exp[-(x - 1)^2], {x, -∞, ∞}] // AbsoluteTiming
{0.389498, (223 Sqrt[π])/8}
gaussianIntegral[(x^2 +x y) Exp[-(x - a1)^2 - (y - a2)^2 - (x - y)^2], {x, y}] 
  // AbsoluteTiming // FullSimplify
{0.007893, ((3 + 2 (a1 + a2) (2 a1 + a2)) E^(-(1/3) (a1 - a2)^2) π)/(6 Sqrt[3])}
Integrate[(x^2 + x y) Exp[-(x - a1)^2 - (y - a2)^2 - (x - y)^2], {x, -∞, ∞}, {y, -∞, ∞}] 
  // AbsoluteTiming
{8.053473, ((3 + 2 (a1 + a2) (2 a1 + a2)) E^(-(1/3) (a1 - a2)^2) π)/(6 Sqrt[3])}
$\endgroup$
12
$\begingroup$

I'm a little bit late to this party, but I had written this function for another question that turned out not to need it, so I'll put this here.

My strategy is to straightforwardly calculate the integral via this formula

$$ \begin{align} \int f(\vec x) &\, \exp\left( - \frac 1 2 \sum_{i,j=1}^{n}A_{ij} x_i x_j \right) d^nx = \\ & \sqrt{(2\pi)^n\over \det A} \, \left. \exp\left({1\over 2}\sum_{i,j=1}^{n}(A^{-1})_{ij}{\partial \over \partial x_i}{\partial \over \partial x_j}\right)f(\vec{x})\right|_{\vec{x}=0} \end{align} $$

I had some help from @Jens and @DanielLichtblau in writing a function for the exponential differential operator. Here is the function for the multidimensional Gaussian integral $f e^g$

gaussianIntegral[f_, g_, vars_] := 
  Module[{expD, fdegree, nvars, amatrix, ainverse, jvector, rest, f2, 
    fnew},
   fdegree = 
    Exponent[# /. ((# -> \[FormalX] RandomReal[]) & /@ 
          Variables[#]), \[FormalX]] &@f;
   nvars = Length@vars;
   {rest, jvector, 
     amatrix} = ({1, 1, -2} CoefficientArrays[g, vars, 
       "Symmetric" -> True]);
   Which[
    Exponent[# /. ((# -> \[FormalX] RandomReal[]) & /@ 
           Variables[#]), \[FormalX]] &@g != 2,
    "Non Gaussian integral",
    Det[amatrix] <= 0,
    "A matrix is not positive definite, divergent integral",
    True,
    ainverse = Inverse[amatrix];
    fnew = f /. Inner[Rule, vars, vars + jvector.ainverse, List];
    expD = Module[{dop},
      dop = 
       Function[{f2}, 
        1/2 Sum[ainverse[[i, j]] D[
            f2, {vars[[i]], 1}, {vars[[j]], 1}], {i, nvars}, {j, 
           nvars}]];
      (1/Range[0, fdegree]!).NestList[dop, fnew, fdegree] /. 
       Thread[vars -> 0]
      ];
    Sqrt[(2 π)^nvars/Det[amatrix]]
      expD Exp[1/2 jvector.ainverse.jvector + rest]
    ]
   ];

This method is a fair bit slower than @ybeltukov's, (below I try a degree 6 polynomial and the timing is similar, but for a degree 30 polynomial my function takes ~10 times longer.

f1 = With[{n = 2},
  Expand[x^Range[0, n - 1].RandomInteger[10, n]*
     y^Range[0, n - 1].RandomInteger[10, n]]*
   z^Range[0, n - 1].RandomInteger[10, n]]
(* (6 + 42 x + 6 y + 42 x y) (4 + 4 z) *)

With[{f = f1, g = x - x^2 - 7 y - x y - y^2 - 7 z^2 + 2 z y},
 {gaussianIntegral[f, g, {x, y, z}] // AbsoluteTiming, 
  gaussianIntegralYbeltukov[f Exp[g], {x, y, z}] // AbsoluteTiming}]
(* {{0.001136, -((2603616 E^(398/17) π^(3/2))/(
   4913 Sqrt[17]))}, {0.004434, -((2603616 E^(398/17) π^(3/2))/(
   4913 Sqrt[17]))}} *)

You see they get the same result, however @ybeltukov's function has a couple points that mine addresses. Firstly, if you try to integrate a function with a covariance matrix that isn't positive definite, an error is thrown,

With[{f = x + y, g = 3*x - x^2 - 7*y - x*y - y^2 - 3*x*y}, {gaussianIntegral[f, g, {x, y}], 
   gaussianIntegral2[f*Exp[g], {x, y}], Integrate[f*Exp[g], {x, -Infinity, Infinity}, {y, -Infinity, Infinity}]}]
(* {"Covariance matrix is not positive definite, divergent integral",
    (2*I*Pi)/(3*Sqrt[3]*E^(71/6)), 
  Integrate[(-(1/2))*E^(49/4 + x*(17 + 3*x))*Sqrt[Pi]*(7 + 2*x)
        , {x, -Infinity, Infinity}]} *)

Secondly, I get an error for ybeltukov's function when approximate numbers are used,

With[{f = 2.0 x - y^2, g = x^2 + y^2},
 {gaussianIntegral[f, g, {x, y}] // AbsoluteTiming, 
  gaussianIntegral2[f Exp[g], {x, y}] // AbsoluteTiming}]

During evaluation of Power::infy: Infinite expression 1/0 encountered. >>

(* {{0.000606, 1.5708}, {0.031715, ComplexInfinity}} *)

Maybe someone will find this function useful

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.