How would you visualize a multivariate Gaussian in high dimension (>3D)?

Question

Obviously, I can't do:

Plot3D[PDF[
  MultinormalDistribution[{0, 0, 
    0}, {{1, 0, 0}, {0, 1, 0} {0, 0, 1}}], {x, y}], {x, -20, 
  20}, {y, -20, 20}]

How would you suggest me to effectively plot this multivariate probability density function?

Answer 1

I am not sure what are the answers you are looking for. One way is to utilize color as a fourth dimension; another is to make multiple 3D plots over some grid for the variables non-visualized in those 3D plots.

Here is an example:

Clear[myPDF]
myPDF[x_, y_, z_] := 
  Evaluate[PDF[
    MultinormalDistribution[{-1, 1, 1}, {{1, 1/2, 0}, {1/2, 1, 0}, {0, 0, 1}}], {x, y, z}]];
myPDF[x, y, z]

(* E^(1/2 (-(1 + x) ((4 (1 + x))/3 - 
      2/3 (-1 + y)) - (-(2/3) (1 + x) + 4/3 (-1 + y)) (-1 + y) - (-1 +
      z)^2))/(Sqrt[6] \[Pi]^(3/2)) *)

Multicolumn[
 Table[Plot3D[myPDF[x, y, z], {x, -3, 3}, {y, -6, 6}, 
   MeshFunctions -> {#3 &}, MeshShading -> {None, Red, None, Yellow}, 
   PlotRange -> {All, All, {0, 0.08}}, PlotPoints -> 25, 
   PlotLabel -> Row[{"z=", z}]], {z, -6, 6, 1}], 3]

Answer 2

Full M-dimensional probability distributions are hard to display. However, the conditional distributions at the planes through the origin, showing the densities as cross-cut through the full distribution, can be plotted as contour plots. The conditional distribution of a multinomial Gaussian distribution is also a Gaussian distribution, and therefore the contours are ellipses.

Here are the conditional distributions for some 4-dimensional distributions (w0,w1,w2,w3).

I have used the ConditionalMultinormalDistribution function from Chris.

Answer 3

A Guassian distribution is a subset of elliptical distributions, and an ellipse is defined by its centroid and each of the higher dimensional versions of the semi-major and semi-minor axes. So you can gain some understanding by visualizing those things in the higher dimensional space.

A way to do that is to use Parallel Coordinates plots, https://en.wikipedia.org/wiki/Parallel_coordinates .

Here's a simplistic implementation. Start with a 5D Multivariate Normal with $\mu$ and $\Sigma$. We've preprocessed $\Sigma$ to get the directions of the axes. First plot the mean in 5D space. Where the crooked line crosses the vertical red line is the value for the Mean in that dimension. More dimensions are just more red vertical lines.

μ = {1, -2, 3, 4, -2}
minμ = Min@μ - 1;
maxμ = Max@μ + 1;
num = Length@\[Mu];

ListLinePlot[\[Mu],
 PlotRange -> {{0, num + 1}, {min\[Mu], max\[Mu]}},
 Epilog -> {
   {Red, Thick, Table[InfiniteLine[{i, 0}, {0, 1}], {i, num}]},
   Table[Text["Dim" <> ToString[i], {i, max\[Mu] - .5},Background -> White], {i, num}]
   }
 ]

For the axes, there will be a max of 5 (could simplify, for example if the ellipsoid is a 5D hypersphere)

axis1 = {6, 3, 3, 1, -2};
axis2 = {1, 1, 2, 1, -1};
axis3 = {.2, .2, 3, -3, 2};
axis4 = {.1, .1, .01, .3, .2};
axis5 = {0, 0, 0, 2, 0};
axes = {axis1, axis2, axis3, axis4, axis5};

minAxesVal = Min@axes - 1;
maxAxesVal = Max@axes + 1;

ListLinePlot[axes,
 PlotRange -> {{0, num + 1}, {minAxesVal, maxAxesVal}},
 Epilog -> {
   {Red, Thick, Table[InfiniteLine[{i, 0}, {0, 1}], {i, num}]},
   Table[Text["dim" <> ToString[i], {i, maxAxesVal - .5}, Background -> White], {i, num}]
   }
 ]

From the plot you can adduce which axes of the ellipsoid are big and which are small, and relations between dimensions. You could pick one axis and Monte Carlo it to get a warm and fuzzy. Include the mean value.

nd = NormalDistribution[];
axisDistro = Table[\[Mu] + axis2 RandomVariate[nd], {50}];
minAD = Min@axisDistro;
maxAD = Max@axisDistro;

ListLinePlot[axisDistro,
 PlotRange -> {{0, num + 1}, {minAD, maxAD}},
 Epilog -> {
   {Red, Thick, Table[InfiniteLine[{i, 0}, {0, 1}], {i, num}]},
   Table[Text["dim" <> ToString[i], {i, maxAD - .5}, Background -> White], {i, num}]
   }
 ]

Lots of options from here!