Does Mathematica have built-in support for multivariate regression?

Question

I know that LinearModelFit can generate a model of a single response variable as a function of a set in input variables. Does Mathematica provide support for matrix regression, i.e. multiple inputs, multiple response variables?

Answer 1

While the terms multivariate and multiple are sometimes confused, in regression, those who make a distinction tend to call many outputs multivariate and many inputs multiple. LinearModelFit does univariate multiple regression, but not multivariate regression.

Here is the example from the U.Va. site I linked in a comment:

amiData = Import["http://static.lib.virginia.edu/statlab/materials/data/ami_data.DAT"]

Each row consists of values for the variables

{TOT, AMI, GEN, AMT, PR, DIAP, QRS}

The problem is to analyze the linear relationship between the response {TOT, AMI} and predictors {GEN, AMT, PR, DIAP, QRS}. The underlying model consists of two linear equations, which are the same whether doing univariate and multivariate analysis. Thus they can be obtained from separate calls to LinearModelFit:

(* individual univariate regressions *)
vars = {GEN, AMT, PR, DIAP, QRS};
totLM = LinearModelFit[amiData[[All, {3, 4, 5, 6, 7, 1}]], vars, vars]
amiLM = LinearModelFit[amiData[[All, {3, 4, 5, 6, 7, 2}]], vars, vars]

The coefficients of the linear model may be obtained from each of these linear models or in a multivariate way via LeastSquares[]:

{totLM["BestFitParameters"], amiLM["BestFitParameters"]} // Transpose
LeastSquares[  (* multivariate approach to the regression equation *)
 PadLeft[N@amiData[[All, {3, 4, 5, 6, 7}]], {Automatic, 6}, 1.],
 amiData[[All, {1, 2}]]
 ]
(*
{{-2879.48, -2728.71}, {675.651, 763.03}, {0.284851, 
  0.306373}, {10.2721, 8.8962}, {7.25117, 7.20556}, {7.59824, 4.98705}}

{{-2879.48, -2728.71}, {675.651, 763.03}, {0.284851, 
  0.306373}, {10.2721, 8.8962}, {7.25117, 7.20556}, {7.59824, 4.98705}}
*)

So far, so good. If that's all you want, then you can use separate calls to LinearModelFit to do independent univariate multiple regressions.

However, if you want to use Multivariate ANOVA (MANOVA) to compare variables across models, then I think there is no support for that. The ANOVA package does not seem to handle MANOVA nor does LinearModelFit. However, this Wikipedia article indicates that Mathematica can handle MANOVA. Perhaps they mean if you write the code yourself. This Community discussion is about the OP's question, but I don't think it really addresses MANOVA.

You can use RLink to call R. The car package contains the Manova command. I had to follow the protocol here to get Mathematica and R to play nicely together on my Mac. I was able to go through the U.Va. example, but the R output is translated back into Mathematica with lots of inconvenient RObject[..] wrappers. To parse it, I had to look simultaneously at R, which knows how to interpret the data structure and displays it in tables with meaningful headers. Somewhat of a headache. If I really needed to do this a lot, I'd write my own Mathematica functions.