General F statistic and p value in LinearModelFit

Question

Many statistical packages provide a "F statistic" and the corresponding p value in their output of a multiple linear regression.

I don't find that in the parameters of the Mathematica function LinearModelFit. All I can find are the t-statistics for the individual coefficients but no general statistic related to the regression as a whole.

I also don't find how to test interactions between predictor variables.

Where can I find the general F statistic and see the interactions between my predictor variables?

Answer 1

Assume that you have assigned the variable "model" as your LinearModelFit result. Then you can get the F statistic and its p-value with:

 model[{"ANOVATableFStatistics", "ANOVATablePValues"}]

For interactions, you can include them when you build your data. For example, assume that you have two independent variables. Build your data list as:

data = Table[{x1[[i]], x2[[i]], x1[[i]] x2[[i]], y[[i]]}, {i, 1, Length[y]}]

Now, run your regression:

model = LinearModelFit[data, {x1, x2, x3}, {x1, x2, x3}]

and examine the results:

model["ParameterTable"]

You can calculate the overall F statistic for the model by averaging the individual F statistics:

Mean[model["ANOVATableFStatistics"]]

To get the p value is a bit of a hassle, but this will do it:

1 - CDF[FRatioDistribution[Total[model["ANOVATableDegreesOfFreedom"][[1;;Length[model["ANOVATableFStatistics"]]]]], model["ANOVATableDegreesOfFreedom"][[-2]]], Mean[model["ANOVATableFStatistics"]]]

and here is a function to give both the F statistic and its p value as a list {f,p}:

fStat[m_FittedModel] := {Mean[m["ANOVATableFStatistics"]],
   1 - CDF[
     FRatioDistribution[
      Total[m["ANOVATableDegreesOfFreedom"][[1 ;; 
          Length[m["ANOVATableFStatistics"]]]]], 
      m["ANOVATableDegreesOfFreedom"][[-2]]], 
     Mean[m["ANOVATableFStatistics"]]]};,

There may be easier ways to get these values, and I'd love to see them, but this is what I was able to figure out. I don't understand why MMA doesn't have these as properties.

Answer 2

Using the data from the first example in LinearModelFit one can construct the overall model F-statistic and the associated P-value.

(* Data and fit *)
data = Map[{#[[1]], #[[2]], #[[3]], 1.2 + (3.7 + RandomReal[{-1, 1}]) #[[1]] - 2 #[[2]] + 
      23.4 #[[3]]} &, RandomReal[10, {100, 3}]];
lm = LinearModelFit[data, {x, y, z}, {x, y, z}];

df = lm["ANOVATableDegreesOfFreedom"];
df = {Total[df[[1 ;; -3]]], df[[-2]]}; (* Model df and error df *)
ss = lm["ANOVATableSumsOfSquares"] ;
ss = {Total[ss[[1 ;; -3]]], ss[[-2]]}; (* Model sum of squares and error sum of squares *)

(* Construct overall model F-ratio *)
overallFratio = (ss[[1]]/df[[1]])/(ss[[2]]/df[[2]])

(* Associated P-value *)
1 - CDF[FRatioDistribution[df[[1]], df[[2]]], overallFratio]

The ANOVA table with all of the individual F-tests is found with

lm["ANOVATable"]

\begin{array}{l|lllll} \text{} & \text{DF} & \text{SS} & \text{MS} & \text{F-Statistic} & \text{P-Value} \\ \hline x & 1 & 10010.1 & 10010.1 & 886.643 & \text{2.789559773927423$\grave{ }$*${}^{\wedge}$-50} \\ y & 1 & 885.605 & 885.605 & 78.4423 & \text{4.2431663209054734$\grave{ }$*${}^{\wedge}$-14} \\ z & 1 & 424541. & 424541. & 37603.7 & \text{2.4832751695208124$\grave{ }$*${}^{\wedge}$-126} \\ \text{Error} & 96 & 1083.83 & 11.2899 & \text{} & \text{} \\ \text{Total} & 99 & 436521. & \text{} & \text{} & \text{} \\ \end{array}