3
$\begingroup$

Suppose that lmModel is a linear model which is fit using LinearModelFit. How come lmModel["EstimatedVariance"] is not equal to Variance[lmModel["FitResiduals"]]? How is lmModel["EstimatedVariance"] actually calculated?

$\endgroup$
7
$\begingroup$

This documentation page spells out how EstimatedVariance is computed. It is the "squared sum of FitResiduals divided by the degrees of freedom $n-p$. A linear regression will also have an intercept term, in addition to terms corresponding to each of the predictor columns.

For example,

headings = ExampleData[{"Statistics", "FisherIris"}, "ColumnHeadings"];
data = ExampleData[{"Statistics", "FisherIris"}];
ds = Dataset[Map[AssociationThread[headings, #] &, data]]

ds

Subsetting the data so that we can see a nice linear trend:

d = ds[Select[#["PetalLength"] > 2 &]];
ListPlot[d[All, {"SepalLength", "PetalLength"}]]

lp

Fitting the model (converting the Dataset data into normal values):

lmModel = LinearModelFit[
  d[All, {"SepalLength", "PetalLength"}] // Normal // Values
 , x, x];

we can extract the Estimated Variance:

lmModel["EstimatedVariance"]
(* 0.21594 *)

The Fitted Residuals can be extracted:

fr = lmModel["FitResiduals"];

and compute its variance with the help of Variance:

Variance[fr]
(* 0.213759 *)

Explicitly computing the unbiased variance (same as Variance):

Total[(fr - Mean[fr])^2]/(Length[fr] - 1)
(* 0.213759 *)

Recalling that we have to subtract out two degrees of freedom $(p=2)$ due to the intercept and the slope term:

Total[(fr - Mean[fr])^2]/(Length[fr] - 2)
(* 0.21594 *)

which explains the origins of these Properties.

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.