Interpreting P-Value on ParameterTable

Question

When using LinearModelFit[] we can generate a table of parameter values and statistics for the corresponding model using the property "ParameterTable". Looking in the documentation I can't find an explicit statement of the hypothesis test being performed for the p-values listed in this table. I would like to make sure I am interpreting these things properly. So for example if I produce the table $$\begin{array}{l|llll} \text{} & \text{Estimate} & \text{Standard Error} & \text{t-Statistic} & \text{P-Value} \\ \hline 1 & 0.0116261 & 0.00199362 & 5.83162 & \text{3.092$\grave{ }$*${10}^{\wedge}$-8} \\ \text{x$\$$2374}(1) & -0.00573076 & 0.00207492 & -2.76191 & 0.0064413 \\ \text{x$\$$2374}(2) & -0.00614132 & 0.00209498 & -2.93144 & 0.0038851 \\ \text{x$\$$2374}(3) & -0.000374603 & 0.00240746 & -0.155601 & 0.87655 \\ \text{x$\$$2374}(4) & -0.00289621 & 0.00256897 & -1.12738 & 0.261323 \\ \end{array}$$ how do I interpret these p-values? I would think at a 95% confidence level the first three parameters are statistically significant but the last two are not.

Answer 1

Here's how to interpret statistical significance, here illustrated for a $\chi^2$ distribution:

Such a distribution tells you the expected probability of finding a value of $x$ as the sum of squares of values chosen from $n$ univariate Gaussian distributions (where we call $n$ the degrees of freedom). Of course, the value of $x$ can never be negative. The below Manipulate shows this distribution for different values of the settable degrees of freedom (denoted $df$).

The $x$ corresponding to a particular $p$-value (the "significance") is the value of $x$ for which a percentage of $p$ cases lie above (>$x$) in the distribution. If you obtain an experimental value $x$ above that criterion you can reject the null hypothesis, i.e., that the value could arise if the value were zero "by chance". We write that value for instance $\chi^2_{0.01(3)}$ which means the criterion value of $x$ for which 0.01 of the cases lie above that $x$ for $3$ degrees of freedom. Adjust the $p$ value to $p=0.05$ in the below Maniplate and see that the criterion value must drop.

Manipulate[Module[{xsol, x},

  xsol = x /. 
     NSolve[1 - CDF[ChiSquareDistribution[df], x] == Significance, 
       x][[1, 1]] // Quiet;

  Plot[
   PDF[ChiSquareDistribution[df], x], {x, 0, 20}, 
   PlotRange -> {0, .4},
   ColorFunction -> Function[x, If [x > xsol, Pink, Lighter[Green]]],
   ColorFunctionScaling -> False,
   Filling -> Axis,
   TicksStyle -> Italic,
   Ticks -> {Range[0, 20, 
       1] \[Union] {{xsol, 
        Column[{" ", 
          Style[Subscript[\[Chi]^2, 
            ToString[Significance] <> "(" <> ToString[df] <> ")"], 
           Red, 12]}, Center], {-.05, .0}, Red}},
     Range[0, .4, .1]},
   AxesLabel -> {Text[Style["x", 14 , Italic]], 
     Text[Style["probability", 14 , Italic]]},
    PlotLabel -> TextCell[
     Row[{Style[
        Subscript[\[Chi]^2, 
         ToString[Significance] <> "(" <> ToString[df] <> ")"]] , 
       " = ", Style[ToString[xsol], Italic, 12]}]
     ]
   ]],

 {{df, 3, Text[Style["df", Italic, 14]]}, 1, 30, 1, 
  AppearanceElements -> All}, {Significance, {0.01, 0.05}}]