2
$\begingroup$

When using LinearModelFit[] we can generate a table of parameter values and statistics for the corresponding model using the property "ParameterTable". Looking in the documentation I can't find an explicit statement of the hypothesis test being performed for the p-values listed in this table. I would like to make sure I am interpreting these things properly. So for example if I produce the table $$\begin{array}{l|llll} \text{} & \text{Estimate} & \text{Standard Error} & \text{t-Statistic} & \text{P-Value} \\ \hline 1 & 0.0116261 & 0.00199362 & 5.83162 & \text{3.092$\grave{ }$*${10}^{\wedge}$-8} \\ \text{x$\$$2374}(1) & -0.00573076 & 0.00207492 & -2.76191 & 0.0064413 \\ \text{x$\$$2374}(2) & -0.00614132 & 0.00209498 & -2.93144 & 0.0038851 \\ \text{x$\$$2374}(3) & -0.000374603 & 0.00240746 & -0.155601 & 0.87655 \\ \text{x$\$$2374}(4) & -0.00289621 & 0.00256897 & -1.12738 & 0.261323 \\ \end{array}$$ how do I interpret these p-values? I would think at a 95% confidence level the first three parameters are statistically significant but the last two are not.

$\endgroup$
4
$\begingroup$

Here's how to interpret statistical significance, here illustrated for a $\chi^2$ distribution:

Such a distribution tells you the expected probability of finding a value of $x$ as the sum of squares of values chosen from $n$ univariate Gaussian distributions (where we call $n$ the degrees of freedom). Of course, the value of $x$ can never be negative. The below Manipulate shows this distribution for different values of the settable degrees of freedom (denoted $df$).

The $x$ corresponding to a particular $p$-value (the "significance") is the value of $x$ for which a percentage of $p$ cases lie above (>$x$) in the distribution. If you obtain an experimental value $x$ above that criterion you can reject the null hypothesis, i.e., that the value could arise if the value were zero "by chance". We write that value for instance $\chi^2_{0.01(3)}$ which means the criterion value of $x$ for which 0.01 of the cases lie above that $x$ for $3$ degrees of freedom. Adjust the $p$ value to $p=0.05$ in the below Maniplate and see that the criterion value must drop.

Manipulate[Module[{xsol, x},

  xsol = x /. 
     NSolve[1 - CDF[ChiSquareDistribution[df], x] == Significance, 
       x][[1, 1]] // Quiet;

  Plot[
   PDF[ChiSquareDistribution[df], x], {x, 0, 20}, 
   PlotRange -> {0, .4},
   ColorFunction -> Function[x, If [x > xsol, Pink, Lighter[Green]]],
   ColorFunctionScaling -> False,
   Filling -> Axis,
   TicksStyle -> Italic,
   Ticks -> {Range[0, 20, 
       1] \[Union] {{xsol, 
        Column[{" ", 
          Style[Subscript[\[Chi]^2, 
            ToString[Significance] <> "(" <> ToString[df] <> ")"], 
           Red, 12]}, Center], {-.05, .0}, Red}},
     Range[0, .4, .1]},
   AxesLabel -> {Text[Style["x", 14 , Italic]], 
     Text[Style["probability", 14 , Italic]]},
    PlotLabel -> TextCell[
     Row[{Style[
        Subscript[\[Chi]^2, 
         ToString[Significance] <> "(" <> ToString[df] <> ")"]] , 
       " = ", Style[ToString[xsol], Italic, 12]}]
     ]
   ]],

 {{df, 3, Text[Style["df", Italic, 14]]}, 1, 30, 1, 
  AppearanceElements -> All}, {Significance, {0.01, 0.05}}]

enter image description here

$\endgroup$
  • $\begingroup$ Thanks for taking the time to write this code, but this is not quite what I was asking. The p-value in the parameter table is used for some type of hypothesis test. I assume the null hypothesis is the parameter is equal to zero. Based on the p-value we can accept or reject the null hypothesis. @ significance level .05 we would reject the null hypothesis for the first 3 parameters which says they add a statistically significant contribution to the model. I want to know if this interpretation is correct since the mathematica documentation never says what the null hypothesis is. $\endgroup$ – Wintermute Apr 30 '15 at 0:21
  • 2
    $\begingroup$ @wintermute: The documentation does specifiy this. Read the "How to Get Results for Fitted Models" and "Statistical Model Analysis" tutorials, e.g., linked in the documentation for the fit functions... $\endgroup$ – ciao Apr 30 '15 at 0:39

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.