2
$\begingroup$

For one of the models I'm building, the p-value in the ANOVA table obtained from the "ANOVATable" properties of a LinearModelFit object is not as expected from calculation using FRatioPValue.

I've included two test data sets to demonstrate the problem. Both are very similar - they contain one categorical "group" variable (can be "A" or "B") and one continuous covariate as the independent variables, and a continuous dependent variable.

data[1] = {{"A", 5.19, -2.408935392973501}, {"A", 44.4, -2.1426675035687315},
           {"A", 150.86, -1.2898826348881838}, {"A", 209.4, -0.7825160557860937}, 
           {"A", 289.74, -0.1272611725273312}, {"A", 32.98, -1.8860566476931633}, 
           {"A", 38.25, -1.8860566476931633}, {"A",  126.2, -1.6716203965612624}, 
           {"A", 154.58, -1.3635121036466347}, {"A", 304.12, -0.5466816599529624}, 
           {"A", 309.6, -0.28483264215154214}, {"B", 298.97, -1.0268721464003014}, 
           {"B", 227.51, -1.7594507517174003}, {"B", 293.04, -1.585026652029182}, 
           {"B", 345.1, -1.2676062401770316}, {"B", 318.72, -1.0695604052332999}, 
           {"A", 190.56, -0.978810700930062}, {"A", 308.3, -0.5199930570428494}, 
           {"A", 282.34, -0.32882715728491674}, {"A", 305.32, -0.15614457737683893}, 
           {"A", 364.9, 0.2787536009528289}, {"A", 398.25, 0.6541765418779605}, 
           {"A", 369.83, 0.33041377334919086}, {"A", 425., 0.9143431571194408}, 
           {"A", 419.5, 0.814913181275074}, {"A", 435.5, 0.9890046156985368}, 
           {"A", 428.6, 0.9503648543761231}, {"A", 417.9, 0.800717078282385}, 
           {"A", 425.12, 0.8859263398014311}, {"A", 407.85, 0.6683859166900001}, 
           {"A", 469.7, 1.3961993470957363}, {"A", 433.75, 0.8744818176994665}, 
           {"A", 460.39, 1.2041199826559248}};

data[2] = {{"A", 5.19, -1.5528419686577808}, {"A",  44.4, -1.3767507096020994}, 
           {"A",  150.86, -0.6478174818886375}, {"A", 209.4, -0.23210238398190938}, 
           {"A", 289.74, 0.28330122870354957}, {"A", 32.98, -1.1487416512809248}, 
           {"A", 38.25, -1.0757207139381184}, {"A", 126.2, -0.9546770212133425}, 
           {"A", 154.58, -0.6882461389442457}, {"A", 304.12, 0.13033376849500614}, 
           {"A", 309.6, 0.28103336724772754}, {"B", 298.97, 0.}, 
           {"B", 227.51, -0.7798919119599449}, {"B", 293.04, -0.41793663708829126}, 
           {"B", 345.1, -0.09528445472131904}, {"B", 318.72, -0.11975822410451964}, 
           {"A", 556.3, 2.4927603890268375}, {"A", 190.56, -0.36051351073141397}, 
           {"A", 308.3, 0.24303804868629444}, {"A", 282.34, 0.24054924828259971}, 
           {"A", 305.32, 0.3692158574101428}, {"A", 364.9, 0.829303772831025}, 
           {"A", 398.25, 1.100370545117563}, {"A", 369.83, 0.8762178405916422}, 
           {"A", 425., 1.3283796034387378}, {"A", 419.5, 1.3031960574204888}, 
           {"A", 435.5, 1.4608978427565478}, {"A", 428.6, 1.3979400086720377}, 
           {"A", 417.9, 1.3031960574204888}, {"A", 425.12, 1.3404441148401183}, 
           {"A", 407.85, 1.1760912590556813}, {"A", 469.7, 1.808885867359812}, 
           {"A", 433.75, 1.3856062735983121}};

I build a model for each of these data sets, incorporating an interaction term:

Do[lm[m] = 
  LinearModelFit[data[m], {group, cov, group*cov}, {group, cov}, 
   NominalVariables -> group], {m, 2}]

And now examine the output of the ANOVA tables:

lm[1]["ANOVATable"]

ANOVA table 1

lm[2]["ANOVATable"]

ANOVA table 2

The p-values provided for model 1 are completely as I would expect from using FRatioPValue with 1 and 29 degrees of freedom:

FRatioPValue[163.12140064423488`, 1, 29]

OneSidedPValue -> 1.96334*10^-13

FRatioPValue[1018.1169124577177`, 1, 29]

OneSidedPValue -> 3.86847*10^-24

FRatioPValue[1.3087690848510989`, 1, 29]

OneSidedPValue -> 0.261973

However, the p-value for "cov group" for model 2 is not what I would expect:

FRatioPValue[82.09313575024596`, 1, 29]

OneSidedPValue -> 5.89074*10^-10

FRatioPValue[1265.3783550427606`, 1, 29]

OneSidedPValue -> 1.78435*10^-25

FRatioPValue[0.14515111086886828`, 1, 29]

OneSidedPValue -> 0.29401

And in fact, the p-value returned in the ANOVA table appears to actually be equal to (1 - FRatioPValue):

1 - FRatioPValue[0.14515111086886828`, 1, 29][[2]]

0.70599

I have no idea why that would be - can anyone shed any light on it?

$\endgroup$
  • 4
    $\begingroup$ From the Details section on FRatioPValue: The one-sided p-value is CDF[FRatioDistribution[n,m],x] if x is less than the median of the F-ratio distribution with n and m degrees of freedom, and 1-CDF[FRatioDistribution[n,m],x] otherwise. $\endgroup$ – JimB Jan 22 at 20:06
  • 1
    $\begingroup$ Well, that explains it. Thanks, JimB. I don't understand the reasoning behind it, so will have to do some more reading! $\endgroup$ – Phil Jan 22 at 22:25
  • $\begingroup$ PS - thanks to MarcoB for the judicial editing. $\endgroup$ – Phil Jan 22 at 22:29
  • 2
    $\begingroup$ My guess as to the reasoning (and not the reasoning that I would use) is that some folks think that a "tail region" has to be smaller than 0.5 otherwise it wouldn't be the "tail". Fortunately the ANOVA table uses the more standard definition of a P-value. $\endgroup$ – JimB Jan 22 at 22:35
  • 1
    $\begingroup$ @Phil my pleasure! Yours was an excellent first question. I look forward to your future contributions to the site! $\endgroup$ – MarcoB Jan 23 at 3:21

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Browse other questions tagged or ask your own question.