Consider two Normally distributed random variables:

x1 = RandomVariate[NormalDistribution[], {1000}];
x2 = RandomVariate[NormalDistribution[], {1000}];

I want to test $H_0 : \rho = 0$ against $H_1 : \rho \neq 0$.

If I use

CorrelationTest[Transpose[{x1, x2}], 0, {"TestDataTable", All}]

I get, for instance,

\begin{array}{l|ll} \text{} & \text{Statistic} & \text{P-Value} \\ \hline \text{Pearson Correlation} & 0.00505234 & 0.873251 \\ \text{Spearman Rank} & 0.00983509 & 0.756138 \\ \end{array}

However, if I use

PearsonCorrelationTest[x1, x2, "TestDataTable"]

I get

\begin{array}{l|ll} \text{} & \text{Statistic} & \text{P-Value} \\ \hline \text{Pearson Correlation} & 0.00505234 & 0.87322 \\ \end{array}

and, finally, if I use

SpearmanRankTest[x1, x2, "TestDataTable"]

I get

\begin{array}{l|ll} \text{} & \text{Statistic} & \text{P-Value} \\ \hline \text{Spearman Rank} & 0.00983509 & 0.756081 \\ \end{array}

Now compare the P-Values of the tests... they are slightly different, although they are supposed to be exactly the same test.

Can anyone confirm this? If yes, could it be a small bug in Mathematica?


If you check the properties and relations sections of the examples for both CorrelationTest and SpearmanRankTest you will see that these are not identical tests.

In the first case we are checking whether the correlation coefficient is a particular value and the p-value is computed using a Fisher Z transformation. This transformation allows us to test against other null values (not just that the Correlation or SpearmanRho is zero).

In the case of SpearmanRankTest it is purely a test of independence (i.e. you cannot test against other null hypotheses) and it is assumed that ρ Sqrt[(n-2)/(1-ρ^2)] follows a StudentTDistribution.

This is not a bug.


Here is some further evidence that these are testing different hypotheses.

v1 = RandomVariate[NormalDistribution[], 100];
v2 = RandomVariate[NormalDistribution[], 100];

SpearmanRankTest[v1, v2, "TestConclusion"]

(* "The null hypothesis that the populations are independent is rejected at the 5 
  percent level based on the Spearman Rank test."*)

CorrelationTest[Transpose[{v1, v2}], 0, {"TestConclusion", "SpearmanRank"}]

(*"The null hypothesis that the population rank correlation coefficient is equal to 0. 
   is rejected at the 5 percent level based on the Spearman Rank test."*)
  • $\begingroup$ Could I assume the same for the Pearson correlation test? $\endgroup$ – Rod May 22 '13 at 18:01
  • $\begingroup$ Yes, it is the same for PearsonCorrelationTest. Again, you can check the properties and relations section to confirm. $\endgroup$ – Andy Ross May 22 '13 at 18:02
  • $\begingroup$ It seems to me they are testing the same hypotheses but are using different approximations to the sampling distribution of the test statistic. $\endgroup$ – whuber May 22 '13 at 19:16
  • $\begingroup$ @whuber in this particular case that is essentially true. However, that is only because we happen to have set the null value to zero for CorrelationTest. $\endgroup$ – Andy Ross May 22 '13 at 19:34

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.