2
$\begingroup$

TTest has an option VerifyTestAssumptions for "EqualVariance", does it automatically switch to Welch's t-test when variance of the two data sample differ a lot?

$\endgroup$

4 Answers 4

4
$\begingroup$

Mathematica can perform the Welch test when asked but it appears only when an F-test for equal variances is rejected. Here are two lists of data with very different variances:

data1 = {1, 2, 3, 4, 5};
data2 = 6 + 8 data1;

(* Welch t-test: P-value is the default output *)
LocationTest[{data1, data2}, 0, "T", VerifyTestAssumptions -> "EqualVariance"]
(* 0.00840227 *)

(* t-test using pooled variance: P-value is the default output *)
LocationTest[{data1, data2}, 0, "T", VerifyTestAssumptions -> None]
(* 0.0014713 *)

But now let y have a just a slightly different sample variance:

data1 = {1, 2, 3, 4, 5};
data2 = 6 + 1.1 data1;
LocationTest[{data1, data2}, 0, "T", VerifyTestAssumptions -> "EqualVariance"]
(* 0.000325851 *)

LocationTest[{data1, data2}, 0, "T", VerifyTestAssumptions -> None]
(* 0.000325851 *)

Here the Welch t-test is not performed as the test for equal variances is not rejected.

Editorial: It is sad that one can't seem to force a Welch t-test because that is the recommendation from most statisticians even when you might think the variances might be equal. In short, one should always perform the Welch t-test if one is going to do a t-test. See https://stats.stackexchange.com/questions/305/when-conducting-a-t-test-why-would-one-prefer-to-assume-or-test-for-equal-vari.

$\endgroup$
2
$\begingroup$

As far as I can tell, there is no Welch test in Mathematica (V12.3). So I wrote (an admittedly simple) one: you can download it from the Wolfram Function Repository. The implementation is based on the Wikipedia entry on the Welch test, and has been checked against the t-test function t.test() in R (version 4.1.2).

$\endgroup$
2
  • 1
    $\begingroup$ Consider adding it as a resource function, resources.wolframcloud.com/FunctionRepository. Any chance of adding the code here? Links rot... $\endgroup$
    – Michael E2
    Commented Apr 1, 2022 at 15:23
  • $\begingroup$ @MichaelE2 Thanks for the suggestion, I'll do this in the coming days. $\endgroup$ Commented Apr 3, 2022 at 16:42
2
$\begingroup$

The Welch T-Test with unequal variances is now available in Mathematica. (I am using version 14.0.) Try this:

In[1]:= data1 = {1, 2, 3, 4, 5};
        data2 = 6 + 1.1  data1;
        LocationTest[{data1, data2}, 0, "T", 
          VerifyTestAssumptions -> "EqualVariance"]
Out[1]= 0.000325851

In[2]:= LocationTest[{data1, data2}, 0, "T", 
          VerifyTestAssumptions -> "EqualVariance" -> False]
Out[2]= 0.000337532

In[3]:= TTest[{data1, data2}, 0, 
          VerifyTestAssumptions -> "EqualVariance" -> False]
Out[3]= 0.000337532

Adding the -> False after VerifyTestAssumptions -> "EqualVariance" forces it to assume the two samples have unequal variances and to use the Welch test.

$\endgroup$
2
  • $\begingroup$ +1 I went back to version 12 and this works on that version. So my answer below was wrong. $\endgroup$
    – JimB
    Commented Mar 1 at 4:26
  • $\begingroup$ The documentation on this was borderline cryptic. I saw that there was a VerifyTestAssumptions -> "EqualVariance" -> False option and had to test it to verify that it did what I posted. $\endgroup$ Commented Mar 4 at 14:05
1
$\begingroup$

Referring to the documentation, VerifyTestAssumptions will throw up an error message if the data doesn't meet the assumptions of the test. It will not automatically change the options of the function.

Source Documentation: https://reference.wolfram.com/language/ref/VerifyTestAssumptions.html

$\endgroup$
1
  • $\begingroup$ Unless I missed it, the warning is for a potential violation of the normality assumption but not for any violation of unequal variances. $\endgroup$
    – JimB
    Commented Apr 1, 2022 at 18:27

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.