10
$\begingroup$
data1 = {19.4, 21.2, 17.1, 21.1, 19.1, 18.2, 19.6};
data2 = {19.6, 21.1, 18.2, 21.8, 19.5, 18.3, 19.1};
PairedTTest[{data1, data2}]

How do I extract the 95% confidence intervals for the difference between the means from the PairedTTest?

$\endgroup$
  • $\begingroup$ 2 StandardDeviation[data2 - data1]? $\endgroup$ – Feyre Sep 19 '16 at 10:05
9
$\begingroup$

Yes it is surprising that such a common task is not available in the latest Statistics framework. PairedTTest[{data1,data2},"ConfidenceInterval"] really ought to return what you're after (i.e. the object from PairedTTest[{data1,data2},0,"HypothesisTestData"] ought to contain "ConfidenceInterval" as a property).

Confidence Intervals existed in V 5.2's package "Statistics`ConfidenceIntervals`" which was upgraded to the package HypothesisTesting`. This, in turn, was superseded by the new (kernel) framework of which PairedTTest is a member. To generate such a confidence interval then, go back to HypothesisTesting`.

Needs["HypothesisTesting`"]
data1 = {19.4, 21.2, 17.1, 21.1, 19.1, 18.2, 19.6};
data2 = {19.6, 21.1, 18.2, 21.8, 19.5, 18.3, 19.1};
MeanCI[data1 - data2]

(* {-0.757051, 0.214194} *)

MeanCI's default assumptions match that for a PairedTTest confidence interval. To explicitly apply a "T-test confidence interval" the package's StudentTCI can instead be used to define "My" expected syntax:

MyPairedTTest[{data1_, data2_}, 0, "ConfidenceInterval"] := Let[
diff = data1 - data2,
n = Length@diff,
m = Mean@diff,
se = StandardDeviation@diff/Sqrt@n,

StudentTCI[m, se, n - 1]]

(* {-0.757051, 0.214194} *)

N.b, the package's MeanDifferenceCI will not do here since one of its assumptions is that both samples are independent (unlike the paired or matched case where individual effects are accounted for meaning that smaller differences can define significance; or equivalently, matched-pairs' confidence intervals are tighter)

 MeanDifferenceCI[data1, data2]

 (* {-1.91857, 1.37571} *)
$\endgroup$
  • $\begingroup$ I didn't realise this package existed. It is very odd that WRI did not encapsulate this package in the HypothesisTestData object. Some CI require a bit of extra calculation and/or contain several methods (e.g. rank correlation statistics) but that should not be an issue for Mma. $\endgroup$ – Edmund Sep 20 '16 at 9:35
  • $\begingroup$ Yes, there are several puzzling things indicated with this omission. The documentation suggests it was the last item on a to-do list that never got completed. That such a query arises after such a long time (the framework was introduced in 2010) confirms something I've long suspected - there is a significant mismatch between the level of built-in statistical functionality and actual, or at least potential usage. I thought the 2010 framework really was an advance ... $\endgroup$ – Ronald Monson Sep 20 '16 at 10:34
  • $\begingroup$ ... in introducing a higher-level abstraction, systematization and subsequent automation but one that doesn't seem to have caught on - I suspect because with such automation, adjuncts to black-box invocations become even more important something that I think will become especially relevant to the newer ML framework. $\endgroup$ – Ronald Monson Sep 20 '16 at 10:35
  • $\begingroup$ Use data1={100}; data2={110};. I cannot reproduce your results with the simple data in Mathematica 11. $\endgroup$ – Léo Léopold Hertz 준영 Sep 21 '16 at 10:28
  • $\begingroup$ @Masi Tests with sample sizes of 1 aren't really going to reveal too much so yes, the wasn't factored into the answer. All the reason why it would be nice to have a built-in version with all the i's crossed and t's dotted. I'm sure the answer can be modified until such time. $\endgroup$ – Ronald Monson Oct 18 '16 at 18:25
12
$\begingroup$

I don't think you can. While I like the options with LinearModelFit and NonlinearModelFit, whoever wrote PairedTTest and TTest (and a few others) seemed to think that all one needs are P-values.

Here's how to get the confidence intervals in a brute-force way:

data1 = {19.4, 21.2, 17.1, 21.1, 19.1, 18.2, 19.6};
data2 = {19.6, 21.1, 18.2, 21.8, 19.5, 18.3, 19.1};

confidenceLevel = 0.95;
diff = data1 - data2;
n = Length[diff];
meanDiff = Mean[diff]
(* -0.271429 *)
se = StandardDeviation[diff]/Sqrt[n];
t = InverseCDF[StudentTDistribution[n - 1], 1 - (1 - confidenceLevel)/2];
{lower, upper} = meanDiff + {-1, 1} t se
(* {-0.757051, 0.214194} *)
$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.