data1 = {19.4, 21.2, 17.1, 21.1, 19.1, 18.2, 19.6};
data2 = {19.6, 21.1, 18.2, 21.8, 19.5, 18.3, 19.1};
PairedTTest[{data1, data2}]
How do I extract the 95% confidence intervals for the difference between the means from the PairedTTest?
$\begingroup$
$\endgroup$
1
2 Answers
$\begingroup$
$\endgroup$
5
Yes it is surprising that such a common task is not available in the latest Statistics framework. PairedTTest[{data1,data2},"ConfidenceInterval"] really ought to return what you're after (i.e. the object from PairedTTest[{data1,data2},0,"HypothesisTestData"] ought to contain "ConfidenceInterval" as a property).
Confidence Intervals existed in V 5.2's package "Statistics`ConfidenceIntervals`" which was upgraded to the package HypothesisTesting`. This, in turn, was superseded by the new (kernel) framework of which PairedTTest is a member. To generate such a confidence interval then, go back to HypothesisTesting`.
Needs["HypothesisTesting`"]
data1 = {19.4, 21.2, 17.1, 21.1, 19.1, 18.2, 19.6};
data2 = {19.6, 21.1, 18.2, 21.8, 19.5, 18.3, 19.1};
MeanCI[data1 - data2]
(* {-0.757051, 0.214194} *)
MeanCI's default assumptions match that for a PairedTTest confidence interval. To explicitly apply a "T-test confidence interval" the package's StudentTCI can instead be used to define "My" expected syntax:
MyPairedTTest[{data1_, data2_}, 0, "ConfidenceInterval"] := Let[
diff = data1 - data2,
n = Length@diff,
m = Mean@diff,
se = StandardDeviation@diff/Sqrt@n,
StudentTCI[m, se, n - 1]]
(* {-0.757051, 0.214194} *)
N.b, the package's MeanDifferenceCI will not do here since one of its assumptions is that both samples are independent (unlike the paired or matched case where individual effects are accounted for meaning that smaller differences can define significance; or equivalently, matched-pairs' confidence intervals are tighter)
MeanDifferenceCI[data1, data2]
(* {-1.91857, 1.37571} *)
-
$\begingroup$ I didn't realise this package existed. It is very odd that WRI did not encapsulate this package in the
HypothesisTestDataobject. Some CI require a bit of extra calculation and/or contain several methods (e.g. rank correlation statistics) but that should not be an issue for Mma. $\endgroup$– EdmundCommented Sep 20, 2016 at 9:35 -
$\begingroup$ Yes, there are several puzzling things indicated with this omission. The documentation suggests it was the last item on a to-do list that never got completed. That such a query arises after such a long time (the framework was introduced in 2010) confirms something I've long suspected - there is a significant mismatch between the level of built-in statistical functionality and actual, or at least potential usage. I thought the 2010 framework really was an advance ... $\endgroup$ Commented Sep 20, 2016 at 10:34
-
$\begingroup$ ... in introducing a higher-level abstraction, systematization and subsequent automation but one that doesn't seem to have caught on - I suspect because with such automation, adjuncts to black-box invocations become even more important something that I think will become especially relevant to the newer ML framework. $\endgroup$ Commented Sep 20, 2016 at 10:35
-
$\begingroup$ Use
data1={100}; data2={110};. I cannot reproduce your results with the simple data in Mathematica 11. $\endgroup$ Commented Sep 21, 2016 at 10:28 -
$\begingroup$ @Masi Tests with sample sizes of 1 aren't really going to reveal too much so yes, the wasn't factored into the answer. All the reason why it would be nice to have a built-in version with all the i's crossed and t's dotted. I'm sure the answer can be modified until such time. $\endgroup$ Commented Oct 18, 2016 at 18:25
$\begingroup$
$\endgroup$
I don't think you can. While I like the options with LinearModelFit and NonlinearModelFit, whoever wrote PairedTTest and TTest (and a few others) seemed to think that all one needs are P-values.
Here's how to get the confidence intervals in a brute-force way:
data1 = {19.4, 21.2, 17.1, 21.1, 19.1, 18.2, 19.6};
data2 = {19.6, 21.1, 18.2, 21.8, 19.5, 18.3, 19.1};
confidenceLevel = 0.95;
diff = data1 - data2;
n = Length[diff];
meanDiff = Mean[diff]
(* -0.271429 *)
se = StandardDeviation[diff]/Sqrt[n];
t = InverseCDF[StudentTDistribution[n - 1], 1 - (1 - confidenceLevel)/2];
{lower, upper} = meanDiff + {-1, 1} t se
(* {-0.757051, 0.214194} *)
2 StandardDeviation[data2 - data1]? $\endgroup$