# Loop and list all p-values

I have the following mathematica code (see below). Is there a way to iterate this approach in such a way to get the p-value results of each test in a list? I would like to get say 1000 p-values for each test and graph their distributions. All at the $\alpha=0.01$ level of significance. Thank you!

DD[a_, b_, c_, μ_, ν_, σ_, τ_] =
MixtureDistribution[{a, b, c, a, b,
c}, {NormalDistribution[-μ, τ],
NormalDistribution[-ν, σ],
NormalDistribution[-μ - ν, σ + τ],
NormalDistribution[μ, τ],
NormalDistribution[ν, σ],
NormalDistribution[μ + ν, σ + τ]}];
data = RandomVariate[\[ScriptD] =
JohnsonDistribution["SU", 0, 1, 0, 1],
1000];  (*Target Distributions here*)
data1 = RandomVariate[f = DD[0.46, 0.435, 0.105, 0, 0, 0.671, 1.7],
1000]; (*SIGMA goes here*)
DistributionFitTest[data, f, {"TestDataTable", All}]

• Not sure what you're trying to accomplish here - you realize p-values under proper nulls will be uniformly distributed, yes? In any case, just create a list of test data lists (e.g., RandomVariate[dist,{1000,1000}]), map DistributionFitTest over that, extract the results and plot as desired. Look at documentation for Map for details on its use. – ciao May 3 '15 at 23:32
• Perhaps I am missing something here. The p-values are only distributed uniformly if the null hypothesis is true. This will not always be the case here. – dsmalenb May 4 '15 at 0:15
• That's essentially correct, (there are subtleties re: the converse), so if your goal is just to show the distributions, map as in prior comment. – ciao May 4 '15 at 0:19
• Also, if you could provide the snippet of code to validate your approach, I would be most thankful. – dsmalenb May 4 '15 at 0:21
• Sure, gimmie a moment, I'll post a quick-n-dirty... NVM - Bob did so already, somewhat similar to what I cobbled up. Do note, the SignificanceLevel argument has no real meaning for this - It's used for the "reporting" and "diagnostic" functions of the test to determine rejection/acceptance, has no bearing on p-values generated. – ciao May 4 '15 at 0:35

DD[a_, b_, c_, μ_, ν_, σ_, τ_] =
MixtureDistribution[{a, b, c, a, b,
c}, {NormalDistribution[-μ, τ],
NormalDistribution[-ν, σ],
NormalDistribution[-μ - ν, σ + τ],
NormalDistribution[μ, τ],
NormalDistribution[ν, σ],
NormalDistribution[μ + ν, σ + τ]}];

tests = {AndersonDarlingTest, CramerVonMisesTest,
KolmogorovSmirnovTest, KuiperTest, PearsonChiSquareTest,
WatsonUSquareTest};

n = 100;(* change to your larger number, i.e., 1000 *)

pValues = Table[
data =
RandomVariate[\[ScriptD] = JohnsonDistribution["SU", 0, 1, 0, 1],
1000];(*Target Distributions here*)

data1 = RandomVariate[
f = DD[0.46, 0.435, 0.105, 0, 0, 0.671, 1.7],
1000];(*SIGMA goes here*)
#[data, f,
SignificanceLevel -> 0.01] & /@ tests,
{n}] // Transpose;

Partition[
Histogram[#[[2]], PlotLabel -> #[[1]]] & /@
Transpose[{tests, pValues}],
2] // Grid


• Nice, +1, see my comment above re: SignificanceLevel – ciao May 4 '15 at 0:40