0
$\begingroup$

I have the following mathematica code (see below). Is there a way to iterate this approach in such a way to get the p-value results of each test in a list? I would like to get say 1000 p-values for each test and graph their distributions. All at the $\alpha=0.01$ level of significance. Thank you!

DD[a_, b_, c_, μ_, ν_, σ_, τ_] = 
  MixtureDistribution[{a, b, c, a, b, 
    c}, {NormalDistribution[-μ, τ], 
    NormalDistribution[-ν, σ], 
    NormalDistribution[-μ - ν, σ + τ], 
    NormalDistribution[μ, τ], 
    NormalDistribution[ν, σ], 
    NormalDistribution[μ + ν, σ + τ]}];
data = RandomVariate[\[ScriptD] = 
   JohnsonDistribution["SU", 0, 1, 0, 1], 
  1000];  (*Target Distributions here*)
data1 = RandomVariate[f = DD[0.46, 0.435, 0.105, 0, 0, 0.671, 1.7], 
  1000]; (*SIGMA goes here*)
DistributionFitTest[data, f, {"TestDataTable", All}]
$\endgroup$
  • $\begingroup$ Not sure what you're trying to accomplish here - you realize p-values under proper nulls will be uniformly distributed, yes? In any case, just create a list of test data lists (e.g., RandomVariate[dist,{1000,1000}]), map DistributionFitTest over that, extract the results and plot as desired. Look at documentation for Map for details on its use. $\endgroup$ – ciao May 3 '15 at 23:32
  • $\begingroup$ Perhaps I am missing something here. The p-values are only distributed uniformly if the null hypothesis is true. This will not always be the case here. $\endgroup$ – dsmalenb May 4 '15 at 0:15
  • $\begingroup$ That's essentially correct, (there are subtleties re: the converse), so if your goal is just to show the distributions, map as in prior comment. $\endgroup$ – ciao May 4 '15 at 0:19
  • $\begingroup$ Also, if you could provide the snippet of code to validate your approach, I would be most thankful. $\endgroup$ – dsmalenb May 4 '15 at 0:21
  • $\begingroup$ Sure, gimmie a moment, I'll post a quick-n-dirty... NVM - Bob did so already, somewhat similar to what I cobbled up. Do note, the SignificanceLevel argument has no real meaning for this - It's used for the "reporting" and "diagnostic" functions of the test to determine rejection/acceptance, has no bearing on p-values generated. $\endgroup$ – ciao May 4 '15 at 0:35
4
$\begingroup$
DD[a_, b_, c_, μ_, ν_, σ_, τ_] = 
  MixtureDistribution[{a, b, c, a, b, 
    c}, {NormalDistribution[-μ, τ], 
    NormalDistribution[-ν, σ], 
    NormalDistribution[-μ - ν, σ + τ], 
    NormalDistribution[μ, τ], 
    NormalDistribution[ν, σ], 
    NormalDistribution[μ + ν, σ + τ]}];

tests = {AndersonDarlingTest, CramerVonMisesTest, 
   KolmogorovSmirnovTest, KuiperTest, PearsonChiSquareTest, 
   WatsonUSquareTest};

n = 100;(* change to your larger number, i.e., 1000 *)

pValues = Table[
    data = 
     RandomVariate[\[ScriptD] = JohnsonDistribution["SU", 0, 1, 0, 1],
       1000];(*Target Distributions here*)

    data1 = RandomVariate[
      f = DD[0.46, 0.435, 0.105, 0, 0, 0.671, 1.7], 
      1000];(*SIGMA goes here*)
    #[data, f, 
       SignificanceLevel -> 0.01] & /@ tests,
    {n}] // Transpose;

Partition[
  Histogram[#[[2]], PlotLabel -> #[[1]]] & /@
   Transpose[{tests, pValues}],
  2] // Grid

enter image description here

|improve this answer|||||
$\endgroup$
  • $\begingroup$ Nice, +1, see my comment above re: SignificanceLevel $\endgroup$ – ciao May 4 '15 at 0:40

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.