0
$\begingroup$

I have 2 custom data sets:

data1 = Import["https://pastebin.com/raw/KHkmEfQB", "List"];
data2 = Import["https://pastebin.com/raw/Hm3mhczi", "List"];

These are available here:

data1_pastebin
data2_pastebin

(I actually import these from files but you get the idea).

The data sets are NOT normally distributed. Here's an image of data1 and data2, respectively:

data1 data2

I create a distribution from both:

skdData1 = SmoothKernelDistribution[data1, Automatic, {"Bounded", {20, 105}, "Gaussian"}];

skdData2 = SmoothKernelDistribution[data2, Automatic, {"Bounded", {20, 105}, "Gaussian"}];

Now, I would like to test data2's $p$-Value for data1's distribution. I am going to be manipulating data2 in a variety of ways, and want to keep testing the $p$-Value to see which manipulation makes the best fit. I won't go into the details of that here, but I need a way to accurately compare data2 to data1 and the only built-in function I found in Mathematica are for normal distributions.

Any help is appreciated!

$\endgroup$
  • $\begingroup$ Please consider posting (a subset of) your data to Pastebin so that other people have something to work on. $\endgroup$ – J. M. will be back soon Aug 5 '17 at 14:04
  • $\begingroup$ data subsets added! $\endgroup$ – Matt Stein Aug 5 '17 at 15:37
  • $\begingroup$ So, you want to do something like DistributionFitTest[data2, skdData1, {"TestDataTable", All}]? $\endgroup$ – J. M. will be back soon Aug 5 '17 at 16:17
  • $\begingroup$ Well, my concern with that is that I this gives me a P-Value less than 1: DistributionFitTest[data1, skdData1, "TestDataTable"] It's close to 0.84, which is weird to me... since I made the distribution from that data set... $\endgroup$ – Matt Stein Aug 5 '17 at 17:10
  • 1
    $\begingroup$ That you will be manipulating data2 sounds fishy. Do you mean you will be manipulating the parameters in the process that produces data sets like data2? It's your odd use of statistical terms such as "my concern with that is that...this give me a P-Value less than 1" - all P-values range from 0 to 1 and P-values from samples from the tested distribution will essentially never be 1. They will have a uniform distribution - U(0,1). And "I would like to test data2's p-Value for `data1's distribution". P-values aren't tested. $\endgroup$ – JimB Aug 6 '17 at 3:15

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Browse other questions tagged or ask your own question.