1
$\begingroup$ 
data = {0.180723, 0.181208, 0.182213, 0.1875, 0.1875, 0.1875, 0.1875,
0.1875, 0.1875, 0.190476, 0.191041, 0.19174, 0.192308, 0.192513
0.193038, 0.194118, 0.194858, 0.195172, 0.196141, 0.196507, 0.196911,
0.19717, 0.199725, 0.2, 0.2, 0.2, 0.2, 0.2, 0.2, 0.2, 0.2, 0.2,
0.204804, 0.204887, 0.206148, 0.207435, 0.208861, 0.211034, 0.213389,
0.214286, 0.214286, 0.214286, 0.215247, 0.218447, 0.22028, 0.221334,
0.221519, 0.222222, 0.224227, 0.224359, 0.225352, 0.226485, 0.230769,
0.230769, 0.230769, 0.230769, 0.230769, 0.230769, 0.231561, 0.23622,
0.239075, 0.24, 0.24, 0.241667, 0.241758, 0.246269, 0.247842, 0.25,
0.25, 0.25, 0.25, 0.25, 0.25, 0.25, 0.25, 0.25, 0.25, 0.25, 0.254902,
0.26087, 0.26087, 0.264706, 0.269461, 0.272727, 0.273684, 0.277778,
0.287576, 0.28934, 0.295775, 0.298013, 0.3, 0.3, 0.3, 0.3, 0.304124,
0.305085, 0.310345, 0.333333, 0.333333, 0.333333, 0.333333, 0.333333,
0.333333, 0.333333, 0.333333, 0.333333, 0.333333, 0.333333, 0.357143,
0.36, 0.375, 0.375, 0.375, 0.375, 0.375, 0.387097, 0.4, 0.409091,
0.409091, 0.5, 0.5, 0.6}

dataDist = FindDistribution[data,3,
   {"CramerVonMises","PearsonChiSquare"},"RandomSeed"->23544325]

{{GammaDistribution[7.34584,0.0353382],{0.896823,0.943734}}, {LogNormalDistribution[-1.41827,0.377612],{0.547505,0.252689}}, {WeibullDistribution[1.81133,0.194603,0.086273],{0.657403,0.786422}}}

Hdata=DistributionFitTest[data,GammaDistribution[7.34584,0.0353382],"HypothesisTestData"]

  Statistic   P-Value
Anderson-Darling    6.04043 0.000934535
Cramér-von Mises    0.923269    0.00372792
Pearson \[Chi]^2    77.2131 3.67716*10^-11

Based on the FindDistribution results the Gamma distribution seems to fit better these data.However, the DistributionFitTest did not confirm this conclusion. Why ? Where I am going the wrong way ?

Addendum:

The FindDistribution function does not handle these data set either:

data1={1.,1.,1.,1.,1.,1.,1.,1.,1.,1.,1.,1.,1.,1.,1.,1.,1.,1.,1.,1.,1.,1.,1.,1.,1.,1.,1.,1.,1.,1.,1.,1.,1.,1.,1.,1.,1.,1.,1.,1.,1.,1.,1.,1.,1.,1.,1.,1.,1.,1.,1.,1.,1.,1.,1.,1.,1.,1.,1.,1.,1.,1.,1.,1.,1.,1.,1.,1.,1.,1.,1.,1.,1.,1.,1.,1.,1.,1.,1.,1.,1.,1.,1.,1.,1.,1.,1.,1.,1.,1.,1.,1.,1.,1.,1.,1.,1.,1.,1.,1.,1.,1.,1.,1.,1.,1.,1.,1.,1.,1.,1.,1.,1.,1.,1.,1.,1.,1.,1.,1.,1.,1.,1.,1.,1.,1.,1.,1.,1.,1.,1.,1.,1.,1.,1.,1.,1.,1.,1.,1.,1.,1.,1.,1.,1.,1.,1.,1.,1.,1.,1.,1.,1.,1.,1.,1.,1.,1.,1.,1.,1.,1.,1.,1.,1.,1.,1.,1.,1.,1.,1.,1.,1.,1.,1.,1.,1.,1.,1.,1.,1.,1.,1.,1.,1.,1.,1.,1.,1.,1.,1.,1.,1.,1.,1.,1.,1.,1.,1.,1.,1.,1.,1.,1.,1.,1.,1.,1.,1.,1.,1.,1.,1.,1.,1.,1.,1.,1.,1.,1.,1.,1.,1.,1.,1.,1.,1.,1.,1.,1.,1.,1.,1.,1.,1.,1.,1.,1.16667,1.33333,1.33333,1.33333,1.33333,1.33333,1.33333,1.33333,1.33333,1.33333,1.33333,1.33333,1.33333,1.33333,1.5,1.5,1.5,1.5,1.5,1.5,1.5,1.5,1.5,1.5,1.5,1.5,1.5,1.5,1.5,1.5,1.5,1.5,1.5,1.5,1.5,1.5,1.5,1.5,1.5,1.5,1.5,1.5,1.5,1.5,1.5,1.5,1.5,1.5,1.5,1.5,1.5,1.5,1.5,1.5,1.5,1.5,1.5,1.5,1.5,1.5,1.5,1.5,1.5,1.5,1.5,1.5,1.5,1.5,1.5,1.5,1.5,1.5,1.5,1.5,1.5,1.5,1.5,1.5,1.5,1.5,1.5,1.5,1.5,1.5,1.5,1.5,1.5,1.5,1.5,1.5,1.5,1.5,1.5,1.5,1.5,1.5,1.5,1.5,1.5,1.5,1.5,1.5,1.5,1.5,1.5,1.5,1.5,1.5,1.5,1.5,1.5,1.5,1.5,1.5,1.5,1.5,1.5,1.5,1.5,1.5,1.5,1.5,1.5,1.5,1.5,1.5,1.5,1.5,1.5,1.5,1.5,1.5,1.5,1.5,1.5,1.5,1.5,1.5,1.5,1.5,1.5,1.5,1.5,1.5,1.5,1.5,1.5,1.5,1.5,1.5,1.5,1.5,1.5,1.5,1.5,1.5,1.5,1.5,1.5,1.5,1.5,1.5,1.5,1.5,1.5,1.5,1.5,1.5,1.5,1.5,1.5,1.5,1.5,1.5,1.5,1.5,1.5,1.5,1.5,1.5,1.5,1.5,1.5,1.5,1.5,1.5,1.5,1.5,1.5,1.5,1.5,1.6,1.6,1.66667,1.66667,1.66667,1.66667,1.66667,1.66667,1.66667,1.66667,1.66667,1.66667,1.66667,1.66667,1.66667,1.66667,1.66667,1.66667,1.66667,1.66667,1.66667,1.66667,1.66667,1.66667,1.66667,1.66667,1.66667,1.66667,1.66667,1.66667,1.66667,1.66667,1.66667,1.66667,1.66667,1.73333,1.73333,1.73333,1.73333,1.73333,1.73333,1.8,1.8,1.8,1.8,1.8,1.86667,1.86667,1.86667,1.92857,1.93333,1.93333,1.93333,1.93333,1.93333,1.93333,1.93333,1.93333,1.93333,1.93333,1.93333,1.93333,1.93333,1.93333,1.93333,1.93333,1.93333,1.93333,1.93333,1.93333,1.93333,1.93333,1.93333,1.93333,1.93333,1.93333,1.93333,1.93333,1.93333,1.93333,1.93333,1.93333,1.93333,1.93333,1.93333,1.93333,1.93333,1.93333,1.93333,1.93333,1.93333,1.93333,1.93333,1.93333,1.93333,1.93333,1.93333,1.93333,1.93333,1.93333,1.93333,1.93333,1.93333,1.93333,1.93333,1.93333,1.93333,1.93333,1.93333,1.93333,1.93333,1.93333,1.93333,1.93333,1.93333,1.93333,2.,2.06667,2.07143,2.07143,2.13333,2.13333,2.13333,2.13333,2.13333,2.13333,2.13333,2.13333,2.13333,2.13333,2.13333,2.13333,2.14286,2.14286,2.14286,2.19048,2.25,2.25,2.25,2.25,2.25,2.27778,2.28571,2.32143,2.32143,2.32143,2.32143,2.32143,2.32143,2.32143,2.32143,2.32143,2.32143,2.32143,2.32143,2.32143,2.32143,2.32143,2.32143,2.32143,2.32143,2.33333,2.35714,2.4,2.5,2.51515,2.57778,2.57778,2.62121,2.64444,2.64444,2.64444,2.66667,2.68889,2.68889,2.68889,2.68889,2.68889,2.68889,2.68889,2.68889,2.68889,2.69091,2.71795,2.84848,2.87879,2.89394,2.98095,3.01515,3.01515,3.02857,3.06061,3.06061,3.15385,3.16484,3.16484,3.19697,3.24242,3.26374,3.27473,3.27473,3.44167,3.45833,3.48366,3.5,3.6,3.66667,3.97044,4.00833,4.07792,4.14667,4.22105,4.61846,6.35829,18.603,18.9127,20.0815,21.4653,21.8075,23.1257,23.174,23.195,24.8591,25.5128,25.6204,25.7491,27.2341,28.6521,29.1464,29.2351,29.9114,30.1133,30.7835,31.6615,32.4578,32.6746,34.5677,34.9822,35.3298,35.3304,35.5266,36.5512,36.9395,38.5341,38.5421,39.0514,44.5931,47.1162,63.3183,64.7268,72.3538,80.9747,87.87,90.8965}

    FindDistribution[data1, 3, {"CramerVonMises", "PearsonChiSquare"}, 
     "RandomSeed" -> 23544325]
{{MixtureDistribution[{0.950331, 
    0.0496689}, {LogNormalDistribution[0.354894, 0.349561], 
    LogNormalDistribution[3.62601, 0.392707]}], {0.0000939677, 
   1.61301*10^-90}}, {MixtureDistribution[{0.9503, 
    0.0496998}, {LogNormalDistribution[0.354858, 0.349509], 
    GammaDistribution[5.66106, 7.257]}], {0.0000935481, 
   1.61301*10^-90}}, {MixtureDistribution[{0.936836, 
    0.0631641}, {NormalDistribution[1.48875, 0.529605], 
    NormalDistribution[33.145, 23.7551]}], {0.0000256599, 
   1.04431*10^-135}}}

How can I make FindDistribution perform better in this case?

NB: This is an experimental data and it is what it is.

Improve this question
$\endgroup$
6
  • $\begingroup$ What version of Mathematica are you using? On Windows 7 Mathematica 10.4.1 and 11.1.1 the GammaDistribution, LogNormalDistribution, and Weibull are not the top 3 models. $\endgroup$
    – JimB
    Jul 3, 2017 at 0:19
  • 1
    $\begingroup$ I wonder if the discrepancy has to do with the large number of ties in your data? The following produces identical P-values: data = RandomVariate[GammaDistribution[7.34584, 0.0353382], 122] dataDist = FindDistribution[data, 1, {"PearsonChiSquare"}] DistributionFitTest[data, dataDist[[1, 1]], {"PearsonChiSquare"}] $\endgroup$
    – JimB
    Jul 3, 2017 at 0:45
  • $\begingroup$ If you add a small amount of noise to get rid of the ties, the differences in the P-values go away: data = data + RandomVariate[UniformDistribution[{-0.00001, 0.00001}], Length[data]]. (This is not a recommended fix. If you have lots of ties, then you don't have appropriate data for these tests. One can do other tests but those you should ask about on CrossValidated.) $\endgroup$
    – JimB
    Jul 3, 2017 at 1:09
  • $\begingroup$ @Jim Baldwin. I use Mathematica 11.Not sure why we don't get the same top 3 models. Did you use ""RandomSeed"->23544325 ? I understood what you said and I will post a question in CrossValidated but still I am not able to wrap my head around it. If ties are the main issue then the Cramer and Pearson tests should have failed anyway regardless of the function used.Am I completely wrong ? also I actually tried all the other tests available under the DistributionFitTest including KS and they failed as well. Intriguingly the KS works just fine with the same data set using R! $\endgroup$
    – Pure Function
    Jul 3, 2017 at 1:52
  • 1
    $\begingroup$ I did use that same random seed on both a Windows 7 and Windows 10 PC and did not get your results. However, it's not FindDistribution that's the problem. Your expectations about finding a continuous distribution with so many ties when FindDistribution does not assume ties are a possibility is the problem. Your other dataset is even worse in that respect. A histogram of the values (and the log of those values) demonstrates that a nice smooth curve form with few parameters is just not achievable. R might not have failed for the test but the fit is still poor. $\endgroup$
    – JimB
    Jul 3, 2017 at 5:38

0

Reset to default

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Browse other questions tagged or ask your own question.