0
$\begingroup$

Here is a sample of data which corresponds to the following histogram

data = Flatten[Import["hist.dat", "Table"], 1];

P00 = Histogram[data, Automatic, "Probability", ChartStyle -> Gray, 
      ChartBaseStyle -> EdgeForm[None], Frame -> True, 
      FrameLabel -> {"N", "P"}, RotateLabel -> False, 
      LabelStyle -> Directive[FontFamily -> "Helvetica", 20], 
      Epilog -> {Red, Thick, Dashed, 
      Line[{{#, 0}, {#, 1}} &@Last@Commonest[data]]}, PlotRange -> All, 
      PlotRangePadding -> 0.001, AspectRatio -> 1, ImageSize -> 550]

enter image description here

Now I want to find the best possible probability distribution function (PDF). So let's try some of the most well-known PDFs:

1. Laplace distribution:

fit1 = FindDistributionParameters[Flatten@data, LaplaceDistribution[m, b]]
m0 = m /. fit1[[1]];
b0 = b /. fit1[[2]]; 
t1 = Plot[PDF[LaplaceDistribution[m0, b0], x], {x, 0, 30}, 
     PlotStyle -> {Magenta, Thick}, PlotRange -> All];
Show[{P00, t1}, Axes -> False]

enter image description here

As we can see, the right-hand side of the distribution is acceptable but the left-hand side is not.

2. Maxwell distribution:

fit2 = FindDistributionParameters[Flatten@data, MaxwellDistribution[m]]
m1 = m /. fit2[[1]];
t2 = Plot[PDF[MaxwellDistribution[m1], x], {x, 0, 30}, PlotStyle -> {Magenta, Thick}];
Show[{P00, t2}, Axes -> False]

enter image description here

3. Reyleigh distribution:

fit3 = FindDistributionParameters[Flatten@data, RayleighDistribution[m]]
m2 = m /. fit3[[1]];
t3 = Plot[PDF[RayleighDistribution[m2], x], {x, 0, 30}, PlotStyle -> {Magenta, Thick}];
Show[{P00, t3}, Axes -> False]

enter image description here

We observe that the last two types of distribution do not fit well the data. In particular, they do not go high enough.

My question: Is there a way to fix/modify/improve these types of PDFs in order to achieve the best possible fit in my data? Moreover, is there any other type of PDF which might work better in this case?

Many thanks in advance!

$\endgroup$
  • $\begingroup$ "Why?" Because that would be an even worse fit, the distributions tie height with width. You can't just guess at the distributions, surely you have an idea which one would fit before you run the experiment? $\endgroup$ – Feyre Oct 28 '16 at 8:39
  • $\begingroup$ Have you tried FindDistribution? $\endgroup$ – corey979 Oct 28 '16 at 9:39
  • 1
    $\begingroup$ If your integer data consists of counts, then attempting to fit continuous distributions will never get you the "correct" distribution (which is taken directly from your question). If your integer data is simply rounded, then treating your data as if it were continuous is not the best way to estimate the parameters in an underlying continuous distribution. So...what is the generating process of your data? As has been suggested by others in your previous question, are there not candidate distributions suggested or implied by your experiment? $\endgroup$ – JimB Oct 28 '16 at 15:19
  • 1
    $\begingroup$ Your data is clearly discrete (iintegers going from 4 to 29) and so fitting a continuous distribution is not likely a great approach. If you could mention something about the data generation process, that would help. For instance, are these numbers counts? Is there just a lot of round-off that results in integer values? And why you think you need to estimate a probability density or probability mass function, would also be helpful. In other words, what would having a function with a few parameters benefit you over just showing the histogram/frequency table? $\endgroup$ – JimB Dec 8 '18 at 17:46
  • 1
    $\begingroup$ I don't know why but I felt if I repeated myself from 2 years ago, it might have an effect. And note that whatever you end up with, it won't be "correct" (as requested in the title). It will be (hopefully) a reasonably parsimonious description of the data (no more, no less). $\endgroup$ – JimB Dec 8 '18 at 17:50
3
$\begingroup$

From a statistical point of view, the question is not well-posed. If you insist on finding some distribution, then use FindDistribution:

data = Import["hist.dat", "List"];
th = Last @ Commonest[data];
data = Select[data, # >= th &]; (*because in a previous question you were interested only in these data *)

dist = FindDistribution[data]

MixtureDistribution[{0.859154, 0.140846}, {PascalDistribution[6, 0.834241], PoissonDistribution[12.6456]}]

t1 = Plot[PDF[dist, x], {x, th, 30}, PlotStyle -> {Black, Thick}, 
   PlotRange -> All];
Show[P00, t1, Axes -> False]

enter image description here

$\endgroup$
  • $\begingroup$ FindDistribution is not recognized in version 9 :( Is there a way to obtain the analytical expression of dist? $\endgroup$ – Vaggelis_Z Oct 28 '16 at 12:56
  • $\begingroup$ You can take the dist from the answer (can take also general parameters of the distributions) and PDF[dist, x] will give an analytical formula. $\endgroup$ – corey979 Oct 28 '16 at 13:01
  • 1
    $\begingroup$ Are there any references as to how exactly FindDistribution works? The entry in Wolfram Reference does not have much information besides the fact there is a Bayesian approach behind the scenes. $\endgroup$ – Skumin Sep 21 '17 at 8:51
  • $\begingroup$ The OP specifies that he is looking for a pdf (which stands for probability density function - not prob distribution function). By standard statistical nomenclature, this means s/he is looking for a continuous distribution, whereas that provided above is discrete. Perhaps the OP could clarify what s/he wants ... but if the OP is after a discrete distribution, the correct expression is pmf (not pdf). $\endgroup$ – wolfies Dec 8 '18 at 13:35

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.