# Finding a Distribution

I'm trying to find the optical distribution for the following data https://drive.google.com/file/d/1z_cSu7w8tlz_uEYHr8Y0fFBTwLNo4jtp/view?usp=sharing

As there are a lot of points, the only way to make a quick Histogram is to use the following command that includes lower and upper bounds/limits

 pEx = Histogram[{dataEx}, {-10 10^6, 10 10^6, 2 10^5}, PlotRange -> {{-10 10^6, 10 10^6}, All}]


If I now ask for the corresponding distribution, through the following command:

FindDistribution[dataEx]


I obtain

CauchyDistribution[10787.5, 3.13764*10^6]


However, if I now plot the Cauch distribution above -- with an amplitude scaling factor -- i.e.,

fit[x_] := 2.5 10^10 PDF[CauchyDistribution[10787.533402804394, 3.13763790539067*^6], x]


I obtain

which clearly disagree with the original data.

However, I have notice however that this distribution fits quite well the region far from the origin, i.e., by redefining now

fit[x_] := .55 10^10 PDF[CauchyDistribution[10787.533402804394, 3.13763790539067*^6], x]


I obtain

which clearly give us a good match with the tail of the distribution.

Therefore, I'm wondering whether I'm doing something wrong here. Thanks in advance!

• Presumably FindDistribution uses something akin to least-squares. Since you have way more points "far from the origin", the points near the center carry little weight, so the fit is poor there – johnny Jun 10 '20 at 18:37
• That makes sense -- would you know how to fit only around the origin? – denis Jun 10 '20 at 18:47
• @denis Can you please grant public access to the gDrive link. I am unable to access it. – Rohit Namjoshi Jun 10 '20 at 18:57
• @RohitNamjoshi Oh, I'm sorry. I've just fixed the problem. Thanks! – denis Jun 10 '20 at 19:10
• @denis Have you tried FindDistribution on the subset of the data that is used in Histogram? Filter the data to be in the range {-10 10^6, 10 10^6}. – Rohit Namjoshi Jun 10 '20 at 20:32

Clear["Global*"]



Since you are interested in matching the distribution of the data in the interval {-10^7, 10^7} filter the data to that interval.

dataEx2 = Select[dataEx, -10^7 <= # <= 10^7 &];

Length /@ {dataEx, dataEx2}

(* {50000, 34069} *)


About 1/3 of the data is outside the range.

Use SmoothKernelDistribution for the distribution

dist2 = SmoothKernelDistribution[dataEx2];


Comparing the distributions,

Show[
Histogram[{dataEx2}, {-10^7, 10^7, 2*^5},
"PDF"],
Plot[PDF[dist2, x], {x, -10^7, 10^7},
PlotRange -> All],
ImageSize -> Large]


An ExponentialPowerDistribution looks like a good fit, even if a little shallow in the far tails:

data = Select[Flatten[Import["test.dat"]], -10^7 < # < 10^7 &];
dist = EstimatedDistribution[data, ExponentialPowerDistribution[k, m, s]]
Show[
Histogram[data, {-10^7, 10^7, 200000}, "PDF", PlotTheme -> "Classic"],
Plot[PDF[dist, x], {x, -10^7, 10^7}, PlotRange -> All, PlotStyle -> Thick]
]
`