I'm trying to find the optical distribution for the following data https://drive.google.com/file/d/1z_cSu7w8tlz_uEYHr8Y0fFBTwLNo4jtp/view?usp=sharing

As there are a lot of points, the only way to make a quick Histogram is to use the following command that includes lower and upper bounds/limits

 pEx = Histogram[{dataEx}, {-10 10^6, 10 10^6, 2 10^5}, PlotRange -> {{-10 10^6, 10 10^6}, All}]

enter image description here

If I now ask for the corresponding distribution, through the following command:


I obtain

CauchyDistribution[10787.5, 3.13764*10^6]

However, if I now plot the Cauch distribution above -- with an amplitude scaling factor -- i.e.,

fit[x_] := 2.5 10^10 PDF[CauchyDistribution[10787.533402804394`, 3.13763790539067`*^6], x]

I obtain

enter image description here

which clearly disagree with the original data.

However, I have notice however that this distribution fits quite well the region far from the origin, i.e., by redefining now

fit[x_] := .55 10^10 PDF[CauchyDistribution[10787.533402804394`, 3.13763790539067`*^6], x]

I obtain

enter image description here

which clearly give us a good match with the tail of the distribution.

Therefore, I'm wondering whether I'm doing something wrong here. Thanks in advance!

  • $\begingroup$ Presumably FindDistribution uses something akin to least-squares. Since you have way more points "far from the origin", the points near the center carry little weight, so the fit is poor there $\endgroup$ – johnny Jun 10 '20 at 18:37
  • $\begingroup$ That makes sense -- would you know how to fit only around the origin? $\endgroup$ – denis Jun 10 '20 at 18:47
  • $\begingroup$ @denis Can you please grant public access to the gDrive link. I am unable to access it. $\endgroup$ – Rohit Namjoshi Jun 10 '20 at 18:57
  • $\begingroup$ @RohitNamjoshi Oh, I'm sorry. I've just fixed the problem. Thanks! $\endgroup$ – denis Jun 10 '20 at 19:10
  • $\begingroup$ @denis Have you tried FindDistribution on the subset of the data that is used in Histogram? Filter the data to be in the range {-10 10^6, 10 10^6}. $\endgroup$ – Rohit Namjoshi Jun 10 '20 at 20:32

dataEx = Import["/Users/roberthanlon/Downloads/test.dat"] // Flatten;

Since you are interested in matching the distribution of the data in the interval {-10^7, 10^7} filter the data to that interval.

dataEx2 = Select[dataEx, -10^7 <= # <= 10^7 &];

Length /@ {dataEx, dataEx2}

(* {50000, 34069} *)

About 1/3 of the data is outside the range.

Use SmoothKernelDistribution for the distribution

dist2 = SmoothKernelDistribution[dataEx2];

Comparing the distributions,

 Histogram[{dataEx2}, {-10^7, 10^7, 2*^5},
 Plot[PDF[dist2, x], {x, -10^7, 10^7},
  PlotRange -> All],
 ImageSize -> Large]

enter image description here


An ExponentialPowerDistribution looks like a good fit, even if a little shallow in the far tails:

data = Select[Flatten[Import["test.dat"]], -10^7 < # < 10^7 &];
dist = EstimatedDistribution[data, ExponentialPowerDistribution[k, m, s]]
 Histogram[data, {-10^7, 10^7, 200000}, "PDF", PlotTheme -> "Classic"],
 Plot[PDF[dist, x], {x, -10^7, 10^7}, PlotRange -> All, PlotStyle -> Thick]

exponential power distribution


Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.