1
$\begingroup$

I'm trying to find the optical distribution for the following data https://drive.google.com/file/d/1z_cSu7w8tlz_uEYHr8Y0fFBTwLNo4jtp/view?usp=sharing

As there are a lot of points, the only way to make a quick Histogram is to use the following command that includes lower and upper bounds/limits

 pEx = Histogram[{dataEx}, {-10 10^6, 10 10^6, 2 10^5}, PlotRange -> {{-10 10^6, 10 10^6}, All}]

enter image description here

If I now ask for the corresponding distribution, through the following command:

FindDistribution[dataEx]

I obtain 

CauchyDistribution[10787.5, 3.13764*10^6]

However, if I now plot the Cauch distribution above -- with an amplitude scaling factor -- i.e., 

fit[x_] := 2.5 10^10 PDF[CauchyDistribution[10787.533402804394`, 3.13763790539067`*^6], x]

I obtain

enter image description here

which clearly disagree with the original data.

However, I have notice however that this distribution fits quite well the region far from the origin, i.e., by redefining now 

fit[x_] := .55 10^10 PDF[CauchyDistribution[10787.533402804394`, 3.13763790539067`*^6], x]

I obtain

enter image description here

which clearly give us a good match with the tail of the distribution.

Therefore, I'm wondering whether I'm doing something wrong here. Thanks in advance!

Improve this question
$\endgroup$
8
  • $\begingroup$ Presumably FindDistribution uses something akin to least-squares. Since you have way more points "far from the origin", the points near the center carry little weight, so the fit is poor there $\endgroup$
    – johnny
    Jun 10, 2020 at 18:37
  • $\begingroup$ That makes sense -- would you know how to fit only around the origin? $\endgroup$
    – sined
    Jun 10, 2020 at 18:47
  • $\begingroup$ @denis Can you please grant public access to the gDrive link. I am unable to access it. $\endgroup$
    – Rohit Namjoshi
    Jun 10, 2020 at 18:57
  • $\begingroup$ @RohitNamjoshi Oh, I'm sorry. I've just fixed the problem. Thanks! $\endgroup$
    – sined
    Jun 10, 2020 at 19:10
  • $\begingroup$ @denis Have you tried FindDistribution on the subset of the data that is used in Histogram? Filter the data to be in the range {-10 10^6, 10 10^6}. $\endgroup$
    – Rohit Namjoshi
    Jun 10, 2020 at 20:32

2 Answers 2

Reset to default
2
$\begingroup$ 
Clear["Global`*"]

dataEx = Import["/Users/roberthanlon/Downloads/test.dat"] // Flatten;

Since you are interested in matching the distribution of the data in the interval {-10^7, 10^7} filter the data to that interval.

dataEx2 = Select[dataEx, -10^7 <= # <= 10^7 &];

Length /@ {dataEx, dataEx2}

(* {50000, 34069} *)

About 1/3 of the data is outside the range.

Use SmoothKernelDistribution for the distribution

dist2 = SmoothKernelDistribution[dataEx2];

Comparing the distributions,

Show[
 Histogram[{dataEx2}, {-10^7, 10^7, 2*^5},
  "PDF"],
 Plot[PDF[dist2, x], {x, -10^7, 10^7},
  PlotRange -> All],
 ImageSize -> Large]

enter image description here

Improve this answer
$\endgroup$
4
$\begingroup$

An ExponentialPowerDistribution looks like a good fit, even if a little shallow in the far tails:

data = Select[Flatten[Import["test.dat"]], -10^7 < # < 10^7 &];
dist = EstimatedDistribution[data, ExponentialPowerDistribution[k, m, s]]
Show[
 Histogram[data, {-10^7, 10^7, 200000}, "PDF", PlotTheme -> "Classic"],
 Plot[PDF[dist, x], {x, -10^7, 10^7}, PlotRange -> All, PlotStyle -> Thick]
]

exponential power distribution

Improve this answer
$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.