6
$\begingroup$ 
SampleData = RandomVariate[RayleighDistribution[3], 5000];
Histogram[SampleData]

This creates a histogram that (in my case) looks something like this:

enter image description here

I want to fit this histogram with a curve. The documentation of RandomVariate shows something similar to what I had in mind:

enter image description here

(However the blue line isn't actually a fitted curve; the documentation is simply plotting the [known] analytical formula.)

How can I fit the histogram? The obvious way is to use Interpolation, but the naive attempt to Interpolation[Histogram[SampleData]] doesn't work; Mathematica complains that the histogram is not a list of data and coordinates.

Edit: My raw data contains about 6000 points and looks like this:

enter image description here

This isn't what I'm trying to fit however; I want to fit the histogram, which looks like this:

enter image description here

Ideally the defined "NewFunction" would be able to take input such as NewFunction[3] and return about 70. A normalized NewFunction would be even better.

Improve this question
$\endgroup$
7
  • 2
    $\begingroup$ how SmoothKernelDistribution, e.g., Show[Histogram[SampleData, Automatic, PDF], Plot[Evaluate@PDF[SmoothKernelDistribution[SampleData]][x], {x, 0, 15}, PlotStyle -> Directive[Red, Thick]]]? $\endgroup$
    – kglr
    Jun 18, 2019 at 5:17
  • $\begingroup$ @kglr that works, thanks! $\endgroup$
    – Allure
    Jun 18, 2019 at 5:24
  • $\begingroup$ Allure, posted the comment as an answer. $\endgroup$
    – kglr
    Jun 18, 2019 at 5:31
  • 1
    $\begingroup$ Allure, you want to use bar width and height information from histogram to construct an estimated PDF? $\endgroup$
    – kglr
    Jun 18, 2019 at 5:39
  • 2
    $\begingroup$ Allure: You have the raw data. Constructing a histogram and fitting it loses information. You certainly don't want to use a regression function (NonlinearModelFit, for example). With the large amount of data you have, using SmoothKernelDistribution as @kglr shows you is what you want. If you have a particular probability distribution in mind, then you should consider EstimatedDistribution. $\endgroup$
    – JimB
    Jun 18, 2019 at 15:32

3 Answers 3

Reset to default
4
$\begingroup$

Update 2: Extracting bin center and height information from Histogram output and using Interpolation:

histogram = Histogram[SampleSampleData];
datafromrectangles =  Cases[histogram, 
   Rectangle[a_, b_, ___] :> {Mean[{a[[1]], b[[1]]}], b[[2]]}, All];
intF = Interpolation[datafromrectangles];

Show[histogram, 
 Quiet@ Plot[intF[x], {x, 0, 15}, PlotStyle -> Directive[Blue, Thick], Filling -> Axis]]

enter image description here

Update: An alternative using WeightedData on histogram bin limits and heights combined with Interpolation:

{binlims, heights} = HistogramList[SampleData];
wd = WeightedData[MovingAverage[binlims, 2], heights];
bw = binlims[[2]] - binlims[[1]];
iF = (1/bw) Interpolation[Transpose[wd["EmpiricalPDF"]]][#] &;
Show[Histogram[SampleData, Automatic, PDF], 
 Quiet@ Plot[iF[x], {x, 0, 
       15}, PlotStyle -> Directive[Blue, Thick], Filling -> Axis]]

enter image description here

Note: This approach works only for fixed bin width and PDF as the height function.

Original answer:

You can use SmoothKernelDistribution with SampleData as input to get a distribution object use its PDF:

SeedRandom[1]
SampleData = RandomVariate[RayleighDistribution[3], 5000];
dist =SmoothKernelDistribution[SampleData];

Show[Histogram[SampleData, Automatic, PDF], 
  Plot[Evaluate@PDF[dist][x], {x, 0, 15}, 
   PlotStyle -> Directive[Blue, Thick], Filling -> Axis]]

enter image description here

Improve this answer
$\endgroup$
2
  • $\begingroup$ Thanks for the answer. The second update worked for the sample data, but not the actual data; still I understand what you're doing and will adapt the code. Two questions: what does the "Empirical PDF" do? Also, for the third update, did you define a histogram somewhere previously? As it is, it doesn't run for me. $\endgroup$
    – Allure
    Jun 18, 2019 at 7:30
  • $\begingroup$ @Allure, forgot to copy/paste the line for histogram. For a WeightedData object w, the property "EmpiricalPDF" of w (accessed using w["EmpiricalPDF"]) gives two lists, the first containing data values and the second estimated weights. $\endgroup$
    – kglr
    Jun 18, 2019 at 7:52
3
$\begingroup$

How about just using EstimatedDistribution?

SampleData = RandomVariate[RayleighDistribution[3], 5000];
fitDist = EstimatedDistribution[SampleData, RayleighDistribution[s]]
Show[
 Histogram[SampleData, Automatic, "PDF"],
 Plot[PDF[fitDist, x], {x, 0, 12}]
]

enter image description here

If you have an arbitrary PDF, you can fit to a ProbabilityDistribution:

fitDist = EstimatedDistribution[
 SampleData, 
 ProbabilityDistribution[(E^(-(x^2/(2 s^2))) x)/s^2, {x, 0, \[Infinity]}],
 {{s, 1}}
]
Improve this answer
$\endgroup$
2
$\begingroup$

A bit late in the game, If you want a more 'purist' approach, you could also use HistogramList to extract the points and then fit using an arbitrary function.

SampleData = RandomVariate[RayleighDistribution[3], 5000];
binSize = #[[1, 2]] - #[[1, 1]] &@HistogramList[SampleData];
SampleDataP = {#[[1, 2 ;;]] - 0.5 binSize, #[[2]]}\[Transpose] &@
   HistogramList[SampleData, Automatic, "PDF"];
fit = NonlinearModelFit[SampleDataP, 
   x/\[Sigma]^2 Exp[-x^2/(2 \[Sigma]^2)] , {\[Sigma]}, x];
fit["BestFitParameters"]
Show[{Histogram[SampleData, Automatic, "PDF"], ListPlot@SampleDataP, 
  Plot[fit[x], {x, 0, 12}]}]

enter image description here

As you can see, the model predicts quite well the parameter in question (don't forget to normalise the distribution! ;) )

Improve this answer
$\endgroup$
5
  • $\begingroup$ You can do this, but a least-square fit of a histogram is generally a very bad method for distribution fitting and should be discouraged. The equal-variance assumption that NonlinearModelFit is not satisfied at all for histograms. A simple maximum likelihood fit would be far preferable. $\endgroup$
    – Sjoerd Smit
    Jul 7, 2021 at 17:23
  • $\begingroup$ I see. This is still new to me, just posted what I thought worked well. Does the EstimatedDistribution follow a maximum likelihood fit? Could one not request NonlinearModelFit to do the same? $\endgroup$
    – alex
    Jul 7, 2021 at 18:11
  • $\begingroup$ EstimatedDistribution has a number of methods, but yes: maximum likelihood is one of them. I don't think you can poor this problem easily into NonlinearModelFit because it uses least-squares, but it's generally quite easy to implement your own maximum likelihood method with NMaximize or FindMaximum. $\endgroup$
    – Sjoerd Smit
    Jul 7, 2021 at 20:35
  • $\begingroup$ Hi again Sjoerd! Would you be able to point me to some literature/concept as to why the equal variance method is generally bad for histograms? Does it have to do with how the bin placement is considered? Doing mini tests with various NonlinearModelFit methods produce virtually identical distributions. Is the issue with the complexity of the distribution or the bin size or something? $\endgroup$
    – alex
    Jul 13, 2021 at 10:01
  • $\begingroup$ I'm not quite sure if I know explicit literature that discusses this, but first and foremost: fitting histograms is simply not something that is done or recommended by any reasonable source on statistics. The main problem is that histograms are kinda arbitrary and depend on your choice of bin size. Max likelihood does not have that problem. The next problem is that histograms are always positive whereas NonlinearModelFit assumes an error model that does not enforce this positivity, so it's conceptually wrong as well. Of course, it can sometimes give reasonable results by accident. $\endgroup$
    – Sjoerd Smit
    Jul 13, 2021 at 10:08

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.