1
$\begingroup$

I would like to estimate the parameters associated with a distribution following the Laplace distribution.

I have created experimental data following the Laplace distribution, using the following code:

data = RandomVariate[ExponentialPowerDistribution[1, 0.5, 1], 1000];

Given the data, I found the shape, location, and scale parameters by using FindDistributionParameters:

FindDistributionParameters[data, ExponentialPowerDistribution[Kappa, Mu, Sigma]]

The obtained result was {1.11375, 0.504605, 1.10716}.

But, if my understanding is correct, the Laplacian distribution is not differentiable at "Mu" (the single peak of the distribution), which gives me trouble understanding the estimation results because MLE assumes the differentiability of the distribution function. So, my questions are:

(Q1) Are the above estimation results reliable (reasonable, or plausible)?

(Q2) If the answer to (Q1) is yes, what is the algorithm used in FindDistributionParameters dealing with the non-differentiability of the distribution function?

(Q3) If the answer to (Q1) is no, then, is there any reasonable way to come up with the parameter estimation for the above example? (In fact, I would like to obtain the standard error of each parameter in a reasonable way too.)

Thank you very much for your help in advance!

Improve this question
$\endgroup$
  • $\begingroup$ I don't know how Mathematica does this, but I suspect that for most built-in distributions it already knows the MLE of different parameters. You will find a derivation for this in various statistics books. If you plug in the explicit PDF instead of using a built-in distribution, it will resolve to numerical maximization which will be much slower. $\endgroup$ – Szabolcs Jun 15 '14 at 15:29
  • $\begingroup$ @David: Thank you very much for editing. I highly appreciate it! $\endgroup$ – user14070 Jun 16 '14 at 16:20
  • $\begingroup$ @Szabolcs: Thank you very much for your comment. Ok, I take your suggestion seriously. After reading a textbook of mathematical statistics, I now understand how to come up with the mean (= median) and the variance in the current case. But I have a trouble finding the standard errors for the parameters. In fact, I will post a question about it sometime later. Anyway, thank you very much again! $\endgroup$ – user14070 Jun 16 '14 at 16:20
  • $\begingroup$ How to find the errors is a very good question. There doesn't seem to be a built-in direct way to do it. Please see here. $\endgroup$ – Szabolcs Jun 16 '14 at 16:44
  • $\begingroup$ Maximum likelihood estimation does not require differentiability to obtain maximum likelihood estimates of the parameters. However, lack of differentiability does affect how one might go about estimating the variance of a maximum likelihood estimate. In such cases, the bootstrap (parametric or nonparametric) is likely your best bet. $\endgroup$ – JimB Jan 14 '17 at 0:50
2
$\begingroup$ 
dist = ExponentialPowerDistribution[1, 0.5, 1];

data = RandomVariate[dist, 1000];

param = FindDistributionParameters[data, 
  ExponentialPowerDistribution[\[Kappa], \[Mu], \[Sigma]]]

{\[Kappa] -> 1.01183, \[Mu] -> 0.45847, \[Sigma] -> 1.02004}

distEst = ExponentialPowerDistribution[
    \[Kappa], \[Mu], \[Sigma]] /. param;

Alternatively, the estimated distribution can be found using EstimatedDistribution

distEst === 
 EstimatedDistribution[data, 
  ExponentialPowerDistribution[\[Kappa], \[Mu], \[Sigma]]]

True

Show[
 Histogram[data, Automatic, "PDF"],
 Plot[Evaluate[PDF[#, x] & /@ {dist, distEst}], {x, Min[data], Max[data]},
  PlotStyle -> {Directive[Blue, AbsoluteDashing[{5, 3}]], 
    Directive[Red, AbsoluteDashing[{7, 3}]]}],
 PlotRange -> All]

enter image description here

The maximum likelihood method attempts to maximize the log-likelihood function so it is the log-likelihood function rather than the PDF being differentiated with respect to the parameters.

Improve this answer
$\endgroup$
  • $\begingroup$ Thank you very much for your answer. I highly appreciate it. In fact, I have a trouble finding the standard errors for the estimated parameters in the current case. So, I will post a question about it sometime later. Anyway, thank you very much again! $\endgroup$ – user14070 Jun 16 '14 at 16:24
  • $\begingroup$ @user14070 see documentation for DistributionFitTest at reference.wolfram.com/mathematica/ref/DistributionFitTest.html $\endgroup$ – Bob Hanlon Jun 16 '14 at 17:02
1
$\begingroup$ 
data = RandomVariate[ExponentialPowerDistribution[1, 0.5, 1], 1000];

ListLinePlot@data

enter image description here

exp = FindDistributionParameters[data, ExponentialPowerDistribution[Kappa, Mu, Sigma]];

{Kappa -> 0.998864, Mu -> 0.511828, Sigma -> 0.988751}

pdf = PDF[ExponentialPowerDistribution[Kappa, Mu, Sigma] /. exp, x]*400;

hist = Histogram@data;

plot = Plot[pdf, {x, -5, 5}, PlotRange -> All, PlotStyle -> Directive[Red, Thick]];

Show[hist, plot]

enter image description here

Your Q1: The result is reliable (see plot)

Your Q2: Read the excellent documentation for FindDistributionParameters

ADDENDUM

Show[Plot[pdf, {x, -6, 6}, PlotRange -> All, PlotStyle -> Red], SmoothHistogram@data]

enter image description here

Improve this answer
$\endgroup$
  • $\begingroup$ Thank you very much for your answer. I highly appreciate it. In fact, I have a trouble finding the standard errors for the estimated parameters in the current case. So, I will post a question about it sometime later. Anyway, thank you very much again! $\endgroup$ – user14070 Jun 16 '14 at 16:25

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.