1
$\begingroup$

I have a set of data that is already ordered as a distribution. Here is a picture worth 999 words:

enter image description here

I tried FindDistributionParameters, and EstimatedDistribution and I keep getting the message "One or more data points are not in support of the process...".

Any advise? Thank you.

P.S.: Here is the data. Note, the max range of the data is not known, so it could be from {0,inf} - but the range of the data shown is [0,10].

0.0258899, 0.047329, 0.0683456, 0.0861312, 0.103529, 0.118554, \
0.124337, 0.127011, 0.131217, 0.134408, 0.132016, 0.130485, 0.127444, \
0.122495, 0.117607, 0.112261, 0.108703, 0.10322, 0.100889, 0.0955182, \
0.0902072, 0.085152, 0.079376, 0.0752159, 0.0722947, 0.0686795, \
0.0653517, 0.062192, 0.0598642, 0.0569383, 0.0548917, 0.0513092, \
0.0485989, 0.0472422, 0.0458829, 0.0434763, 0.0406302, 0.03872, \
0.0371639, 0.0370529, 0.0353547, 0.0332693, 0.0320803, 0.0302454, \
0.0300313, 0.0294249, 0.0281058, 0.0271762, 0.0256051, 0.0257485, \
0.0250776, 0.0237297, 0.0227674, 0.0225286, 0.0221572, 0.0213558, \
0.0206458, 0.0199427, 0.0191173, 0.0172471, 0.017145, 0.0170844, \
0.016805, 0.0162183, 0.0150205, 0.0149596, 0.0148139, 0.0142276, \
0.0141969, 0.0139716, 0.0132321, 0.0128165, 0.0129189, 0.0126378, \
0.0123681, 0.0122255, 0.0121722, 0.0117113, 0.010789, 0.0105248, \
0.0100393, 0.0103667, 0.0102826, 0.0102321, 0.0104435, 0.0100861, \
0.00972111, 0.00957839, 0.00966471, 0.00963415, 0.00880081, \
0.00873474, 0.00859673, 0.00816511, 0.00792854, 0.0075657, \
0.00747507, 0.00734244, 0.00729432}
|improve this question
$\endgroup$
  • 2
    $\begingroup$ Besides removing the zero ... Can you include a link of all the data because I was able to use FindDistributionParameters on the selection you provided $\endgroup$ – Young Jul 18 '16 at 2:24
  • $\begingroup$ @Young Thanks, that was not supposed to be a zero but I guess it was being truncated. Now the two functions work but the parameters I get back they don't give me anything similar to the data I have. $\endgroup$ – Edv Beq Jul 18 '16 at 2:24
  • 2
    $\begingroup$ What distribution are you using as the model to match? As stated by @Young you need to post a link to your data since the sample data that you provided is not a good representation of the data shown in your plot. $\endgroup$ – Bob Hanlon Jul 18 '16 at 2:52
  • 1
    $\begingroup$ It does not appear that your data is from a random sample from any particular distribution but rather you have "measurements" that have the shape of a probability distribution. Another indication that you don't have a real density function is that the area under the curve is around 0.43 and not 1.0. If so, you should avoid any maximum likelihood estimation or any other estimation technique that assumes you have a random sample. Is there some reason why you think it a probability density function is what you need? $\endgroup$ – JimB Jul 18 '16 at 3:20
  • 1
    $\begingroup$ @JimBaldwin I dont know what the t-max is so now I realize my info above is a little misleading. I know its a probability distribution because I was simulating the Faynman-Kac fomrula which gives distribution of certain Wiener functionals. $\endgroup$ – Edv Beq Jul 18 '16 at 3:30
6
$\begingroup$

You don't seem to have a random sample from any probability distribution but rather have measurements/observations that might have a similar functional form as a probability density function.

Your curve has an area of about 0.43 going from 0 to 10. While the tail of the distribution you envision might be heavy enough to hold the remaining 0.57 of a probability distribution, I don't believe it.

So as @Young suggested, you can certainly fit a curve that happens to have a similar form as a probability distribution (but you can't pretend you're fitting a probability distribution).

To fit such a curve I'm assuming that the data presented are the "y-values" and that the "x-values" are uniformly spaced from 0.1 to 9.9. So the first step is to make a data set that reflects that:

data = {0.0258899, 0.047329, 0.0683456, 0.0861312, 0.103529, 0.118554,
    0.124337, 0.127011, 0.131217, 0.134408, 0.132016, 0.130485, 
   0.127444, 0.122495, 0.117607, 0.112261, 0.108703, 0.10322, 
   0.100889, 0.0955182, 0.0902072, 0.085152, 0.079376, 0.0752159, 
   0.0722947, 0.0686795, 0.0653517, 0.062192, 0.0598642, 0.0569383, 
   0.0548917, 0.0513092, 0.0485989, 0.0472422, 0.0458829, 0.0434763, 
   0.0406302, 0.03872, 0.0371639, 0.0370529, 0.0353547, 0.0332693, 
   0.0320803, 0.0302454, 0.0300313, 0.0294249, 0.0281058, 0.0271762, 
   0.0256051, 0.0257485, 0.0250776, 0.0237297, 0.0227674, 0.0225286, 
   0.0221572, 0.0213558, 0.0206458, 0.0199427, 0.0191173, 0.0172471, 
   0.017145, 0.0170844, 0.016805, 0.0162183, 0.0150205, 0.0149596, 
   0.0148139, 0.0142276, 0.0141969, 0.0139716, 0.0132321, 0.0128165, 
   0.0129189, 0.0126378, 0.0123681, 0.0122255, 0.0121722, 0.0117113, 
   0.010789, 0.0105248, 0.0100393, 0.0103667, 0.0102826, 0.0102321, 
   0.0104435, 0.0100861, 0.00972111, 0.00957839, 0.00966471, 
   0.00963415, 0.00880081, 0.00873474, 0.00859673, 0.00816511, 
   0.00792854, 0.0075657, 0.00747507, 0.00734244, 0.00729432};
(* Include the "x" value *)
data = Table[{i/10, data[[i]]}, {i, Length[data]}];

Now try a curve that has the same shape as a log-normal density function:

nlm = NonlinearModelFit[data, 
   k Exp[-(Log[x] - m)^2/(2 s^2)]/x, {m, s, k}, x];
nlm["BestFitParameters"]
(* {m -> 0.795688, s -> 0.951072, k -> 0.186508} *)
Show[ListPlot[data], Plot[nlm[x], {x, 0, 10}]]

Data and fit

The fit is not great (in my mind) but you need to determine if it meets your needs.

|improve this answer
$\endgroup$
  • 3
    $\begingroup$ "...but you can't pretend you're fitting a probability distribution."... the salient point, +1 $\endgroup$ – ciao Jul 18 '16 at 6:16
  • $\begingroup$ I could be wrong but the literature says I should be getting back a probability density. en.wikipedia.org/wiki/Feynman%E2%80%93Kac_formula I was doing the example in the remarks with v(x)=0.25*x^2, and diffusion dx=dw. $\endgroup$ – Edv Beq Jul 18 '16 at 11:31
  • 2
    $\begingroup$ I'm not disagreeing with the literature (especially Feynman). But if you could show how you obtained even just one of the observations (and include what "x-value" is associated with that "observed probability density value"), that would allow someone to help. $\endgroup$ – JimB Jul 18 '16 at 12:45
1
$\begingroup$

I agree with Jim Baldwin, but here is the code I put together based on this data originating from a random sample (Just in case that helps you for some reason). Your original problem was the 0 data point causing the functions not to evaluate.

dist = {0.0258899, 0.047329, 0.0683456, 0.0861312, 0.103529, 0.118554,
    0.124337, 0.127011, 0.131217, 0.134408, 0.132016, 0.130485, 
   0.127444, 0.122495, 0.117607, 0.112261, 0.108703, 0.10322, 
   0.100889, 0.0955182, 0.0902072, 0.085152, 0.079376, 0.0752159, 
   0.0722947, 0.0686795, 0.0653517, 0.062192, 0.0598642, 0.0569383, 
   0.0548917, 0.0513092, 0.0485989, 0.0472422, 0.0458829, 0.0434763, 
   0.0406302, 0.03872, 0.0371639, 0.0370529, 0.0353547, 0.0332693, 
   0.0320803, 0.0302454, 0.0300313, 0.0294249, 0.0281058, 0.0271762, 
   0.0256051, 0.0257485, 0.0250776, 0.0237297, 0.0227674, 0.0225286, 
   0.0221572, 0.0213558, 0.0206458, 0.0199427, 0.0191173, 0.0172471, 
   0.017145, 0.0170844, 0.016805, 0.0162183, 0.0150205, 0.0149596, 
   0.0148139, 0.0142276, 0.0141969, 0.0139716, 0.0132321, 0.0128165, 
   0.0129189, 0.0126378, 0.0123681, 0.0122255, 0.0121722, 0.0117113, 
   0.010789, 0.0105248, 0.0100393, 0.0103667, 0.0102826, 0.0102321, 
   0.0104435, 0.0100861, 0.00972111, 0.00957839, 0.00966471, 
   0.00963415, 0.00880081, 0.00873474, 0.00859673, 0.00816511, 
   0.00792854, 0.0075657, 0.00747507, 0.00734244, 0.00729432};

fdist = FindDistribution[dist]
paramsA = FindDistributionParameters[dist, LogNormalDistribution[μ, σ]]
paramsB = FindDistributionParameters[dist, UniformDistribution[{min, max}]]

Plot[PDF[LogNormalDistribution[μ, σ] /. paramsA, {x}], {x, 0, 0.1}, 
     Filling -> Axis, PlotRange -> All]
Plot[PDF[UniformDistribution[{min, max}] /. paramsB, {x}], {x, 0, 0.1},
     Filling -> Axis, PlotRange -> All]
Plot[PDF[fdist, {x}], {x, 0, 0.1}, Filling -> Axis, PlotRange -> All]

MixtureDistribution[{0.747238, 0.252762}, {LogNormalDistribution[-3.99373, 0.633476], UniformDistribution[{0.00729432, 0.134408}]}]

enter image description here

|improve this answer
$\endgroup$
0
$\begingroup$ 
data = {0.0258899, 0.047329, 0.0683456, 0.0861312, 0.103529, 0.118554, 
   0.124337, 0.127011, 0.131217, 0.134408, 0.132016, 0.130485, 0.127444, 
   0.122495, 0.117607, 0.112261, 0.108703, 0.10322, 0.100889, 0.0955182, 
   0.0902072, 0.085152, 0.079376, 0.0752159, 0.0722947, 0.0686795, 0.0653517, 
   0.062192, 0.0598642, 0.0569383, 0.0548917, 0.0513092, 0.0485989, 0.0472422,
    0.0458829, 0.0434763, 0.0406302, 0.03872, 0.0371639, 0.0370529, 0.0353547,
    0.0332693, 0.0320803, 0.0302454, 0.0300313, 0.0294249, 0.0281058, 
   0.0271762, 0.0256051, 0.0257485, 0.0250776, 0.0237297, 0.0227674, 
   0.0225286, 0.0221572, 0.0213558, 0.0206458, 0.0199427, 0.0191173, 
   0.0172471, 0.017145, 0.0170844, 0.016805, 0.0162183, 0.0150205, 0.0149596, 
   0.0148139, 0.0142276, 0.0141969, 0.0139716, 0.0132321, 0.0128165, 
   0.0129189, 0.0126378, 0.0123681, 0.0122255, 0.0121722, 0.0117113, 0.010789,
    0.0105248, 0.0100393, 0.0103667, 0.0102826, 0.0102321, 0.0104435, 
   0.0100861, 0.00972111, 0.00957839, 0.00966471, 0.00963415, 0.00880081, 
   0.00873474, 0.00859673, 0.00816511, 0.00792854, 0.0075657, 0.00747507, 
   0.00734244, 0.00729432};

Try a mix of distributions

distModel = MixtureDistribution[{w, 1 - w},
   {LogNormalDistribution[m, s1],
    RayleighDistribution[s2]}];

dist = distModel /.
   FindDistributionParameters[data,
    distModel];

Integrate[PDF[dist, x], {x, 0, Infinity}]

(*  1.  *)

Show[
 Histogram[data, 11, "PDF"],
 Plot[PDF[dist, x], {x, 0, Max[data]}],
 PlotRange -> All]

enter image description here

|improve this answer
$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.