2
$\begingroup$

I use SmoothKernelDistribution (with no options and a single argument — a list of values) to get an approximate reconstruction of a continuous distribution based on a finite sample of values. It is known a priori that the actual distribution is unimodal and its PDF is exactly 0 for x ≤ 0.

Can I provide some options to SmoothKernelDistribution so that it tries to find an approximate distribution within these constraints?

On the plot below horizontal positions of the vertical gray lines represent the sample values, the blue curve is an approximate distribution returned by SmoothKernelDistribution, and the orange curve conveys a general idea how the actual distribution may look like; its exact shape, kurtosis, height and position of the peak may vary significantly, so I cannot just fit some known parameterized distribution to the sample using EstimatedDistribution.

distribution plot

$\endgroup$
4
  • 2
    $\begingroup$ maybe soemthing like dist = SmoothKernelDistribution[data, Automatic, {"Bounded", {0, 1}, "Gaussian"}]? $\endgroup$
    – kglr
    Commented May 28, 2020 at 20:22
  • $\begingroup$ A sample of your data would help. $\endgroup$
    – MarcoB
    Commented May 28, 2020 at 20:32
  • 3
    $\begingroup$ "Data don't lie" and "In theory there is no difference between theory and practice. In practice there is." It sure looks like the distribution has two bumps despite knowing a priori that it shouldn't. $\endgroup$
    – JimB
    Commented May 28, 2020 at 21:05
  • 1
    $\begingroup$ A more constructive comment: To force the pdf to be 0 at 0 there is the negative reflection technique in the following paper: ned.ipac.caltech.edu/level5/March02/Silverman/paper.pdf (Equation 2.16, next to last page). I'll see if I can cook up an example. $\endgroup$
    – JimB
    Commented May 28, 2020 at 21:23

2 Answers 2

8
$\begingroup$

The "Bounded" option of SmoothKernelDistribution (which uses a reflection of the data technique) usually ends up with a non-zero value for the pdf at zero (which is many times exactly what you want). But when you need the pdf of zero to be zero a "negative reflection technique" is one possible approach. (Silverman, 1986)

Fortunately, it's easily programmed. Using @BobHanlon 's example:

SeedRandom[1234]
data = RandomVariate[LogNormalDistribution[1, 1], 1000];

(* Using original data *)
skdPlus = SmoothKernelDistribution[data];
(* Using the reflected data *)
skdMinus = SmoothKernelDistribution[-data];

Plot[{PDF[LogNormalDistribution[1, 1], x],
  PDF[skdPlus, x] - PDF[skdMinus, x]}, {x, 0, 20}, PlotRange -> All, 
 PlotLegends -> {"True", "Estimated"}]

True and estimated pdf

Addition:

When there is a pronounced peak, then using an "Adaptive" bandwidth technique can work out better. (This is where the bandwidth used is narrower where the data is denser and wider out in the tails.)

SeedRandom[1234]
data = RandomVariate[LogNormalDistribution[1, 1], 10000];

(* Using original data *)
skdPlus = SmoothKernelDistribution[data, {"Adaptive", 1, Automatic}];
(* Using the reflected data *)
skdMinus = SmoothKernelDistribution[-data, {"Adaptive", 1, Automatic}];

Plot[{PDF[LogNormalDistribution[1, 1], x],
  PDF[skdPlus, x] - PDF[skdMinus, x]}, {x, 0, 20}, PlotRange -> All, 
 PlotLegends -> {"True", "Estimated"}]

True and estimated using adaptive bandwidth

Now, I've purposely chosen 1 as the bandwidth to make it look good. (Shame on me.) Here's what you get with "Automatic":

True and estimated with adaptive and automatically chosen bandwidth

$\endgroup$
4
$\begingroup$
SeedRandom[1234]

data = RandomVariate[LogNormalDistribution[1, 1], 1000];

From the documentation, "The kernel function ker can be specified to account for known bounding on the underlying density using {"Bounded", c, ker}, where c can be any real number, a list {Subscript[c, 1],Subscript[c, 2]} such that Subscript[c, 1] < Subscript[c, 2], or a list {{Subscript[c, 11], Subscript[c, 12]}, {Subscript[c, 21], Subscript[c, 22]},…}, with length equal to the dimension of data."

Show[
 Histogram[data, Automatic, "PDF"],
 Plot[
  PDF[SmoothKernelDistribution[data, 
    Automatic, {"Bounded", {0, 20}, "Gaussian"}], x],
  {x, -1, 20}, PlotStyle -> {{Thick, Red}}],
 PlotRange -> {{0, 20}, All}]

enter image description here

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.