# Best way to include a "singularity" in a SmoothHistogram?

I have a dataset like this:

data = {0,0,1.1,1.3,0,0,3.4,0,0,2.3,0,0 .....}


You can imagine that data is generated from a probability distribution of the form:

$$p(x)=w_0\delta(x) + (1 - w_0)f(x)$$

where $f(x)$ is a smooth probability density function, $\delta$ is Dirac-delta function, and $0\le w_0 \le 1$.

That is, there is a finite probability that $x=0$.

I want to plot an histogram of data. If I exclude 0, then a SmoothHistogram is fine. But now I want to include also in this plot the frequency of 0. In this case, SmoothHistogram performs poorly, since it tries to draw a smooth peak centered at 0.

Can you suggest a better way to visualize data? Note that I know that the location of the singularity is at 0.

• might data look something like this Join[RandomVariate[NormalDistribution[2, 1], 2000], ConstantArray[0, {100}]] ? Commented Mar 24, 2016 at 18:26
• @george2079 Try putting the 0's in random positions. Other than that, yes.
– a06e
Commented Mar 24, 2016 at 18:29
• First you might consider rewriting the density to $p(x)=w_0 \delta(x)+(1-w_0)f(x)$. That makes $f$ a legitimate probability density function. Second, why not just display a legend that states $X$ takes the value zero with probability $w_0$ otherwise $X$ takes the random value from the distribution $f(x)$ described by the smooth histogram? If you want to display a density, then as stated below by @george2079, there really is no appropriate scaling if a smooth histogram is displayed. Alternatively, you could plot the cumulative distribution function which would have a jump at zero.
– JimB
Commented Mar 24, 2016 at 20:21
• @JimBaldwin The main issue is that I am comparing different datasets (which probably have different $w_0$'s). If I just do a SmoothHistogram of the datasets removing 0, I don't get a plot that reflects that there may be different values of $w_0$ involved, since all the SmoothHistograms will get normalized to 1, instead of to $1-w_0$ (I rewrote the density as you suggested, thanks).
– a06e
Commented Mar 24, 2016 at 20:25
• As mentioned below as long as the bar widths are constant and the area of the bar at zero is the estimate of $w_0$, then you'd be imparting information in a consistent fashion. And because it is the density/probability mass function $p$ that you want to describe, the area under $f$ should be $1-w_0$ - so just multiply the resulting density values for $f$ by $1-w_0$.
– JimB
Commented Mar 24, 2016 at 20:36

I guess you just do something like this:

data = RandomSample[Join[RandomVariate[NormalDistribution[2, 1], 2000],
ConstantArray[0, {100}]]];
Histogram[data]


SmoothHistogram[Select[data, # != 0 &],
Epilog -> Line[{{0, 0}, {0, .15}}], AxesOrigin -> {-2, 0}]


I cant see a sensible way to scale that line (It really goes to Infinity).

also the overall smooth plot ought to be scaled down somehow something like this should do:

 SmoothHistogram[...]/.Line[x_?(Length[#] > 3 &)] :>
Line[{1, Count[data, 0]/Length@data} # & /@ x]


In this case the scale change is so small its not worth showing another plot.

Edit:

here is an idea, pick a bin width around your singularity, then you can assign an appropriate height to a bar on the chart:

bin = .1;
inbin = Select[data, Abs[#] < bin &] // Length;
hbin = inbin/Length[data]/(2 bin) // N
SmoothHistogram[Select[data, Abs[#] > bin &],
Epilog -> Rectangle[{-bin, 0}, {bin, hbin}],
AxesOrigin -> {-2, 0}] /.
Line[x_?(Length[#] > 3 &)] :>
Line[{1, 1 - inbin/Length@data} # & /@ x]


note this counts all the data in the bin not just the exact zeros. This last chart looks a little different because its using a new random data set by the way.

• I suspect the SmoothedHistogram will always be constructed such that the density under the curve is 1 (even if we remove the 0 data from the curve). If so, one would need to manually 'scale down' the Smootherhistrogram plot by whatever proportion of the data are excluded (the zeroes) Commented Mar 24, 2016 at 19:28
• Could you use WeightedData[] as an input to SmoothHistogram[] to weight according to if it is zero or not, then add back in the line? Commented Mar 24, 2016 at 19:48
• I don't think the line should go to infinity. The height of the line (maybe it is more stylish to use a bar?) should represent the weight $w_0$ of $0$, or the frequency of $0$'s in the dataset.
– a06e
Commented Mar 24, 2016 at 20:12
• @becko: To make any sense the area of the bar at zero would have to be $w_0$. And because the width is completely arbitrary (and "technically" should have a width of zero), I don't see that any appropriate knowledge/information would be forthcoming unless...there were multiple figures and an arbitrary bar width would be consistent among the figures.
– JimB
Commented Mar 24, 2016 at 20:25
• You estimate $w_0$ from the data, say by counting the 0's. The width is of course arbitrary, it's just visually more appealing, I think.
– a06e
Commented Mar 24, 2016 at 20:27

This is what we get playing with the parameters of SmoothHistogram and SmoothKernelDistribution:

data = RandomSample[Join[RandomVariate[NormalDistribution[2, 1], 2000],
ConstantArray[0, {100}]]];
Histogram[data]


SmoothHistogram[data, {"Adaptive", .1, .001}]


SmoothHistogram[data, {"Adaptive", .05, .01}]


dist = SmoothKernelDistribution[data, {"Adaptive", .1, .001}];

Plot[PDF[dist, x], {x, -4, 4}]