# How to evaluate differential entropy from raw data?

I want to evaluate the differential entropy according to here

$$h(X) = - \int f(x) \log{f(x)} dx$$ where $f$ is the probability density function.

Lets create some test data (normal distribution):

data1D = RandomVariate[NormalDistribution[], 50000];


Create histograms from the data using 2 different methods:

smoothDistribution1D = SmoothKernelDistribution[data1D];
distribution1D = HistogramDistribution[data1D];


Test if the histograms are normalised:

Integrate[
PDF[smoothDistribution1D, x],
{x, -∞, ∞}
]
Integrate[
PDF[distribution1D, x],
{x, -∞, ∞}
]


The above integrals do indeed return the value of 1.

Now lets try to evaluate the differential entropy:

Integrate[
PDF[smoothDistribution1D, x] Log[PDF[smoothDistribution1D, x]],
{x, -∞, ∞}
]
Integrate[
PDF[distribution1D, x] Log[PDF[smoothDistribution1D, x]],
{x, -∞, ∞}
]


It seems like both Integrate and NIntegrate cannot evaluate.

For such distribution, it is possible to calculate the differential entropy analytically.

normalDistribution[x_, σ_, μ_] :=
1/(σ Sqrt[2 π]) Exp[-((x - μ)^2/(2 σ^2))]

Integrate[
normalDistribution[x, σ, μ]*
Log[normalDistribution[x, σ, μ]],
{x, -∞, ∞},
Assumptions -> Element[{σ, μ}, Reals] && σ > 0
]


$$-\frac{1}{2} \left( 1 + \log{(2 \pi \sigma^2)} \right)$$

The error I get is that the integrals do not converge. I expect that this is because the empirical PDFs are zero far from the data, and Mathematica does not automatically take $0\log 0$ to be $0$. You can do it explicitly as follows:
NIntegrate[With[{f = PDF[smoothDistribution1D, x]}, If[f > 0, f Log[f], 0]], {x, -∞, ∞}]

Compared to the analytical solution, $-\frac12 (\log 2\pi + 1)\approx-1.41894$, it's not too far off.