I want to evaluate the differential entropy according to [here][1]


  [1]: https://en.wikipedia.org/wiki/Differential_entropy

$$h(X) = \int f(x) \log{f(x)} dx$$
where $f$ is the probability density function.

Lets create some test data (normal distribution):

    data1D = RandomVariate[NormalDistribution[], 50000];

Create histograms from the data using 2 different methods:

    smoothDistribution1D = SmoothKernelDistribution[data1D];
    distribution1D = HistogramDistribution[data1D];

Test if the histograms are normalised:

    Integrate[
      PDF[smoothDistribution1D, x],
      {x, -∞, ∞}
    ]
    Integrate[
      PDF[distribution1D, x],
      {x, -∞, ∞}
    ]
The above integrals do indeed return the value of 1.

Now lets try to evaluate the differential entropy:

    Integrate[
      PDF[smoothDistribution1D, x] Log[PDF[smoothDistribution1D, x]],
      {x, -∞, ∞}
    ]
    Integrate[
      PDF[distribution1D, x] Log[PDF[smoothDistribution1D, x]],
      {x, -∞, ∞}
    ]
It seems like both Integrate and NIntegrate cannot evaluate.

For such distribution, it is possible to calculate the differential entropy analytically.

    normalDistribution[x_, σ_, μ_] := 
    1/(σ Sqrt[2 π]) Exp[-((x - μ)^2/(2 σ^2))]

    Integrate[
      normalDistribution[x, σ, μ]*
      Log[normalDistribution[x, σ, μ]],
      {x, -∞, ∞},
      Assumptions -> Element[{σ, μ}, Reals] && σ > 0
    ]

$$-\frac{1}{2} \left( 1 +  \log{(2 \pi \sigma^2)} \right)$$