I know the definition of entropy of a probability distribution:
$$H = - \sum_i p_i \log p_i $$
So for example, in a Bernoulli distribution with $p = 0.2$, $1-p=0.8$, the entropy is $0.5$. However, in Mathematica
Log. So either the Mathematica has a bug, or I don't understand what it is that
Entropy[...] calculates in Mathematica. Can someone clarify this for me?