1
$\begingroup$

This is a really dumb question, but here goes: is there a way to compute cross-entropy of two probability distributions? There is CrossEntropyLayer[], so Mathematica knows about cross-entropy, but no actual function, it seems, unless I am missing something obvious...

Improve this question
$\endgroup$
4
  • 2
    $\begingroup$ There doesn't seem to be one, but you can always use Expectation[]... $\endgroup$
    – J. M.'s missing motivation ♦
    Commented May 17, 2017 at 18:31
  • $\begingroup$ @J.M. So, for symbolic dist, Expectation[Log[PDF[...], ...]? $\endgroup$
    – Igor Rivin
    Commented May 17, 2017 at 18:44
  • $\begingroup$ Like that, yes. The bonus is that Expectation[] will automatically handle it depending on whether both distributions are discrete or continuous. $\endgroup$
    – J. M.'s missing motivation ♦
    Commented May 17, 2017 at 18:47
  • 2
    $\begingroup$ Also, for the case that this can't be computed symbolically, there's NExpectation $\endgroup$
    – yohbs
    Commented May 17, 2017 at 19:29

0

Reset to default

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.