This is a really dumb question, but here goes: is there a way to compute cross-entropy of two probability distributions? There is CrossEntropyLayer[], so Mathematica knows about cross-entropy, but no actual function, it seems, unless I am missing something obvious...

  • 2
    $\begingroup$ There doesn't seem to be one, but you can always use Expectation[]... $\endgroup$ – J. M.'s technical difficulties May 17 '17 at 18:31
  • $\begingroup$ @J.M. So, for symbolic dist, Expectation[Log[PDF[...], ...]? $\endgroup$ – Igor Rivin May 17 '17 at 18:44
  • $\begingroup$ Like that, yes. The bonus is that Expectation[] will automatically handle it depending on whether both distributions are discrete or continuous. $\endgroup$ – J. M.'s technical difficulties May 17 '17 at 18:47
  • 2
    $\begingroup$ Also, for the case that this can't be computed symbolically, there's NExpectation $\endgroup$ – yohbs May 17 '17 at 19:29

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Browse other questions tagged or ask your own question.