In the following a,b is a probability distribution, i.e. a+b==1 with a,b real, and in the interval [0,1]. If we try to maximize the Shannon entropy over this distribution using Mathematica, we can either set b=1-a, or state the total probability condition as an assumptions.

Maximizing using b=1-a works as expected. This input Maximize[{-Log[2, a]*a - Log[2, 1 - a]*(1 - a), a <= 1 && a >= 0}, a, Reals] yields {1, {a -> 1/2}}.

However, when we give Maximize[{-Log[2, a]*a - Log[2, b]*b, a + b == 1 && a >= 0 && a <= 1 && b >= 0 && b <= 1}, {a, b}, Reals] as input, Mathematica does not find the solution.

Any suggestions how the later input can be altered to find the correct solution?


The approach you tried actually works perfectly. However, Mathematica decided to keep the result in symbolic form because the input involved only exact numbers and symbols. You just have to tell it that you'd like a number:

 -Log[2, a]*a - Log[2, b]*b, 
 a + b == 1 && a >= 0 && a <= 1 && b >= 0 && b <= 1}, 
 {a, b}, Reals]//N

I only appended the //N to get the desired result.

An alternative would be to make one of the numbers in your input a real number, e.g., replacing 1 by 1. in the constraint.

As an aside, you could simplify the statement a little:

  -Log[2, b] b - Log[2, a] a, 
  (a + b == 1) && (0 <= a <= 1) && (0 <= b <= 1)}, 
  {a, b}] // N


Because Mathematica refuses to give a result unless we add the //N, you might as well replace Maximize by either NMaximize or FindMaximum.

Obviously in your first example you get the exact input to evaluate fully with Maximize, and the unevaluated result in the second example looks somewhat inconsistent with the first result. But this does happen occasionally. A simple analogy might be Sin[Pi/4], which returns $1/\sqrt{2}$, whereas Sin[1] remains unevaluated until you operate on it with N.

  • $\begingroup$ If one doesn't need exact results, one could use NMaximize[] to begin with... $\endgroup$
    – J. M.'s torpor
    May 16 '12 at 0:45
  • $\begingroup$ @J.M. You wrote this at the same time I was editing my answer to that effect... $\endgroup$
    – Jens
    May 16 '12 at 0:50

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.