Assumptions seem not to be considered

Question

In the following a,b is a probability distribution, i.e. a+b==1 with a,b real, and in the interval [0,1]. If we try to maximize the Shannon entropy over this distribution using Mathematica, we can either set b=1-a, or state the total probability condition as an assumptions.

Maximizing using b=1-a works as expected. This input Maximize[{-Log[2, a]*a - Log[2, 1 - a]*(1 - a), a <= 1 && a >= 0}, a, Reals] yields {1, {a -> 1/2}}.

However, when we give Maximize[{-Log[2, a]*a - Log[2, b]*b, a + b == 1 && a >= 0 && a <= 1 && b >= 0 && b <= 1}, {a, b}, Reals] as input, Mathematica does not find the solution.

Any suggestions how the later input can be altered to find the correct solution?

Answer 1

The approach you tried actually works perfectly. However, Mathematica decided to keep the result in symbolic form because the input involved only exact numbers and symbols. You just have to tell it that you'd like a number:

Maximize[{
 -Log[2, a]*a - Log[2, b]*b, 
 a + b == 1 && a >= 0 && a <= 1 && b >= 0 && b <= 1}, 
 {a, b}, Reals]//N

I only appended the //N to get the desired result.

An alternative would be to make one of the numbers in your input a real number, e.g., replacing 1 by 1. in the constraint.

As an aside, you could simplify the statement a little:

Maximize[{
  -Log[2, b] b - Log[2, a] a, 
  (a + b == 1) && (0 <= a <= 1) && (0 <= b <= 1)}, 
  {a, b}] // N

Edit

Because Mathematica refuses to give a result unless we add the //N, you might as well replace Maximize by either NMaximize or FindMaximum.

Obviously in your first example you get the exact input to evaluate fully with Maximize, and the unevaluated result in the second example looks somewhat inconsistent with the first result. But this does happen occasionally. A simple analogy might be Sin[Pi/4], which returns $1/\sqrt{2}$, whereas Sin[1] remains unevaluated until you operate on it with N.