1
$\begingroup$

I am trying to find a reciprocal function to the binary shannon entropy $H_2(p)$ (chosing the branch were $p>1/2$).

I used a method based on solving the differential equation that the reciprocal follows but it was a little too long for what i need to do.

Thus, I want to do a polynomial approximation of the shannon entropy and then find the reciprocal of this polynomial function.

But mathematica answers me things I don't understand.

Here is my code :

H2[x_] := 
  If[x != 0, If[x != 1, (-x)*Log[x] - (1 - x)*Log[1 - x], 0], 0]; 

tableauValeursH2 = Table[{x, H2[x]}, {x, 1/2, 1, 0.001}]; 

ff[y_] = Fit[tableauValeursH2, Table[y^k, {k, 0, 10}], y]


Solve[y == ff[x], x]

And mathematica answers me :

{{x -> Root[
    502925249036334454715336778553984003440864263790494070 + 
      79507334695638259109060376450677753198917644793560 y - 
      7228015973937440731974941433834202407604387494117592200 #1 + 
      46469470965681832229663230673595151067237874818752782840 #1^2 - 
      176009396354156927628824727628578542386502238219708082250 #1^3 \
+ 434953160110610020035094377442281900757943807833308766200 #1^4 - 
      732785165581324716157030456066865775762616415498186198190 #1^5 \
+ 852422521044952102836379759843676923790865671573802051255 #1^6 - 
      676099528944398287173100625127955582322701524364225111800 #1^7 \
+ 349947592110433543619114105190630648009305990596089919920 #1^8 - 
      106747371466686462957887174434503958626612109573992670672 #1^9 \
+ 14573808550521644768419974300540238469473534471735953168 #1^10 &, 
    1]}, {x -> 
   Root[502925249036334454715336778553984003440864263790494070 + 
      79507334695638259109060376450677753198917644793560 y - 
      7228015973937440731974941433834202407604387494117592200 #1 + 
      46469470965681832229663230673595151067237874818752782840 #1^2 - 
      176009396354156927628824727628578542386502238219708082250 #1^3 \
+ 434953160110610020035094377442281900757943807833308766200 #1^4 - 
      732785165581324716157030456066865775762616415498186198190 #1^5 \
+ 852422521044952102836379759843676923790865671573802051255 #1^6 - 
      676099528944398287173100625127955582322701524364225111800 #1^7 \
+ 349947592110433543619114105190630648009305990596089919920 #1^8 - 
      106747371466686462957887174434503958626612109573992670672 #1^9 \
+ 14573808550521644768419974300540238469473534471735953168 #1^10 &, 
    2]}, {x -> 
   Root[502925249036334454715336778553984003440864263790494070 + 
      79507334695638259109060376450677753198917644793560 y - 
      7228015973937440731974941433834202407604387494117592200 #1 + 
      46469470965681832229663230673595151067237874818752782840 #1^2 - 
      176009396354156927628824727628578542386502238219708082250 #1^3 \
+ 434953160110610020035094377442281900757943807833308766200 #1^4 - 
      732785165581324716157030456066865775762616415498186198190 #1^5 \
+ 852422521044952102836379759843676923790865671573802051255 #1^6 - 
      676099528944398287173100625127955582322701524364225111800 #1^7 \
+ 349947592110433543619114105190630648009305990596089919920 #1^8 - 
      106747371466686462957887174434503958626612109573992670672 #1^9 \
+ 14573808550521644768419974300540238469473534471735953168 #1^10 &, 
    3]}, {x -> 
   Root[502925249036334454715336778553984003440864263790494070 + 
      79507334695638259109060376450677753198917644793560 y - 
      7228015973937440731974941433834202407604387494117592200 #1 + 
      46469470965681832229663230673595151067237874818752782840 #1^2 - 
      176009396354156927628824727628578542386502238219708082250 #1^3 \
+ 434953160110610020035094377442281900757943807833308766200 #1^4 - 
      732785165581324716157030456066865775762616415498186198190 #1^5 \
+ 852422521044952102836379759843676923790865671573802051255 #1^6 - 
      676099528944398287173100625127955582322701524364225111800 #1^7 \
+ 349947592110433543619114105190630648009305990596089919920 #1^8 - 
      106747371466686462957887174434503958626612109573992670672 #1^9 \
+ 14573808550521644768419974300540238469473534471735953168 #1^10 &, 
    4]}, {x -> 
   Root[502925249036334454715336778553984003440864263790494070 + 
      79507334695638259109060376450677753198917644793560 y - 
      7228015973937440731974941433834202407604387494117592200 #1 + 
      46469470965681832229663230673595151067237874818752782840 #1^2 - 
      176009396354156927628824727628578542386502238219708082250 #1^3 \
+ 434953160110610020035094377442281900757943807833308766200 #1^4 - 
      732785165581324716157030456066865775762616415498186198190 #1^5 \
+ 852422521044952102836379759843676923790865671573802051255 #1^6 - 
      676099528944398287173100625127955582322701524364225111800 #1^7 \
+ 349947592110433543619114105190630648009305990596089919920 #1^8 - 
      106747371466686462957887174434503958626612109573992670672 #1^9 \
+ 14573808550521644768419974300540238469473534471735953168 #1^10 &, 
    5]}, {x -> 
   Root[502925249036334454715336778553984003440864263790494070 + 
      79507334695638259109060376450677753198917644793560 y - 
      7228015973937440731974941433834202407604387494117592200 #1 + 
      46469470965681832229663230673595151067237874818752782840 #1^2 - 
      176009396354156927628824727628578542386502238219708082250 #1^3 \
+ 434953160110610020035094377442281900757943807833308766200 #1^4 - 
      732785165581324716157030456066865775762616415498186198190 #1^5 \
+ 852422521044952102836379759843676923790865671573802051255 #1^6 - 
      676099528944398287173100625127955582322701524364225111800 #1^7 \
+ 349947592110433543619114105190630648009305990596089919920 #1^8 - 
      106747371466686462957887174434503958626612109573992670672 #1^9 \
+ 14573808550521644768419974300540238469473534471735953168 #1^10 &, 
    6]}, {x -> 
   Root[502925249036334454715336778553984003440864263790494070 + 
      79507334695638259109060376450677753198917644793560 y - 
      7228015973937440731974941433834202407604387494117592200 #1 + 
      46469470965681832229663230673595151067237874818752782840 #1^2 - 
      176009396354156927628824727628578542386502238219708082250 #1^3 \
+ 434953160110610020035094377442281900757943807833308766200 #1^4 - 
      732785165581324716157030456066865775762616415498186198190 #1^5 \
+ 852422521044952102836379759843676923790865671573802051255 #1^6 - 
      676099528944398287173100625127955582322701524364225111800 #1^7 \
+ 349947592110433543619114105190630648009305990596089919920 #1^8 - 
      106747371466686462957887174434503958626612109573992670672 #1^9 \
+ 14573808550521644768419974300540238469473534471735953168 #1^10 &, 
    7]}, {x -> 
   Root[502925249036334454715336778553984003440864263790494070 + 
      79507334695638259109060376450677753198917644793560 y - 
      7228015973937440731974941433834202407604387494117592200 #1 + 
      46469470965681832229663230673595151067237874818752782840 #1^2 - 
      176009396354156927628824727628578542386502238219708082250 #1^3 \
+ 434953160110610020035094377442281900757943807833308766200 #1^4 - 
      732785165581324716157030456066865775762616415498186198190 #1^5 \
+ 852422521044952102836379759843676923790865671573802051255 #1^6 - 
      676099528944398287173100625127955582322701524364225111800 #1^7 \
+ 349947592110433543619114105190630648009305990596089919920 #1^8 - 
      106747371466686462957887174434503958626612109573992670672 #1^9 \
+ 14573808550521644768419974300540238469473534471735953168 #1^10 &, 
    8]}, {x -> 
   Root[502925249036334454715336778553984003440864263790494070 + 
      79507334695638259109060376450677753198917644793560 y - 
      7228015973937440731974941433834202407604387494117592200 #1 + 
      46469470965681832229663230673595151067237874818752782840 #1^2 - 
      176009396354156927628824727628578542386502238219708082250 #1^3 \
+ 434953160110610020035094377442281900757943807833308766200 #1^4 - 
      732785165581324716157030456066865775762616415498186198190 #1^5 \
+ 852422521044952102836379759843676923790865671573802051255 #1^6 - 
      676099528944398287173100625127955582322701524364225111800 #1^7 \
+ 349947592110433543619114105190630648009305990596089919920 #1^8 - 
      106747371466686462957887174434503958626612109573992670672 #1^9 \
+ 14573808550521644768419974300540238469473534471735953168 #1^10 &, 
    9]}, {x -> 
   Root[502925249036334454715336778553984003440864263790494070 + 
      79507334695638259109060376450677753198917644793560 y - 
      7228015973937440731974941433834202407604387494117592200 #1 + 
      46469470965681832229663230673595151067237874818752782840 #1^2 - 
      176009396354156927628824727628578542386502238219708082250 #1^3 \
+ 434953160110610020035094377442281900757943807833308766200 #1^4 - 
      732785165581324716157030456066865775762616415498186198190 #1^5 \
+ 852422521044952102836379759843676923790865671573802051255 #1^6 - 
      676099528944398287173100625127955582322701524364225111800 #1^7 \
+ 349947592110433543619114105190630648009305990596089919920 #1^8 - 
      106747371466686462957887174434503958626612109573992670672 #1^9 \
+ 14573808550521644768419974300540238469473534471735953168 #1^10 &, 
    10]}}

As my polynomial is of degree 10 I have possibly 10 solutions to this equation. But actually I am looking what happens for $1/2 \leq x \leq 1$. Then, I changed my solve line by :

Solve[y == ff[x] && 1/2 <= x <= 1, x] 

But then I have 0 solutions :

Solve::ratnz: Solve was unable to solve the system with inexact coefficients. The answer was obtained by solving a corresponding exact system and numericizing the result.

But of course there is a solution we can see it graphically.

How can I solve this problem ?

$\endgroup$
  • 2
    $\begingroup$ Possible duplicate of How do I work with Root objects? $\endgroup$ – Lukas Lang Jun 7 '18 at 12:22
  • $\begingroup$ What is returned are the ten solutions to the equation you gave to Solve, so exactly what you want $\endgroup$ – Lukas Lang Jun 7 '18 at 12:23
  • $\begingroup$ @Mathe172 ok I edited, indeed i didn't told me i was looking for things on $[1/2;1]$ but then I still have a problem $\endgroup$ – StarBucK Jun 7 '18 at 12:31
  • $\begingroup$ Try Reduce instead of Solve, optionally with Simplify[…,y∈Reals] $\endgroup$ – Lukas Lang Jun 7 '18 at 13:03
  • $\begingroup$ It seems to me that the NDSolve approach is much simpler, e.g., if = NDSolveValue[{H2'[x[y]] x'[y] == 1, x[H2[3/4]] == 3/4}, x, {y, 0, Log[2]}, WorkingPrecision->16] $\endgroup$ – Carl Woll Jun 7 '18 at 16:23
3
$\begingroup$

Finding a good polynomial fit

Just as an alternative way to solve the first part of original problem, finding a good fit to the shannon entropy, here's another way.

First let's write down the expression we want to approximate

shannonentropy = (-x)*Log[x] - (1 - x)*Log[1 - x]

and a (polynomial) linear model with free parameters

model = \[Alpha] x (1 - x) + \[Beta] x^2 (1 - x)^2 + \[Gamma] x^3 (1 - x)^3

, that we want to optimize. Here we used our knowledge about the symmetry of the shannonentropy function to omit some other possible polynomial terms and also only used terms that fulfill the boundary condition, that the function goes to zero at x==0 and x==1.

Now because, in terms of integration, the expression we want to find a fit for, is not that complicated, we can directly integrate the squared error between our desired function and our model, fully symbollically without needing to discretize!

(shannonentropy - model)^2
Expand[%]
errsq = Integrate[Evaluate[%], {x, 0, 1}]

5/6 - [Pi]^2/18 - (37 [Beta])/900 - (533 [Gamma])/58800 + ( 143 (7 [Alpha] (-35 + 6 [Alpha]) + 18 [Alpha] [Beta] + 2 [Beta]^2) + 26 (22 [Alpha] + 5 [Beta]) [Gamma] + 15 [Gamma]^2)/180180

Now we just need to find the parameter set that minimizes the squared error and insert this into our model to get the best fit!

solparams = Minimize[errsq, {\[Alpha], \[Beta], \[Gamma]}]

{10833173/19756800 - [Pi]^2/ 18, {[Alpha] -> 1297/280, [Beta] -> -(451/35), [Gamma] -> 6149/ 280}}

bestfit = model /. solparams[[2]]

1297/280 (1 - x) x - 451/35 (1 - x)^2 x^2 + 6149/280 (1 - x)^3 x^3

Plot[{shannonentropy, bestfit}, {x, 0, 1}]

Shannon entropy and best fit

Getting the inverse of the polynomial

With this simpler model it's also easy to get the correct branch of the function inverse:

Solve[y == bestfit && 1/2 < x < 1, x, Reals]

{{x -> ConditionalExpression[ Root[280 y - 1297 #1 + 4905 #1^2 - 13365 #1^3 + 22055 #1^4 - 18447 #1^5 + 6149 #1^6 &, 2], 0 < y < 12469/17920]}}

, where Solve gives back the right branch as a Root expression, if the function value is in the correct value range (0 < y < 12469/17920). Here giving the Reals domain to Solve is important to get an explicit solution.

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.