4
$\begingroup$

I'm working on a minimization problem that involves the standard logistic function

$f(x) = \frac{1}{1+e^{-x}}$

along with its simple derivative

$f'(x) = f(x) \cdot (1 - f(x))$

The correctness of this derivative is easily proven, but I wonder how to get to this specific form using Mathematica. Deriving $f(x)$ and simplifying gives me:

In[1]:= f[x_] := 1 / (1 + Exp[-x])
In[2]:= der = Simplify[D[f[x], x]]

            x
           E
Out[2]= ---------
              x 2
        (1 + E )

Obviously this is correct, but since I will have to calculate $f(x)$ and $f'(x)$ at the same time, expressing $f'(x)$ in terms of $f(x)$ allows for faster computation.

I tried to use assumptions for Simplify like this:

Simplify[der, f[x] == fx]

but that doesn't work; neither did similar things with Reduce. The /. operator doesn't work either because 1 / (1 + Exp[-x]) doesn't appear in Out[2] exactly in that form.

I will have to calculate the derivatives of other functions as well and it would be nice to see if those are more easily expressed in terms of $f(x)$.

$\endgroup$
  • $\begingroup$ If you simply want to prove that the two forms are equivalent try Simplify[D[f[x], x] - f[x] (1 - f[x])] $\endgroup$ – Jack LaVigne Sep 17 '15 at 15:01
  • $\begingroup$ The function LogisticSigmoid[] is built-in, FYI. $\endgroup$ – J. M. will be back soon Sep 8 '17 at 22:10
5
$\begingroup$

You can use the third argument of Solve to eliminate variables from the system $$y=f(x),\quad dy=f'(x)$$

f[x_] := 1/(1 + Exp[-x]);

sol = Solve[{y == f[x], dy == f'[x]}, {dy}, {x}, Method -> Reduce]
(* {{dy -> y - y^2}} *)

(Or you could use Reduce directly, for the answer in a different form.)

If you want an ODE, then massage sol:

sol[[1, 1]] /. {y -> y[x], dy -> y'[x]} /. Rule -> Equal
(* y'[x] == y[x] - y[x]^2  *)
$\endgroup$
  • $\begingroup$ This is very neat and makes sense. However I don't understand why the third argument to Solve is necessary. According to the docs that argument is supposed to be a domain (Reals, Complexes, etc). What does {x} mean as a domain? And why can't I put Reals in there? (It doesn't work if I do.) $\endgroup$ – jlh Sep 18 '15 at 10:21
  • 2
    $\begingroup$ @jlh It's an old syntax, that has become undocumented. See the link I added to a Q&A about it. See also this search for more examples. $\endgroup$ – Michael E2 Sep 18 '15 at 10:25
  • $\begingroup$ Thank you for the clarifications. Too bad it's not properly documented in the newest version. $\endgroup$ – jlh Sep 18 '15 at 12:47
1
$\begingroup$

One way might be to solve the defining equation for the exponential function

fx = 1/(1-Exp[-x])

sol = First@Solve[ f==fx/. Exp[-x]-> ex, ex]

which will yield

{ex -> (1 - f)/f}

Put that into the differentiated function and simplify the result

Simplify[ D[fx, x] /. Exp[-x] -> ex /. sol]

which results in

-(-1 + f) f
$\endgroup$
1
$\begingroup$
f := 1/(1 + Exp[-x])

solx = First[x /. Solve[f == g, x] /. C[1] -> 0]

(* Out[233]= Log[-(g/(-1 + g))] *)

Some derivatives

(D[f, x] /. x -> solx // Simplify) /. g -> "f[x]"

(*
Out[244]= -(-1 + "f[x]") "f[x]" 
*)

(D[f, {x, 2}] /. x -> solx // Simplify) /. g -> "f[x]" // Expand

(*
Out[247]= "f[x]" - 3 ("f[x]")^2 + 2 ("f[x]")^3
*)

(D[f, {x, 3}] /. x -> solx // Simplify) /. g -> "f[x]" // Expand

(*
Out[246]= "f[x]" - 7 ("f[x]")^2 + 12 ("f[x]")^3 - 6 ("f[x]")^4
*)

Table of derivatives (output not shown here)

t = Table[(D[f, {x, k}] /. x -> solx // Simplify) /. g -> ff // Expand, {k, 0, 10}];

List of coefficients, and identification in OEIS

List @@@ t /. ff -> 1 // Column

(* https://oeis.org A163626 Triangle read by rows: The n-th derivative of the logistic function written in terms of y,where y=1/(1+exp(-x)) *)

$$ \begin{array}{c} 1 \\ \{1,-1\} \\ \{1,-3,2\} \\ \{1,-7,12,-6\} \\ \{1,-15,50,-60,24\} \\ \{1,-31,180,-390,360,-120\} \\ \{1,-63,602,-2100,3360,-2520,720\} \\ \{1,-127,1932,-10206,25200,-31920,20160,-5040\} \\ \{1,-255,6050,-46620,166824,-317520,332640,-181440,40320\} \\ \{1,-511,18660,-204630,1020600,-2739240,4233600,-3780000,1814400,-362880\} \\ \{1,-1023,57002,-874500,5921520,-21538440,46070640,-59875200,46569600,-19958400,3628800\} \\ \end{array} $$

$\endgroup$
0
$\begingroup$

This answer doesn't address the correctness of your derivative. Instead, it shows how to implement such a derivative implicitly:

Derivative[1][f] = f[#] (1-f[#])&;

Then, we have:

Do[CellPrint[ExpressionCell[D[f[x], {x, n}] //Expand, "Output"]], {n, 5}]

f[x] - f[x]^2

f[x] - 3 f[x]^2 + 2 f[x]^3

f[x] - 7 f[x]^2 + 12 f[x]^3 - 6 f[x]^4

f[x] - 15 f[x]^2 + 50 f[x]^3 - 60 f[x]^4 + 24 f[x]^5

f[x] - 31 f[x]^2 + 180 f[x]^3 - 390 f[x]^4 + 360 f[x]^5 - 120 f[x]^6

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.