# Express function derivative in terms of the function itself

I'm working on a minimization problem that involves the standard logistic function

$f(x) = \frac{1}{1+e^{-x}}$

along with its simple derivative

$f'(x) = f(x) \cdot (1 - f(x))$

The correctness of this derivative is easily proven, but I wonder how to get to this specific form using Mathematica. Deriving $f(x)$ and simplifying gives me:

In[1]:= f[x_] := 1 / (1 + Exp[-x])
In[2]:= der = Simplify[D[f[x], x]]

x
E
Out[2]= ---------
x 2
(1 + E )


Obviously this is correct, but since I will have to calculate $f(x)$ and $f'(x)$ at the same time, expressing $f'(x)$ in terms of $f(x)$ allows for faster computation.

I tried to use assumptions for Simplify like this:

Simplify[der, f[x] == fx]


but that doesn't work; neither did similar things with Reduce. The /. operator doesn't work either because 1 / (1 + Exp[-x]) doesn't appear in Out[2] exactly in that form.

I will have to calculate the derivatives of other functions as well and it would be nice to see if those are more easily expressed in terms of $f(x)$.

• If you simply want to prove that the two forms are equivalent try Simplify[D[f[x], x] - f[x] (1 - f[x])] – Jack LaVigne Sep 17 '15 at 15:01
• The function LogisticSigmoid[] is built-in, FYI. – J. M. will be back soon Sep 8 '17 at 22:10

You can use the third argument of Solve to eliminate variables from the system $$y=f(x),\quad dy=f'(x)$$

f[x_] := 1/(1 + Exp[-x]);

sol = Solve[{y == f[x], dy == f'[x]}, {dy}, {x}, Method -> Reduce]
(* {{dy -> y - y^2}} *)


(Or you could use Reduce directly, for the answer in a different form.)

If you want an ODE, then massage sol:

sol[[1, 1]] /. {y -> y[x], dy -> y'[x]} /. Rule -> Equal
(* y'[x] == y[x] - y[x]^2  *)

• This is very neat and makes sense. However I don't understand why the third argument to Solve is necessary. According to the docs that argument is supposed to be a domain (Reals, Complexes, etc). What does {x} mean as a domain? And why can't I put Reals in there? (It doesn't work if I do.) – jlh Sep 18 '15 at 10:21
• @jlh It's an old syntax, that has become undocumented. See the link I added to a Q&A about it. See also this search for more examples. – Michael E2 Sep 18 '15 at 10:25
• Thank you for the clarifications. Too bad it's not properly documented in the newest version. – jlh Sep 18 '15 at 12:47

One way might be to solve the defining equation for the exponential function

fx = 1/(1-Exp[-x])

sol = First@Solve[ f==fx/. Exp[-x]-> ex, ex]


which will yield

{ex -> (1 - f)/f}


Put that into the differentiated function and simplify the result

Simplify[ D[fx, x] /. Exp[-x] -> ex /. sol]


which results in

-(-1 + f) f

f := 1/(1 + Exp[-x])

solx = First[x /. Solve[f == g, x] /. C[1] -> 0]

(* Out[233]= Log[-(g/(-1 + g))] *)


Some derivatives

(D[f, x] /. x -> solx // Simplify) /. g -> "f[x]"

(*
Out[244]= -(-1 + "f[x]") "f[x]"
*)

(D[f, {x, 2}] /. x -> solx // Simplify) /. g -> "f[x]" // Expand

(*
Out[247]= "f[x]" - 3 ("f[x]")^2 + 2 ("f[x]")^3
*)

(D[f, {x, 3}] /. x -> solx // Simplify) /. g -> "f[x]" // Expand

(*
Out[246]= "f[x]" - 7 ("f[x]")^2 + 12 ("f[x]")^3 - 6 ("f[x]")^4
*)


Table of derivatives (output not shown here)

t = Table[(D[f, {x, k}] /. x -> solx // Simplify) /. g -> ff // Expand, {k, 0, 10}];


List of coefficients, and identification in OEIS

List @@@ t /. ff -> 1 // Column

(* https://oeis.org A163626 Triangle read by rows: The n-th derivative of the logistic function written in terms of y,where y=1/(1+exp(-x)) *)


$$\begin{array}{c} 1 \\ \{1,-1\} \\ \{1,-3,2\} \\ \{1,-7,12,-6\} \\ \{1,-15,50,-60,24\} \\ \{1,-31,180,-390,360,-120\} \\ \{1,-63,602,-2100,3360,-2520,720\} \\ \{1,-127,1932,-10206,25200,-31920,20160,-5040\} \\ \{1,-255,6050,-46620,166824,-317520,332640,-181440,40320\} \\ \{1,-511,18660,-204630,1020600,-2739240,4233600,-3780000,1814400,-362880\} \\ \{1,-1023,57002,-874500,5921520,-21538440,46070640,-59875200,46569600,-19958400,3628800\} \\ \end{array}$$

Derivative[1][f] = f[#] (1-f[#])&;


Then, we have:

Do[CellPrint[ExpressionCell[D[f[x], {x, n}] //Expand, "Output"]], {n, 5}]


f[x] - f[x]^2

f[x] - 3 f[x]^2 + 2 f[x]^3

f[x] - 7 f[x]^2 + 12 f[x]^3 - 6 f[x]^4

f[x] - 15 f[x]^2 + 50 f[x]^3 - 60 f[x]^4 + 24 f[x]^5

f[x] - 31 f[x]^2 + 180 f[x]^3 - 390 f[x]^4 + 360 f[x]^5 - 120 f[x]^6