Symbolic derivative of $n$-term product

Question

I want to determine the relationship that must exist between the $x_i$ and $y_i$ such that

$$ \frac{\partial}{\partial\theta} \prod_{i=1}^n \frac{f(x_i,\theta)}{f(y_i,\theta)} = 0, $$

where

$$ f(x,\theta) = \frac{e^{-(\theta - x)}}{(1+e^{-(\theta - x)})^2}, \;\; \forall x \in {\mathbb R}, \theta \in {\mathbb R} $$

---

Clarification: what I'm trying to find is a condition on the $x_i$ and $y_i$ such that the derivative above (viewed as a function of $\theta$) is zero for all $\theta$ only if this condition holds. Clearly, this derivative is zero for all $\theta$ if, $\forall i\in\{1,\dots,n\}$, the condition $x_i = y_i$ holds, since then the product in the derivative expression above is identically 1. But this condition is not necessary: the derivative will be identically zero also if there is an $n$-permutation $\sigma$ such that $\forall i,\,x_i = y_{\sigma(i)}$. My problem is to prove that the derivative is identically zero (i.e. it is zero for all $\theta$) only if such a $\sigma$ exists, for given $x_i$ and $y_i$.

---

So, hoping to have a look at the derivative above, I input this into Mathematica:

Block[{f, θ, x, y, i, n},
 f[x_][θ_] := E^(-x + θ)/(1 + E^(-x + θ))^2;
 D[Product[f[x[i]][θ]/f[y[i]][θ], {i, n}], θ]
]

...but Mathematically basically spits back the last formula (after replacing the various expressions in f):

D[Product[(E^(-x[i] + y[i])*(1 + E^(θ - y[i]))^2)/(1 + E^(θ - x[i]))^2, {i, n}], θ]

If instead of using a symbolic product (with an unspecified number of terms) I attempt the same thing with a product of three terms, namely

(f[x1][θ]/f[y1][θ]) (f[x2][θ]/f[y2][θ]) (f[x3][θ]/f[y3][θ])

...Mathematica does compute the derivative (though the resulting expression is hairy, and I can't extract any insight from it). So my first question is

How can I get Mathematica to produce the expression for the derivative for the general case?

(After all, the derivative of an $n$-term product has a form that Mathematica should be able to express relatively easily.)

In any case, the results I got for a three-term product were not encouraging. Of course, I really don't care for the derivative per se, but rather, what I'm after are the conditions on the $x_i$ and $y_i$ that make this derivative vanish.

Is there a way that Mathematica can show me the relationship between the $x_i$ and $y_i$ when this derivative is 0?

Answer 1

Teach Mathematica the rules. The fundamental one is

$$\frac{d}{dx} \prod_i f_i(x) = \prod_i f_i(x) \sum_i \frac{f_i'(x)}{f_i(x)},$$

at least where none of the $f_i(x)=0$. Iterate this to obtain higher-order derivatives:

Unprotect[D];
D[Product[f_, i___], x_Except[List]] := Product[f, i] Sum[D[f, x]/f, i];
D[Product[f_, i___], {x_, n_Integer}] := Nest[D[#, x] &, Product[f, i], n];
D[Product[f_, i___], x_Except[List], y__] := D[Product[f, i] Sum[D[f, x]/f, i], y];
Protect[D]

Example from the question:

f[x_][\[Theta]_] := E^(-x + \[Theta])/(1 + E^(-x + \[Theta]))^2;
D[Product[f[x[i]][\[Theta]]/f[y[i]][\[Theta]], {i, n}], \[Theta]]

$$\left(\prod _i^n \frac{e^{y(i)-x(i)} \left(e^{\theta -y(i)}+1\right)^2}{\left(e^{\theta -x(i)}+1\right)^2}\right) \sum _i^n \frac{\left(e^{\theta -x(i)}+1\right)^2 e^{x(i)-y(i)} \left(\frac{2 e^{\theta -x(i)} \left(e^{\theta -y(i)}+1\right)}{\left(e^{\theta -x(i)}+1\right)^2}-\frac{2 \left(e^{\theta -y(i)}+1\right)^2 e^{\theta -2 x(i)+y(i)}}{\left(e^{\theta -x(i)}+1\right)^3}\right)}{\left(e^{\theta -y(i)}+1\right)^2}$$

Examples (variants of the help page examples):

D[Product[x^i, {i, 1, n}], x]

$\frac{1}{2} n (n+1) x^{\frac{1}{2} n (n+1)-1}$

D[Product[Sin[x], {n}], {x, 4}]

$(n-1)^2 \sin ^{n-1}(x)+2 (n-2) (n-1) \sin ^{n-1}(x)+(n-4) (n-3) (n-2) (n-1) \cos ^4(x) \sin ^{n-5}(x)-(n-2) (n-1)^2 \cos ^2(x) \sin ^{n-3}(x)-2 (n-2)^2 (n-1) \cos ^2(x) \sin ^{n-3}(x)-3 (n-3) (n-2) (n-1) \cos ^2(x) \sin ^{n-3}(x)$

D[Product[Sin[x y]^i, {i, 1, n}], x, y]

$-\frac{1}{2} n (n+1) x y \sin ^{\frac{1}{2} n (n+1)}(x y)+\frac{1}{2} n (n+1) \left(\frac{1}{2} n (n+1)-1\right) x y \cos ^2(x y) \sin ^{\frac{1}{2} n (n+1)-2}(x y)+\frac{1}{2} n (n+1) \cos (x y) \sin ^{\frac{1}{2} n (n+1)-1}(x y)$

D[Product[Subscript[f, i][x], {i, 1, 3}], x]

$f_2(x) f_3(x) f_1'(x)+f_1(x) f_3(x) f_2'(x)+f_1(x) f_2(x) f_3'(x)$

D[Product[x Sin[y]^i, {i, 0, n}], {{x, y}, 2}]

$\left( \begin{array}{cc} n (n+1) x^{n-1} \sin ^{\frac{1}{2} n (n+1)}(y) & \frac{1}{2} n (n+1)^2 x^n \cos (y) \sin ^{\frac{1}{2} n (n+1)-1}(y) \\ \frac{1}{2} n (n+1)^2 x^n \cos (y) \sin ^{\frac{1}{2} n (n+1)-1}(y) & \frac{1}{2} n (n+1) \left(\frac{1}{2} n (n+1)-1\right) x^{n+1} \cos ^2(y) \sin ^{\frac{1}{2} n (n+1)-2}(y)-\frac{1}{2} n (n+1) x^{n+1} \sin ^{\frac{1}{2} n (n+1)}(y) \end{array} \right)$

---

The answer to the second question is generic: you are imposing one (differentiable) relationship among $2n$ variables, which therefore describes a $2n-1$ dimensional manifold. For instance, with $n=2$ applying Solve gives

$$y(2)\to \log \left(\frac{e^{2 \theta +x(1)}+e^{2 \theta +x(2)}+2 e^{\theta +x(1)+x(2)}+e^{x(1)+x(2)+y(1)}-e^{2 \theta +y(1)}}{e^{2 \theta }+e^{x(1)+y(1)}+e^{x(2)+y(1)}-e^{x(1)+x(2)}+2 e^{\theta +y(1)}}\right)$$

To see that this is nondegenerate, you can explore the solution space dynamically if you like:

Manipulate[
 ContourPlot[
  Log[(E^(2 t + x1) + E^(2 t + x2) + 2 E^(t + x1 + x2) - E^(2 t + y1) + E^(x1 + x2 + y1))/(
   E^(2 t) - E^(x1 + x2) + 2 E^(t + y1) + E^(x1 + y1) + E^(x2 + y1))], {x1, -1, 1}, {x2, -1, 1}],
 {{y1, 0}, -1, 1}, {t, -1, 1}]

Answer 2

You can get the result you want as follows. This does not really use Mathematica except for purposes of exposition.

I'll truncate in powers of Exp[theta-x] and rewrite that as e[theta-x] to suppress evaluation and make obvious the low order terms of the series. We will throw away the numerator terms because they clearly contribute a multiplicativbe constant with respect to theta. I will separately handle the case where we have the (1 + e[(-x + th)])^2 in the numerator (for y_j's) vs. denominator (for x_j's). For the former we have the function g[x,th] below.

f[x_, th_] := (1 - e[(-x + th)] + e[(-x + th)]^2)^2
g[x_, th_] := (1 + e[(-x + th)])^2

Let's see what it looks like for the case n=1. I'll just call the variables x' andy`.

Expand[f[x, th]*g[y, th]]

(* Out[16]= 1 - 2 e[th - x] + 3 e[th - x]^2 - 2 e[th - x]^3 + 
 e[th - x]^4 + 2 e[th - y] - 4 e[th - x] e[th - y] + 
 6 e[th - x]^2 e[th - y] - 4 e[th - x]^3 e[th - y] + 
 2 e[th - x]^4 e[th - y] + e[th - y]^2 - 2 e[th - x] e[th - y]^2 + 
 3 e[th - x]^2 e[th - y]^2 - 2 e[th - x]^3 e[th - y]^2 + 
 e[th - x]^4 e[th - y]^2 *)

Notice that the first order terms (in powers of e[theta+something], that is) do not vanish unless x==y (the first order term is -2 e[th - x] + 2 e[th - y]). Now suppose we have n>1. What will happen is we'll have, at the lowest order, products of factors of the form

(1 -2 e[th - x[j]] + 2 e[th - y[j]])

It is easy to see that the first order terms will then be, in Mathematica notation,

Sum[-2 e[th - x[j]] + 2 e[th - y[j]], {j,n}]

Now let e[arg_] = Exp[arg]. As we have a power series in powers of Exp[theta-x[j]] and Exp[theta-y[j]], and as it must vanish in theta, in particular the first order terms must vanish in theta. It is straightforward to show that this can only happen if these terms pairwise cancel. (If nothing else, expand as power series in theta at the origin. You can use some polynomial algebra reasoning from there.)

Answer 3

One trick that hasn't been pointed out yet is this:

Extremizing f is equivalent to extremizing any monotonic function of f. Since your f is positive the product is too. Therefore, we can in particular choose the Log of the product as a convenient monotonic function to extremize. Then the equivalent problem is

f[x_, θ_] := E^(-x + θ)/(1 + E^(-x + θ))^2;
D[Sum[Log@f[x[i], θ] - Log@f[y[i], θ], {i, n}], θ]

$\sum _i^n \left(e^{x(i)-\theta } \left(e^{\theta -x(i)}+1\right)^2 \left(\frac{e^{\theta -x(i)}}{\left(e^{\theta -x(i)}+1\right)^2}-\frac{2 e^{2 \theta -2 x(i)}}{\left(e^{\theta -x(i)}+1\right)^3}\right)-e^{y(i)-\theta } \left(e^{\theta -y(i)}+1\right)^2 \left(\frac{e^{\theta -y(i)}}{\left(e^{\theta -y(i)}+1\right)^2}-\frac{2 e^{2 \theta -2 y(i)}}{\left(e^{\theta -y(i)}+1\right)^3}\right)\right)$

As you see, the derivative has been done without any problems. Setting this to zero is equivalent to the original problem, and the rest just depends on what relationship between the variables you want to extract (i.e., following whuber's or Daniel Lichtblau's interpretation). I'll leave that open.