6
$\begingroup$

I want to determine the relationship that must exist between the $x_i$ and $y_i$ such that

$$ \frac{\partial}{\partial\theta} \prod_{i=1}^n \frac{f(x_i,\theta)}{f(y_i,\theta)} = 0, $$

where

$$ f(x,\theta) = \frac{e^{-(\theta - x)}}{(1+e^{-(\theta - x)})^2}, \;\; \forall x \in {\mathbb R}, \theta \in {\mathbb R} $$


Clarification: what I'm trying to find is a condition on the $x_i$ and $y_i$ such that the derivative above (viewed as a function of $\theta$) is zero for all $\theta$ only if this condition holds. Clearly, this derivative is zero for all $\theta$ if, $\forall i\in\{1,\dots,n\}$, the condition $x_i = y_i$ holds, since then the product in the derivative expression above is identically 1. But this condition is not necessary: the derivative will be identically zero also if there is an $n$-permutation $\sigma$ such that $\forall i,\,x_i = y_{\sigma(i)}$. My problem is to prove that the derivative is identically zero (i.e. it is zero for all $\theta$) only if such a $\sigma$ exists, for given $x_i$ and $y_i$.


So, hoping to have a look at the derivative above, I input this into Mathematica:

Block[{f, θ, x, y, i, n},
 f[x_][θ_] := E^(-x + θ)/(1 + E^(-x + θ))^2;
 D[Product[f[x[i]][θ]/f[y[i]][θ], {i, n}], θ]
]

...but Mathematically basically spits back the last formula (after replacing the various expressions in f):

D[Product[(E^(-x[i] + y[i])*(1 + E^(θ - y[i]))^2)/(1 + E^(θ - x[i]))^2, {i, n}], θ]

If instead of using a symbolic product (with an unspecified number of terms) I attempt the same thing with a product of three terms, namely

(f[x1][θ]/f[y1][θ]) (f[x2][θ]/f[y2][θ]) (f[x3][θ]/f[y3][θ])

...Mathematica does compute the derivative (though the resulting expression is hairy, and I can't extract any insight from it). So my first question is

How can I get Mathematica to produce the expression for the derivative for the general case?

(After all, the derivative of an $n$-term product has a form that Mathematica should be able to express relatively easily.)

In any case, the results I got for a three-term product were not encouraging. Of course, I really don't care for the derivative per se, but rather, what I'm after are the conditions on the $x_i$ and $y_i$ that make this derivative vanish.

Is there a way that Mathematica can show me the relationship between the $x_i$ and $y_i$ when this derivative is 0?

$\endgroup$
4
  • $\begingroup$ The product must be constant in theta. I believe this will force the set of x's to be the same as the set of y's (that is, same lists up to ordering). $\endgroup$ Feb 12, 2013 at 0:39
  • $\begingroup$ @DanielLichtblau: Yes, that's my guess too. $\endgroup$
    – kjo
    Feb 12, 2013 at 0:40
  • $\begingroup$ Okay, so I guess you were hoping for a proof. I don't think Mathematica will be able to help here (I'll be happy if som ebody shows I am wrong about this). The thetas in the numerator of f will all cancel. So I think some fiddling with the denominator might show that those do not go away unless x's equal y's as sets. I'll give it some more thought. $\endgroup$ Feb 12, 2013 at 0:43
  • $\begingroup$ @DanielLichtblau Actually, this condition is easy to see from the expression for the derivative when $n=1$, so maybe I can set up an induction.. $\endgroup$
    – kjo
    Feb 12, 2013 at 0:55

3 Answers 3

12
$\begingroup$

Teach Mathematica the rules. The fundamental one is

$$\frac{d}{dx} \prod_i f_i(x) = \prod_i f_i(x) \sum_i \frac{f_i'(x)}{f_i(x)},$$

at least where none of the $f_i(x)=0$. Iterate this to obtain higher-order derivatives:

Unprotect[D];
D[Product[f_, i___], x_Except[List]] := Product[f, i] Sum[D[f, x]/f, i];
D[Product[f_, i___], {x_, n_Integer}] := Nest[D[#, x] &, Product[f, i], n];
D[Product[f_, i___], x_Except[List], y__] := D[Product[f, i] Sum[D[f, x]/f, i], y];
Protect[D]

Example from the question:

f[x_][\[Theta]_] := E^(-x + \[Theta])/(1 + E^(-x + \[Theta]))^2;
D[Product[f[x[i]][\[Theta]]/f[y[i]][\[Theta]], {i, n}], \[Theta]]

$$\left(\prod _i^n \frac{e^{y(i)-x(i)} \left(e^{\theta -y(i)}+1\right)^2}{\left(e^{\theta -x(i)}+1\right)^2}\right) \sum _i^n \frac{\left(e^{\theta -x(i)}+1\right)^2 e^{x(i)-y(i)} \left(\frac{2 e^{\theta -x(i)} \left(e^{\theta -y(i)}+1\right)}{\left(e^{\theta -x(i)}+1\right)^2}-\frac{2 \left(e^{\theta -y(i)}+1\right)^2 e^{\theta -2 x(i)+y(i)}}{\left(e^{\theta -x(i)}+1\right)^3}\right)}{\left(e^{\theta -y(i)}+1\right)^2}$$

Examples (variants of the help page examples):

D[Product[x^i, {i, 1, n}], x]

$\frac{1}{2} n (n+1) x^{\frac{1}{2} n (n+1)-1}$

D[Product[Sin[x], {n}], {x, 4}]

$(n-1)^2 \sin ^{n-1}(x)+2 (n-2) (n-1) \sin ^{n-1}(x)+(n-4) (n-3) (n-2) (n-1) \cos ^4(x) \sin ^{n-5}(x)-(n-2) (n-1)^2 \cos ^2(x) \sin ^{n-3}(x)-2 (n-2)^2 (n-1) \cos ^2(x) \sin ^{n-3}(x)-3 (n-3) (n-2) (n-1) \cos ^2(x) \sin ^{n-3}(x)$

D[Product[Sin[x y]^i, {i, 1, n}], x, y]

$-\frac{1}{2} n (n+1) x y \sin ^{\frac{1}{2} n (n+1)}(x y)+\frac{1}{2} n (n+1) \left(\frac{1}{2} n (n+1)-1\right) x y \cos ^2(x y) \sin ^{\frac{1}{2} n (n+1)-2}(x y)+\frac{1}{2} n (n+1) \cos (x y) \sin ^{\frac{1}{2} n (n+1)-1}(x y)$

D[Product[Subscript[f, i][x], {i, 1, 3}], x]

$f_2(x) f_3(x) f_1'(x)+f_1(x) f_3(x) f_2'(x)+f_1(x) f_2(x) f_3'(x)$

D[Product[x Sin[y]^i, {i, 0, n}], {{x, y}, 2}]

$\left( \begin{array}{cc} n (n+1) x^{n-1} \sin ^{\frac{1}{2} n (n+1)}(y) & \frac{1}{2} n (n+1)^2 x^n \cos (y) \sin ^{\frac{1}{2} n (n+1)-1}(y) \\ \frac{1}{2} n (n+1)^2 x^n \cos (y) \sin ^{\frac{1}{2} n (n+1)-1}(y) & \frac{1}{2} n (n+1) \left(\frac{1}{2} n (n+1)-1\right) x^{n+1} \cos ^2(y) \sin ^{\frac{1}{2} n (n+1)-2}(y)-\frac{1}{2} n (n+1) x^{n+1} \sin ^{\frac{1}{2} n (n+1)}(y) \end{array} \right)$


The answer to the second question is generic: you are imposing one (differentiable) relationship among $2n$ variables, which therefore describes a $2n-1$ dimensional manifold. For instance, with $n=2$ applying Solve gives

$$y(2)\to \log \left(\frac{e^{2 \theta +x(1)}+e^{2 \theta +x(2)}+2 e^{\theta +x(1)+x(2)}+e^{x(1)+x(2)+y(1)}-e^{2 \theta +y(1)}}{e^{2 \theta }+e^{x(1)+y(1)}+e^{x(2)+y(1)}-e^{x(1)+x(2)}+2 e^{\theta +y(1)}}\right)$$

To see that this is nondegenerate, you can explore the solution space dynamically if you like:

Manipulate[
 ContourPlot[
  Log[(E^(2 t + x1) + E^(2 t + x2) + 2 E^(t + x1 + x2) - E^(2 t + y1) + E^(x1 + x2 + y1))/(
   E^(2 t) - E^(x1 + x2) + 2 E^(t + y1) + E^(x1 + y1) + E^(x2 + y1))], {x1, -1, 1}, {x2, -1, 1}],
 {{y1, 0}, -1, 1}, {t, -1, 1}]

Contour plot

$\endgroup$
6
  • 1
    $\begingroup$ In response to a now-deleted comment: these new rules for differentiating an arbitrary finite product are not precisely the familiar "product rule" for derivatives; they are a generalization thereof. Because they apply to a general symbolic limit $n$, they do not follow by means of any finite sequence of operations from the product rule itself ($(fg)' = f'g + fg'$), but rather require an induction. The distinction perhaps is subtle but--as seen in this case--it's real. $\endgroup$
    – whuber
    Feb 12, 2013 at 4:20
  • $\begingroup$ Thanks! I picked your answer reluctantly, because I really would have preferred to pick all three: each shows something different and valuable. I figured that it would be better to choose (however inaccurately) than not to, so I finally went with yours on the grounds that it contains most Mathematica-specific stuff... $\endgroup$
    – kjo
    Feb 12, 2013 at 10:08
  • $\begingroup$ @whuber x_Except[List] doesn't really make sense to me - and, according to my tests, it doesn't seem to work - perhaps you mean x : Except[List[___]]? $\endgroup$
    – VF1
    Feb 13, 2013 at 0:05
  • $\begingroup$ @VF1 It works fine for me--MMA 8.0. In what fashion do your tests fail? $\endgroup$
    – whuber
    Feb 13, 2013 at 0:33
  • 1
    $\begingroup$ Congrats on your 10k! Your answers are always a treat :) $\endgroup$
    – rm -rf
    Feb 26, 2013 at 3:02
8
$\begingroup$

You can get the result you want as follows. This does not really use Mathematica except for purposes of exposition.

I'll truncate in powers of Exp[theta-x] and rewrite that as e[theta-x] to suppress evaluation and make obvious the low order terms of the series. We will throw away the numerator terms because they clearly contribute a multiplicativbe constant with respect to theta. I will separately handle the case where we have the (1 + e[(-x + th)])^2 in the numerator (for y_j's) vs. denominator (for x_j's). For the former we have the function g[x,th] below.

f[x_, th_] := (1 - e[(-x + th)] + e[(-x + th)]^2)^2
g[x_, th_] := (1 + e[(-x + th)])^2

Let's see what it looks like for the case n=1. I'll just call the variables x' andy`.

Expand[f[x, th]*g[y, th]]

(* Out[16]= 1 - 2 e[th - x] + 3 e[th - x]^2 - 2 e[th - x]^3 + 
 e[th - x]^4 + 2 e[th - y] - 4 e[th - x] e[th - y] + 
 6 e[th - x]^2 e[th - y] - 4 e[th - x]^3 e[th - y] + 
 2 e[th - x]^4 e[th - y] + e[th - y]^2 - 2 e[th - x] e[th - y]^2 + 
 3 e[th - x]^2 e[th - y]^2 - 2 e[th - x]^3 e[th - y]^2 + 
 e[th - x]^4 e[th - y]^2 *)

Notice that the first order terms (in powers of e[theta+something], that is) do not vanish unless x==y (the first order term is -2 e[th - x] + 2 e[th - y]). Now suppose we have n>1. What will happen is we'll have, at the lowest order, products of factors of the form

(1 -2 e[th - x[j]] + 2 e[th - y[j]])

It is easy to see that the first order terms will then be, in Mathematica notation,

Sum[-2 e[th - x[j]] + 2 e[th - y[j]], {j,n}]

Now let e[arg_] = Exp[arg]. As we have a power series in powers of Exp[theta-x[j]] and Exp[theta-y[j]], and as it must vanish in theta, in particular the first order terms must vanish in theta. It is straightforward to show that this can only happen if these terms pairwise cancel. (If nothing else, expand as power series in theta at the origin. You can use some polynomial algebra reasoning from there.)

$\endgroup$
4
  • $\begingroup$ To be clear: it appears your interpretation of the second question is that it asks for the relationship among the $x_i$ and $y_i$ when the derivative is the zero function; not when the derivative merely vanishes! $\endgroup$
    – whuber
    Feb 12, 2013 at 3:57
  • $\begingroup$ @DanielLichtblau: Thanks! This is probably the road to the proof I'm looking for, even though, as you say, it does not use Mathematica much. (See my comment to whuber's answer.) $\endgroup$
    – kjo
    Feb 12, 2013 at 10:10
  • $\begingroup$ @whuber: I've added a clarification to my post. $\endgroup$
    – kjo
    Feb 12, 2013 at 10:44
  • 1
    $\begingroup$ @whuber Yes, my interpretation was vanishing as a function rather than finding critical points. One reason is that the latter would give a relation involving the xs, ys, and theta rather than just the xs and ys. I guess there was a latter clarification that states the vanishing function interpretation. $\endgroup$ Feb 12, 2013 at 14:33
8
$\begingroup$

One trick that hasn't been pointed out yet is this:

Extremizing f is equivalent to extremizing any monotonic function of f. Since your f is positive the product is too. Therefore, we can in particular choose the Log of the product as a convenient monotonic function to extremize. Then the equivalent problem is

f[x_, θ_] := E^(-x + θ)/(1 + E^(-x + θ))^2;
D[Sum[Log@f[x[i], θ] - Log@f[y[i], θ], {i, n}], θ]

$\sum _i^n \left(e^{x(i)-\theta } \left(e^{\theta -x(i)}+1\right)^2 \left(\frac{e^{\theta -x(i)}}{\left(e^{\theta -x(i)}+1\right)^2}-\frac{2 e^{2 \theta -2 x(i)}}{\left(e^{\theta -x(i)}+1\right)^3}\right)-e^{y(i)-\theta } \left(e^{\theta -y(i)}+1\right)^2 \left(\frac{e^{\theta -y(i)}}{\left(e^{\theta -y(i)}+1\right)^2}-\frac{2 e^{2 \theta -2 y(i)}}{\left(e^{\theta -y(i)}+1\right)^3}\right)\right)$

As you see, the derivative has been done without any problems. Setting this to zero is equivalent to the original problem, and the rest just depends on what relationship between the variables you want to extract (i.e., following whuber's or Daniel Lichtblau's interpretation). I'll leave that open.

$\endgroup$
4
  • 2
    $\begingroup$ +1 Because $d(\log(f(x))/dx = f'(x)/f(x)$, this is identical to the first formula in my answer (but without the initial factor of the product itself): that's how the formula is usually derived. $\endgroup$
    – whuber
    Feb 12, 2013 at 4:11
  • 1
    $\begingroup$ That's right - I simply worked with the OP's statement "I really don't care about the derivative per se," where the prefactor becomes irrelevant to the extremization. $\endgroup$
    – Jens
    Feb 12, 2013 at 4:19
  • 1
    $\begingroup$ By ignoring the "prefactor," you could be overlooking some critical points. For instance, consider $\prod_{i=1}^2 x = x^2$ defined on the entire real line. Your formula would set $\sum_{i=1}^2 1/x = 2/x=0$, having no solutions, whereas $x=0$ is the (unique) critical point. Although I said my formula is derived with logs, which is true, in the end--because it does not use logarithms--it is fully general (because the apparent singularities are seen to be removable). $\endgroup$
    – whuber
    Feb 12, 2013 at 4:24
  • 2
    $\begingroup$ I think I covered that in my intro. It doesn't apply here. $\endgroup$
    – Jens
    Feb 12, 2013 at 5:26

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.