Take the 2-minute tour ×
Mathematica Stack Exchange is a question and answer site for users of Mathematica. It's 100% free, no registration required.

I want to determine the relationship that must exist between the $x_i$ and $y_i$ such that

$$ \frac{\partial}{\partial\theta} \prod_{i=1}^n \frac{f(x_i,\theta)}{f(y_i,\theta)} = 0, $$

where

$$ f(x,\theta) = \frac{e^{-(\theta - x)}}{(1+e^{-(\theta - x)})^2}, \;\; \forall x \in {\mathbb R}, \theta \in {\mathbb R} $$


Clarification: what I'm trying to find is a condition on the $x_i$ and $y_i$ such that the derivative above (viewed as a function of $\theta$) is zero for all $\theta$ only if this condition holds. Clearly, this derivative is zero for all $\theta$ if, $\forall i\in\{1,\dots,n\}$, the condition $x_i = y_i$ holds, since then the product in the derivative expression above is identically 1. But this condition is not necessary: the derivative will be identically zero also if there is an $n$-permutation $\sigma$ such that $\forall i,\,x_i = y_{\sigma(i)}$. My problem is to prove that the derivative is identically zero (i.e. it is zero for all $\theta$) only if such a $\sigma$ exists, for given $x_i$ and $y_i$.


So, hoping to have a look at the derivative above, I input this into Mathematica:

Block[{f, θ, x, y, i, n},
 f[x_][θ_] := E^(-x + θ)/(1 + E^(-x + θ))^2;
 D[Product[f[x[i]][θ]/f[y[i]][θ], {i, n}], θ]
]

...but Mathematically basically spits back the last formula (after replacing the various expressions in f):

D[Product[(E^(-x[i] + y[i])*(1 + E^(θ - y[i]))^2)/(1 + E^(θ - x[i]))^2, {i, n}], θ]

If instead of using a symbolic product (with an unspecified number of terms) I attempt the same thing with a product of three terms, namely

(f[x1][θ]/f[y1][θ]) (f[x2][θ]/f[y2][θ]) (f[x3][θ]/f[y3][θ])

...Mathematica does compute the derivative (though the resulting expression is hairy, and I can't extract any insight from it). So my first question is

How can I get Mathematica to produce the expression for the derivative for the general case?

(After all, the derivative of an $n$-term product has a form that Mathematica should be able to express relatively easily.)

In any case, the results I got for a three-term product were not encouraging. Of course, I really don't care for the derivative per se, but rather, what I'm after are the conditions on the $x_i$ and $y_i$ that make this derivative vanish.

Is there a way that Mathematica can show me the relationship between the $x_i$ and $y_i$ when this derivative is 0?

share|improve this question
    
The product must be constant in theta. I believe this will force the set of x's to be the same as the set of y's (that is, same lists up to ordering). –  Daniel Lichtblau Feb 12 '13 at 0:39
    
@DanielLichtblau: Yes, that's my guess too. –  kjo Feb 12 '13 at 0:40
    
Okay, so I guess you were hoping for a proof. I don't think Mathematica will be able to help here (I'll be happy if som ebody shows I am wrong about this). The thetas in the numerator of f will all cancel. So I think some fiddling with the denominator might show that those do not go away unless x's equal y's as sets. I'll give it some more thought. –  Daniel Lichtblau Feb 12 '13 at 0:43
    
@DanielLichtblau Actually, this condition is easy to see from the expression for the derivative when $n=1$, so maybe I can set up an induction.. –  kjo Feb 12 '13 at 0:55

3 Answers 3

up vote 8 down vote accepted

Teach Mathematica the rules. The fundamental one is

$$\frac{d}{dx} \prod_i f_i(x) = \prod_i f_i(x) \sum_i \frac{f_i'(x)}{f_i(x)},$$

at least where none of the $f_i(x)=0$. Iterate this to obtain higher-order derivatives:

Unprotect[D];
D[Product[f_, i___], x_Except[List]] := Product[f, i] Sum[D[f, x]/f, i];
D[Product[f_, i___], {x_, n_Integer}] := Nest[D[#, x] &, Product[f, i], n];
D[Product[f_, i___], x_Except[List], y__] := D[Product[f, i] Sum[D[f, x]/f, i], y];
Protect[D]

Example from the question:

f[x_][\[Theta]_] := E^(-x + \[Theta])/(1 + E^(-x + \[Theta]))^2;
D[Product[f[x[i]][\[Theta]]/f[y[i]][\[Theta]], {i, n}], \[Theta]]

$$\left(\prod _i^n \frac{e^{y(i)-x(i)} \left(e^{\theta -y(i)}+1\right)^2}{\left(e^{\theta -x(i)}+1\right)^2}\right) \sum _i^n \frac{\left(e^{\theta -x(i)}+1\right)^2 e^{x(i)-y(i)} \left(\frac{2 e^{\theta -x(i)} \left(e^{\theta -y(i)}+1\right)}{\left(e^{\theta -x(i)}+1\right)^2}-\frac{2 \left(e^{\theta -y(i)}+1\right)^2 e^{\theta -2 x(i)+y(i)}}{\left(e^{\theta -x(i)}+1\right)^3}\right)}{\left(e^{\theta -y(i)}+1\right)^2}$$

Examples (variants of the help page examples):

D[Product[x^i, {i, 1, n}], x]

$\frac{1}{2} n (n+1) x^{\frac{1}{2} n (n+1)-1}$

D[Product[Sin[x], {n}], {x, 4}]

$(n-1)^2 \sin ^{n-1}(x)+2 (n-2) (n-1) \sin ^{n-1}(x)+(n-4) (n-3) (n-2) (n-1) \cos ^4(x) \sin ^{n-5}(x)-(n-2) (n-1)^2 \cos ^2(x) \sin ^{n-3}(x)-2 (n-2)^2 (n-1) \cos ^2(x) \sin ^{n-3}(x)-3 (n-3) (n-2) (n-1) \cos ^2(x) \sin ^{n-3}(x)$

D[Product[Sin[x y]^i, {i, 1, n}], x, y]

$-\frac{1}{2} n (n+1) x y \sin ^{\frac{1}{2} n (n+1)}(x y)+\frac{1}{2} n (n+1) \left(\frac{1}{2} n (n+1)-1\right) x y \cos ^2(x y) \sin ^{\frac{1}{2} n (n+1)-2}(x y)+\frac{1}{2} n (n+1) \cos (x y) \sin ^{\frac{1}{2} n (n+1)-1}(x y)$

D[Product[Subscript[f, i][x], {i, 1, 3}], x]

$f_2(x) f_3(x) f_1'(x)+f_1(x) f_3(x) f_2'(x)+f_1(x) f_2(x) f_3'(x)$

D[Product[x Sin[y]^i, {i, 0, n}], {{x, y}, 2}]

$\left( \begin{array}{cc} n (n+1) x^{n-1} \sin ^{\frac{1}{2} n (n+1)}(y) & \frac{1}{2} n (n+1)^2 x^n \cos (y) \sin ^{\frac{1}{2} n (n+1)-1}(y) \\ \frac{1}{2} n (n+1)^2 x^n \cos (y) \sin ^{\frac{1}{2} n (n+1)-1}(y) & \frac{1}{2} n (n+1) \left(\frac{1}{2} n (n+1)-1\right) x^{n+1} \cos ^2(y) \sin ^{\frac{1}{2} n (n+1)-2}(y)-\frac{1}{2} n (n+1) x^{n+1} \sin ^{\frac{1}{2} n (n+1)}(y) \end{array} \right)$


The answer to the second question is generic: you are imposing one (differentiable) relationship among $2n$ variables, which therefore describes a $2n-1$ dimensional manifold. For instance, with $n=2$ applying Solve gives

$$y(2)\to \log \left(\frac{e^{2 \theta +x(1)}+e^{2 \theta +x(2)}+2 e^{\theta +x(1)+x(2)}+e^{x(1)+x(2)+y(1)}-e^{2 \theta +y(1)}}{e^{2 \theta }+e^{x(1)+y(1)}+e^{x(2)+y(1)}-e^{x(1)+x(2)}+2 e^{\theta +y(1)}}\right)$$

To see that this is nondegenerate, you can explore the solution space dynamically if you like:

Manipulate[
 ContourPlot[
  Log[(E^(2 t + x1) + E^(2 t + x2) + 2 E^(t + x1 + x2) - E^(2 t + y1) + E^(x1 + x2 + y1))/(
   E^(2 t) - E^(x1 + x2) + 2 E^(t + y1) + E^(x1 + y1) + E^(x2 + y1))], {x1, -1, 1}, {x2, -1, 1}],
 {{y1, 0}, -1, 1}, {t, -1, 1}]

Contour plot

share|improve this answer
1  
In response to a now-deleted comment: these new rules for differentiating an arbitrary finite product are not precisely the familiar "product rule" for derivatives; they are a generalization thereof. Because they apply to a general symbolic limit $n$, they do not follow by means of any finite sequence of operations from the product rule itself ($(fg)' = f'g + fg'$), but rather require an induction. The distinction perhaps is subtle but--as seen in this case--it's real. –  whuber Feb 12 '13 at 4:20
    
Thanks! I picked your answer reluctantly, because I really would have preferred to pick all three: each shows something different and valuable. I figured that it would be better to choose (however inaccurately) than not to, so I finally went with yours on the grounds that it contains most Mathematica-specific stuff... –  kjo Feb 12 '13 at 10:08
    
@whuber x_Except[List] doesn't really make sense to me - and, according to my tests, it doesn't seem to work - perhaps you mean x : Except[List[___]]? –  VF1 Feb 13 '13 at 0:05
    
@VF1 It works fine for me--MMA 8.0. In what fashion do your tests fail? –  whuber Feb 13 '13 at 0:33
1  
Congrats on your 10k! Your answers are always a treat :) –  rm -rf Feb 26 '13 at 3:02

You can get the result you want as follows. This does not really use Mathematica except for purposes of exposition.

I'll truncate in powers of Exp[theta-x] and rewrite that as e[theta-x] to suppress evaluation and make obvious the low order terms of the series. We will throw away the numerator terms because they clearly contribute a multiplicativbe constant with respect to theta. I will separately handle the case where we have the (1 + e[(-x + th)])^2 in the numerator (for y_j's) vs. denominator (for x_j's). For the former we have the function g[x,th] below.

f[x_, th_] := (1 - e[(-x + th)] + e[(-x + th)]^2)^2
g[x_, th_] := (1 + e[(-x + th)])^2

Let's see what it looks like for the case n=1. I'll just call the variables x' andy`.

Expand[f[x, th]*g[y, th]]

(* Out[16]= 1 - 2 e[th - x] + 3 e[th - x]^2 - 2 e[th - x]^3 + 
 e[th - x]^4 + 2 e[th - y] - 4 e[th - x] e[th - y] + 
 6 e[th - x]^2 e[th - y] - 4 e[th - x]^3 e[th - y] + 
 2 e[th - x]^4 e[th - y] + e[th - y]^2 - 2 e[th - x] e[th - y]^2 + 
 3 e[th - x]^2 e[th - y]^2 - 2 e[th - x]^3 e[th - y]^2 + 
 e[th - x]^4 e[th - y]^2 *)

Notice that the first order terms (in powers of e[theta+something], that is) do not vanish unless x==y (the first order term is -2 e[th - x] + 2 e[th - y]). Now suppose we have n>1. What will happen is we'll have, at the lowest order, products of factors of the form

(1 -2 e[th - x[j]] + 2 e[th - y[j]])

It is easy to see that the first order terms will then be, in Mathematica notation,

Sum[-2 e[th - x[j]] + 2 e[th - y[j]], {j,n}]

Now let e[arg_] = Exp[arg]. As we have a power series in powers of Exp[theta-x[j]] and Exp[theta-y[j]], and as it must vanish in theta, in particular the first order terms must vanish in theta. It is straightforward to show that this can only happen if these terms pairwise cancel. (If nothing else, expand as power series in theta at the origin. You can use some polynomial algebra reasoning from there.)

share|improve this answer
    
To be clear: it appears your interpretation of the second question is that it asks for the relationship among the $x_i$ and $y_i$ when the derivative is the zero function; not when the derivative merely vanishes! –  whuber Feb 12 '13 at 3:57
    
@DanielLichtblau: Thanks! This is probably the road to the proof I'm looking for, even though, as you say, it does not use Mathematica much. (See my comment to whuber's answer.) –  kjo Feb 12 '13 at 10:10
    
@whuber: I've added a clarification to my post. –  kjo Feb 12 '13 at 10:44
1  
@whuber Yes, my interpretation was vanishing as a function rather than finding critical points. One reason is that the latter would give a relation involving the xs, ys, and theta rather than just the xs and ys. I guess there was a latter clarification that states the vanishing function interpretation. –  Daniel Lichtblau Feb 12 '13 at 14:33

One trick that hasn't been pointed out yet is this:

Extremizing f is equivalent to extremizing any monotonic function of f. Since your f is positive the product is too. Therefore, we can in particular choose the Log of the product as a convenient monotonic function to extremize. Then the equivalent problem is

f[x_, θ_] := E^(-x + θ)/(1 + E^(-x + θ))^2;
D[Sum[Log@f[x[i], θ] - Log@f[y[i], θ], {i, n}], θ]

$\sum _i^n \left(e^{x(i)-\theta } \left(e^{\theta -x(i)}+1\right)^2 \left(\frac{e^{\theta -x(i)}}{\left(e^{\theta -x(i)}+1\right)^2}-\frac{2 e^{2 \theta -2 x(i)}}{\left(e^{\theta -x(i)}+1\right)^3}\right)-e^{y(i)-\theta } \left(e^{\theta -y(i)}+1\right)^2 \left(\frac{e^{\theta -y(i)}}{\left(e^{\theta -y(i)}+1\right)^2}-\frac{2 e^{2 \theta -2 y(i)}}{\left(e^{\theta -y(i)}+1\right)^3}\right)\right)$

As you see, the derivative has been done without any problems. Setting this to zero is equivalent to the original problem, and the rest just depends on what relationship between the variables you want to extract (i.e., following whuber's or Daniel Lichtblau's interpretation). I'll leave that open.

share|improve this answer
2  
+1 Because $d(\log(f(x))/dx = f'(x)/f(x)$, this is identical to the first formula in my answer (but without the initial factor of the product itself): that's how the formula is usually derived. –  whuber Feb 12 '13 at 4:11
1  
That's right - I simply worked with the OP's statement "I really don't care about the derivative per se," where the prefactor becomes irrelevant to the extremization. –  Jens Feb 12 '13 at 4:19
1  
By ignoring the "prefactor," you could be overlooking some critical points. For instance, consider $\prod_{i=1}^2 x = x^2$ defined on the entire real line. Your formula would set $\sum_{i=1}^2 1/x = 2/x=0$, having no solutions, whereas $x=0$ is the (unique) critical point. Although I said my formula is derived with logs, which is true, in the end--because it does not use logarithms--it is fully general (because the apparent singularities are seen to be removable). –  whuber Feb 12 '13 at 4:24
2  
I think I covered that in my intro. It doesn't apply here. –  Jens Feb 12 '13 at 5:26

Your Answer

 
discard

By posting your answer, you agree to the privacy policy and terms of service.

Not the answer you're looking for? Browse other questions tagged or ask your own question.