Take the 2-minute tour ×
Mathematica Stack Exchange is a question and answer site for users of Mathematica. It's 100% free, no registration required.

I have been wrapping my head around this for a while now and I have not found a solution so far. I want to work with an arbitrary number of variables in mathematica and use some built in functions.

To make things more specific for starters I want to do the following:

Define a sum with n summands each containing a new variable x[i] (in the i-th summand):

sum[n_] = Sum[i*x[i], {i, 1, n}]

Then I want to differentiate the expression with respect to some x[i] like:

D[sum[n],x[2]]

Mathematica returns 0 instead of 2.

If I supply a specific n like:

D[sum[2],x[2]]

everything works fine.

I thought about using Assumptions for n, but with no success so far.

How can I do that right?

share|improve this question
    
Evaluating D[sum[n], x[2]] is essentially the same as evaluating D[Sum[n, y], x[2]], where y is value-free, which is nothing D recognizes as depending on x[2]. –  m_goldberg Dec 15 '12 at 16:10
2  
This question is closely related to and might even be considered a duplicate of this one –  m_goldberg Dec 15 '12 at 16:25
1  
Extending Mathematica to allow sums of symbolic length really would require extended it to allow lists of symbolic length; only then could one properly worry about differentiating such objects. But how to make a robust, general design for such an extension is hardly obvious and raises all sorts of issues, e.g.: Should Infinity to an acceptable value for n? What would the effect be upon processing speed for specific-length objects? –  murray Dec 15 '12 at 17:41
1  
I've changed your title to something more specific. please change it to something else if you think it's better –  acl Dec 15 '12 at 19:39
3  
For formal differentiation, what I've noticed is if it's in a tuxedo it's probably a guy, and if it's in a gown it's usually a gal. But this is only a rough guideline, and anyway times have changed. –  Daniel Lichtblau Dec 15 '12 at 22:15

4 Answers 4

up vote 14 down vote accepted

Here is the simplest answer:

sum[n_] := Sum[i x[i], {i, 1, n}]

x /: D[x[i_], x[j_], NonConstants -> {x}] := 
 KroneckerDelta[i, j]

D[sum[n], x[2], NonConstants -> x]

$\begin{cases} 2 & n>1 \\ 1-n & \text{True} \end{cases}$

The trick here is the use of the NonConstants option of the derivative operator. This then has to be combined with a definition stating that the x[i] are independent variables for the purposes of this differentiation (hence the KroneckerDelta on the second line).

Edit: more discussion

And here is another cool result, completely symbolic:

Assuming[m ∈ Integers, D[sum[n], x[m], NonConstants -> x]]

$\left( \begin{array}{cc} \{ & \begin{array}{cc} m & m\geq 1 \\ 0 & \text{True} \\ \end{array} \\ \end{array} \right)-\left( \begin{array}{cc} \{ & \begin{array}{cc} m & m>2\land m\geq n+1 \\ n+1 & m=2\land n=1 \\ \end{array} \\ \end{array} \right)$

This isn't easy to absorb, but it works if you check it with specific examples by doing

condition = %;

Simplify[condition /. m -> 10]

$\begin{cases} 10 & n>9 \\ 0 & \text{True} \end{cases}$

In summary, it's worth pointing out that a lot of symbolic differentiation tasks can be achieved by using either NonConstants specifications in D or conversely using Constants specifications in Dt.

share|improve this answer
    
This is really helpful. Thank you very much. –  Wizard Dec 22 '12 at 17:03

One solution might be ...

Method 1.

Define some variables:

x = Table[Unique[], {5}];

Form the inner product and differentiate:

D[Inner[Times, x, Range@Length@x], x[[2]]]

2

Or if you prefer it in a functional form:

sum[n_] := Inner[Times, x[[1 ;; n]], Range@n] /; ( Length@x >= n)

D[sum[4], x[[3]]]

3

Method 2.

You could take a different approach, but you need to be aware that it leaks variables of the form {x1,x2,x3,...} out into the general context:

sum[n_] := Inner[Times, Symbol["x" <> ToString@#] & /@ Range@n, Range@n] 

sum[5]

x1 + 2 x2 + 3 x3 + 4 x4 + 5 x5

D[sum3[4], x3]

3

share|improve this answer
    
This is exactly what I was trying to avoid. The exact amount of variables should not have to be specified. –  Wizard Dec 22 '12 at 17:04

I like image_doctor's solution better, but how about using Array and looking for the position using that index each time? Like this:

xx = Array[x, 10, 1];
sum[n_] := Times[List @@ xx , Range[10]]

sum[n]
(* {x[1], 2 x[2], 3 x[3], 4 x[4], 5 x[5], 6 x[6], 7 x[7], 8 x[8], 9 x[9], 10 x[10]} *)

Now

sum[n_] := Times[List @@ xx^4, Range[10]]
D[sum[n], xx[[5]]][[5]]
(*  20 x[5]^3 *)

and

sum[n_] := Times[List @@ xx, Range[10]]
D[sum[n], xx[[2]]][[2]]

(* 2 *)

It is kinda clumsy though. The problem, of course, is that one can't treat

enter image description here

as a single variable as we do on paper. P.S., I tried to see if one can do this in Maple and could not do it directly as you wanted. Had to do a hack as above. (But I do not know Maple much, though.)

share|improve this answer
1  
Nasser, if it matters to you, your "gravatar" is changing along with your I.P. address. You can make it a fixed one by entering an email address in your profile. This email is only visible to moderators and furthermore it doesn't need to be a real one anyway. Also see: mathematica.stackexchange.com/q/13525/121 –  Mr.Wizard Dec 15 '12 at 19:42

I did some computation of formal derivatives a while back which might be of interest in this context (though keep in mind that this is anything but bullet proof! it does work for the cases I bothered to check though).

Clear[a]; Format[a[k_]] = Subscript[a, k]

Let us say we have an objective function which is formally a function of the vector a[i]

Q = Sum[Log[Sum[a[r] Subscript[B, r][Subscript[x, i]], {r, 1, p}]/
    Sum[a[r] , {r, 1, p}]], {i, 1, n}]

Mathematica graphics

Let us define a couple of rules for formal differentiation

Clear[d];
d[Log[x_], a[k_]] := 1/x d[x, a[k]]
d[Sum[x_, y__], a[k_]] := Sum[d[x, a[k]], y]
d[ a[k_] b_., a[k_]] := b /; FreeQ[b, a]
d[ a[q_] b_., a[k_]] := b Subscript[δ, k, q] /; FreeQ[b, a]
d[ c_  b_, a[k_]] := d[c, a[k]] b + d[b, a[k]] c
d[ b_ +  c_, a[k_]] := d[c, a[k]] + d[b, a[k]]
d[Subscript[δ, r_, q_], a[k_]] := 0
d[x_, a[k_]] := 0 /; FreeQ[x, a]
 d[G_^n_, a[k_]] := n G^(n - 1) d[G , a[k]] /; ! FreeQ[G, a]
 d[Exp[G_], a[q_]] := Exp[G] d[G , a[q]] /; ! FreeQ[G, a]

Unprotect[Sum]; Attributes[Sum] = {ReadProtected};Protect[Sum];

And a rule to deal with Kroneckers

ds = {Sum[a_ + b_, {s_, 1, p_}] :> Sum[a, {s, 1, p}] + Sum[b, {s, 1, p}],
      Sum[ y_ Subscript[δ, r_, s_], {s_, 1, p_}] :> (y /. s -> r),
  Sum[ y_ Subscript[δ, s_, r_], {s_, 1, p_}] :> (y /. s -> r),
  Sum[ Subscript[δ, s_, r_], {r_, 1, p_}] :> 1,
  Sum[δ[i_, k_] δ[j_, k_] y_. , {k_, n_}] -> δ[i, j] (y /. k -> i),
  Sum[a_ b_, {r_, 1, p_}] :> a Sum[b, {r, 1, p}] /; NumberQ[a]}

Then the gradient of Q with respect to one of the a[k] reads

grad = d[Q, a[k]]  /. ds // Simplify;

Mathematica graphics

Similarly the tensor of second derivatives w.r.t. a[k] and a[s] is

hess = d[d[Q, a[k]], a[s]] /. ds // Simplify

Mathematica graphics

As a last less trivial example let us consider the 4th order derivatives of Q

 d[d[d[d[Q, a[k]], a[s]], a[m]], a[t]]; /. ds // Simplify

Mathematica graphics

For the problem at hand we check easily that

 Q = Sum[r a[r] , {r, 1, p}];

 grad = d[Q, a[k]] // Simplify;
 grad //. ds

returns k as it should

EDIT

This process can be made a bit more general, say, on this Objective function

  Q = 1/2 Sum[(Sum[a[r] Subscript[B, r, i][a[q]], {r, 1, p}] - 
      Subscript[y, i])^2, {i, 1, n}]

Mathematica graphics

which depends non linearly on a[k] via B.

All we need is to add a new rule for d

d[H_[a[q_]], 
  a[k_]] := (D[H[x] , x] /. x -> a[k] ) Subscript[δ, k, q]

Now we readily have

grad = d[Q, a[k]] // Simplify;
hess = d[d[Q, a[k]], a[s]];

grad //. ds

Mathematica graphics

hess /. ds // Simplify

Mathematica graphics

As a other example, let us look at a parametrized entropy distance,

 Q = -Sum[(Sum[a[r] Subscript[B, r, i], {r, 1, p}]/
  Subscript[y, i]) Log[(Sum[a[r] Subscript[B, r, i], {r, 1, p}]/
   Subscript[y, i])], {i, 1, n}]

Mathematica graphics

we can compute its Hessian while mapping twice the sum rule

 Map[# /. ds &, d[d[Q, a[k]], a[s]] /. ds]

Mathematica graphics

As a final example, consider a Poisson likelihood

 Q = Sum[Log[Exp[-a[k]] a[k]^Subscript[y, k]/Subscript[y, k]!], {k, 1, n}]

Mathematica graphics

so that

 grad = d[Q, a[k]] // Simplify

Mathematica graphics

and

  hess =d[d[Q, a[k]], a[s]]  /. ds // Simplify

Mathematica graphics

share|improve this answer
    
@Nasser have you seen this? –  Mr.Wizard Dec 15 '12 at 19:51

Your Answer

 
discard

By posting your answer, you agree to the privacy policy and terms of service.

Not the answer you're looking for? Browse other questions tagged or ask your own question.