6
$\begingroup$

I don't know if I've just found a bug (and if so, whether there's a workaround), or if I'm just losing my mind. Without further ado, here is my code (really, a minimal working example), run on Mathematica 12.0:

In[1]:= F[j_] := (r^2 + z^2 - 1) Sum[(-1)^k Binomial[2 j + 2, 2 k + 1] z^(2 (j - k) + 1) r^(2 k), 
    {k, 0, j}]

In[2]:= G[j_] := Sum[(-1)^(k + 1) F[k], {k, 0, j}]/(
 4 (j + 1) (j + 2) (2 j + 3))

In[3]:= G[3]

Out[3]= 1/720 (-2 z (-1 + r^2 + z^2) + (-1 + r^2 + z^2) (-4 r^2 z + 
      4 z^3) - (-1 + r^2 + z^2) (6 r^4 z - 20 r^2 z^3 + 6 z^5) + (-1 +
       r^2 + z^2) (-8 r^6 z + 56 r^4 z^3 - 56 r^2 z^5 + 8 z^7))

In[4]:= G[j]

Out[4]= -(((1 + (-1)^j + 2 r^2 - 2 (-1)^j r^2 + r^4 + (-1)^j r^4 - 
    6 r^(4 + 2 j) - 2 j r^(4 + 2 j) + 2 r^(8 + 2 j) + 
    2 j r^(8 + 2 j)) z (-1 + r^2 + z^2))/(
 4 (1 + j) (2 + j) (3 + 2 j) (-1 + r^2)^2 (1 + r^2)^2))

In[5]:= % /. j -> 3

Out[5]= -(((4 r^2 - 12 r^10 + 8 r^14) z (-1 + r^2 + z^2))/(
 720 (-1 + r^2)^2 (1 + r^2)^2))

The point of Out[5] is just to drive home the already-obvious point that Out[4] is not going to be correct. Indeed, it's not just 3: if I plug any (specific) value of j in, I get a correct answer that does not agree with G[j]. And why does G[j] have r's in the denominator anyway?!

I keep wondering if I'm just missing something obvious, which sometimes happen when I stare at code for a very long time.

Any help will be greatly appreciated.

$\endgroup$
4
  • 6
    $\begingroup$ Hi. The title makes it a bit difficult for users to know what the problem is about and for users in the future to find this question. It could be better if keywords like sum and function definition were in the title like "Unexpected result with sums in set delayed functions". $\endgroup$ Sep 15 at 11:11
  • $\begingroup$ For the record, the original title of this post was "Bizarre behavior -- losing my mind", and the view count was 800 after 20 hours, far above average. It was also tweeted and all of that. The title of this post was just replaced by a more sober title, whereby "Bizarre behavior -- losing my mind" has become available for some new question :) $\endgroup$
    – user293787
    Sep 16 at 4:52
  • $\begingroup$ @user293787 ohhh I wondered if it got on social media as I noticed an unusual number of views after only a few hours. I was planning to comment that I change my mind on my first comment given the apparent success of the title then I changed my mind. Not sure what the community bot is or how it managed to change the title to the suggestion I gave. $\endgroup$ Sep 16 at 12:07
  • $\begingroup$ Sorry about my unspecific title! I'm fine with it having been changed. I agree it was unhelpful. $\endgroup$
    – stuart
    Sep 18 at 4:04

2 Answers 2

9
$\begingroup$

To avoid unintended variable capture, one can use

F[j_]:=Module[{k},
         (r^2+z^2-1) Sum[(-1)^k Binomial[2 j+2,2 k+1] z^(2 (j-k)+1) r^(2 k),{k,0,j}]];
G[j_]:=Module[{k},
          Sum[(-1)^(k+1) F[k],{k,0,j}]/(4 (j+1) (j+2) (2 j+3))];

for example. Now

G[3]-(G[j]/.{j->3})//Simplify
(* 0 *)

Concerning r in the denominator, consider the much simpler case of a geometric sum:

Sum[r^k,{k,0,j}]
(* (-1+r^(1+j))/(-1+r) *)

Of course it is a polynomial for every fixed positive integer j.

$\endgroup$
4
  • $\begingroup$ Thank you! This worked! $\endgroup$
    – stuart
    Sep 15 at 9:01
  • $\begingroup$ If you don't mind my asking -- what is variable capture? (I googled it, but did not find anything.) $\endgroup$
    – stuart
    Sep 15 at 9:01
  • 4
    $\begingroup$ I think it is called variable capture;) see this for example. Your code uses "the same" k in both sums. I think that was causing the problem, though I did not analyze how exactly. The Module just make sure that the k in F and the k in G cannot interfere. $\endgroup$
    – user293787
    Sep 15 at 9:04
  • $\begingroup$ Oh I see. Wow. Thank you! $\endgroup$
    – stuart
    Sep 15 at 9:06
9
$\begingroup$

I decided to move my comment from @user293787 to an answer as it made the comments a bit cluttered and made it unlikely that someone else encountering this issue would read it. I emphasize however that the answer that follows relies on user293787's explanation that there was conflict between the two k variables.

A simpler case where that happens :

 Clear[F,G];

 F[j_]:=Sum[j,{k,0,j}];

 G[j_]:=Sum[F[k],{k,0,j}];

 G[3] == G[j]/.j->3

output: (* False *)

TL;DR quick solution : You can use F[j_]= instead of F[j_]:= to avoid that problem.

[Edit: added the solution below that localizes the iterator in a convenient way.

More general solution : we can automatically localize all the iterators by defining a new sum and restricting the name choice for the iterators to belong to a set of one or more names.

Defining a sum that localizes the iterator k :

unique = Inactive@*Unique;
local = k -> unique[k];
sum = ReplaceAll[#, Activate@local] &@*Inactive[Sum]

One could probably include more iterators in "local" using local={k1->unique[k1],k2-> unique[k2],...}. Then instead of using Sum the user may define F and G using the new sum function :

    Clear[F,G]
F[j_]=sum[j,{k,0,j}];
G[j_]= sum[F[k],{k,0,j}];

We can check that the iterators are correct by evaluating G on a symbolic argument:

sum

Finally we may check that the problem has been solved using sum:

G[t] == G[3] // Activate // ReplaceAll[t -> 3]

 (*True*)

End of Edit]

TL;DR Origin of the problem: symbolic handling of F[k] in G[j] where k appears in the argument of F and also in the iterator of the sum in the definition of F.

The solution above that consists of replacing := with = evades the double presence of k by simply evaluating the sum so that k no longer appears (see discussion below if difference between := and = is not clear (nothing fancy)).

The symbolic handling occurs when the upper bound for the iterator in G (the last element of the second argument in Sum) is symbolic but not when it is an integer. This leads to a difference between G[3] and G[j]/.j->3. The solution

The problem is tricky and interesting and it could be quite difficult to debug in a long code.

Explanation

Pictorial summary and reference for the discussion below using traceView2 from this answer (the images and discussion below will focus on the definitions of F and G of the simpler example above and should not be confused with the original F and G from the original question. Also, the discussion below does not require the reader to understand the images below):

G3

G[j]

Consider first G[3].

From traceview2[G[3]] (output shown in first picture), one sees that the evaluation process is as expected:

G[3]=F[0]+F[1]+F[2]+F[3]

Consider now G[j] with a symbolic j. Mathematica does not know the upper bound in this case so it can not do:

F[0]+F[1]+...

Instead, as can be checked using traceView2[G[j]] (second image), Mathematica attempts to do pure symbolic manipulations replacing F[k] by the symbolic expression it gets from the definition of F, that is, the output of F[k], but...

F[k]

evaluates to

Sum[k,{k,0,k}]

which according to Mathematica is k(1+k)/2 but according to Mathematics, it is an invalid request.

The issue there is that maybe Mathematica applies the formula Sum[k,{k,0,n}] then takes n=k at the end where k is a global variable.

So,

Computing F[j], we obtain the expected result j(j+1) and then replacing j by k with /.j->k, we find a result that is different than the result obtained from F[k].


Ok, but why did that not happen with G[3] ?

Attributes[Sum]

output: {HoldAll, Protected, ReadProtected}

For the following mini-discussion that answers the above question,the important component in the above output is HoldAll.

Sum holds F[k] meaning F[k] is not replaced by its definition right away. Instead at the beginning, basically, Mathematica sees the string of characters "F" "[" "k" "]" without checking what is under the hood (at least conceptually not sure what Mathematica is actually doing). Then using the iterator k in the second argument of Sum, Sum successively makes replacements for k, k->0; F[0], k->1; F[1], k->2; F[2], k->3; F[3] and each F[0],F[1],F[2],F[3] will be replaced by its definition (I do not know if replacement rules are actually being used).

When the upper bound is symbolic that sequence can not happen and so F[k] is evaluated with a symbolic k


Solution:

If instead F[j_]=Sum[k,{k,0,j}] (no :=) then there are no surprises because the sum is evaluated before assigning the expression to the function F. Hence, Mathematica sees F[j_]=j(j+1).

So a possible solution for the initial question of OP would be (replaced F[j_]:= with F[j_]=) :

Clear[F,G];
F[j_]=(r^2+z^2-1) Sum[(-1)^k Binomial[2 j+2,2 k+1] z^(2 (j-k)+1) r^(2 k),{k,0,j}];
G[j_]:=Sum[(-1)^(k+1) F[k],{k,0,j}]/(4 (j+1) (j+2) (2 j+3));

Appendix: Difference between F[j_]= and F[j_]:= and when to use one or the other (nothing fancy in the discussion below).

If I remember well, for a somewhat long time working with Mathematica I thought I always had to define a function with the construct F[j_]:=expression.

However, often it is better to use F[j_]= either to avoid surprises like in the above example or to increase the speed of execution as F[j_]:= means nothing is being evaluated until it is called with an argument. For example, compare (* The second one takes 2 seconds before it outputs something*):

Clear[f];AbsoluteTiming[f[j_]:=(Pause[2];Print["that took a long time"])]

Clear[f];AbsoluteTiming[f[j_]=(Pause[2];Print["that took a long time"])]

I use f[j_]:= when a symbolic expression can not be obtained or I need a black box numerical function for certain numerical computations in which case I would use f[j_?NumericQ]:=.

$\endgroup$
7
  • 2
    $\begingroup$ Note that = (as opposed to :=) is also prone to unexpected behavior, as in i=4;(*lots of code*)f[i_]=i;f[12] which returns 4. The root cause of OPs problem is the fact that Sum, Table, Integrate and friends use dynamical scoping for the iteration variable, effectively using Block, which is a deliberate design choice of Mathematica in order to (I think) comfort the casual user that expects f=Exp[-x]; Integrate[f,{x,0,1}] to just work. People that do actual programming in Mathematica should be wary of this and always appropriately localize variables. $\endgroup$
    – user293787
    Sep 16 at 7:42
  • 1
    $\begingroup$ @user293787 Thank you. I would imagine that that example with i=4 is mainly a source of potential errors in the environment of an automatically executed script or package. In an interactive environment such as a notebook, the user is likely to notice the color change of the variable after assigning it a definition. An experienced user would then probably avoid using that letter. The only example in a notebook where I could imagine that problem might occur is when manipulating very complicated expressions where the variables are hidden in the complexity. $\endgroup$ Sep 16 at 12:32
  • $\begingroup$ That said, I was thinking earlier about a convenient way to localize the variables without having to use Module each time which is a bit tedious. I will edit my answer with the solution I found but while we are on the topic of localizing variables, the reader might be interested in this answer that localizes all variables in a cell $\endgroup$ Sep 16 at 12:38
  • $\begingroup$ Thank you very much for all of this. It's interesting and I didn't realize this was how things worked. A philosophical question: was the original behavior a bug? Certainly it seems almost inconceivable that the behavior would ever be desired (but is that wrong?), and so at least as a design issue, I wonder if there should not be some kind of default assumption that iterator-type variables are dummies? But it is reassuring to know that it flows from consistent application of rules, at least, and perhaps there are great reasons for the rules that I do not fully understand. $\endgroup$
    – stuart
    Sep 18 at 4:11
  • $\begingroup$ @stuart Hi, not localizing all the variables might have been a design choice to make it more convenient for the user as user293787 mentioned in the first comment above but I don't know. $\endgroup$ Sep 18 at 4:32

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.