8
$\begingroup$

As the title asks, why does

b = a;
f[a_] := b;
f[2]

evaluate to a rather than 2?

$\endgroup$
10
  • $\begingroup$ For me this behaviour is so natural. Just not to be too theoretical, do you know any programming language where the answer would be 2? $\endgroup$
    – yarchik
    Commented Apr 29 at 18:19
  • 1
    $\begingroup$ What is strange here? f is defined to have the value of b and b is defined to have the value of a. The dummy variable "a_" is never used. Therefore, f is always a. $\endgroup$ Commented Apr 29 at 19:16
  • $\begingroup$ @y I'd say it's a variant of the issue I discuss in Final Remark section :) . First of all, we can't assume that askers know about other language, we can't say e.g. "This is quite natural in C! Learn some C first! What? You learned C in school but still don't understand? Then you don't really know C, go back to school and retake the course! " Then, there doesn't seem to exist things like Set/SetDelayed/Evaluate/Unevaluate in traditional languages that don't naturally support symbolic calculation AFAIK, so it's not a very good idea to make an analogy between mma and C etc. in my view. $\endgroup$
    – xzczd
    Commented Apr 30 at 0:14
  • 3
    $\begingroup$ @DanielHuber OK, then why does b = a; f[a_] = b; f[2] evaluate to 2 :) ? $\endgroup$
    – xzczd
    Commented Apr 30 at 0:22
  • 3
    $\begingroup$ Is it really so complicated? $\endgroup$ Commented Apr 30 at 1:57

3 Answers 3

11
$\begingroup$

Caution

The post is long.

If you just want to know how to fix the code, please search "fix the code in title in 10 ways" in the page.

If you want to read discussions about global/local variable, please search Final Remark in the page.


Short Answer

The a hidden in b isn't explicit so a_ cannot notice it.

If you can't understand this short answer, please go on reading.


Long Answer

I've waited for years but don't see a carnonical answer for this in this site, so decide to step up posting this as my 1000th undeleted answer :) .

The Chinese edition of this answer can be found here.

Before dealing with the question in title, let's consider another problem as a start:

Why doesn't the following sample work?

f[max_] := Plot[Sin[a x], {x, 0, max}]
Manipulate[f[max], {a, 1, 5}, {max, 10, 20}]

Reading this, those familiar with document of Manipulate would smile: "This is an example from Possible Issues section of document of Manipulate!" Yeah, and the answer is already documented there:

Manipulate only "notices" explicit visible parameters.

And we need to "redefine f to include the parameter a explicitly":

f[a_, max_] := Plot[Sin[a x], {x, 0, max}]
Manipulate[f[a, max], {a, 1, 5}, {max, 10, 20}]

"OK, so what? This is just kind of bizarreness of Manipulate!" Well, it's not bizare. Actually the explicit existence requirement (forgive me for coining a phrase) is almost ubiquitous in Mathematica. You may seldom notice this, because you can freely use Mathematica without knowing it in most cases (mainly because of automatic evaluation, which is also ubiquitous). But once you've dived deep enough into the core language, the requirement will be nonnegligible. Let's see another example:

Why doesn't

Clear[x,y]
y = x^2;
Module[{x = 5}, x + y]

evaluate to 30?

The sample is short, so Trace is handy enough for checking the evaluation process:

Module[{x = 5}, x + y] // Trace

enter image description here

Remark

Still, if you feel the output of Trace hard to read, consider the tools in this post.

"OK, so what?"… Let's replace Module with Block and compare the outputs:

Block[{x = 5}, x + y] // Trace

enter image description here

By comparing the outputs, we can see the difference: Module doesn't notice the x hidden in y. In other words, just like Manipulate, the localization of Module is also enslaved by the explicit existence requirement.

So does With:

With[{x = 5}, x + y] // Trace

enter image description here

And once the x inside y becomes explicit in some way, Module/With will be able to notice it, for example:

With[{x = 5}, x + y // Evaluate]
(* 30 *)

In this example, Evaluate has temporally penetrated the HoldAll attribute of With, so x + y has evaluated to x + x^2 before going into With. This fix doesn't make much sense though, because localization of Module/With will be broken:

x = aaa; With[{x = 5}, x + y // Evaluate]
(* aaa + aaa^2 *)

The followings are two better fixes to preserve the localization:

Hold@With[{x = 5}, x + y] /. OwnValues@y // ReleaseHold
Unevaluated@With[{x = 5}, x + y] /. OwnValues@y

Remark

  1. With[{x = 5}, x + Evaluate@y] won't work, because Evaluate only has effect on the function next to it, see the document of Evaluate for more info.

  2. With[{y = y}, With[{x = 5}, x + y]] won't work, because the inner x will be renamed. This is also an interesting topic but not quite related to this question so I won't talk about it here, you may start from this post to learn more.

  3. If you want to learn more about Unevaluated, here is a good tutorial.

Now, we see how special the function Block is: even if the variable isn't explicitly there, the localization of Block will succeed. This feature is convenient (Names["*Plot"] functions are all localized with Block internally as far as I know) and dangerous (if you don't use it properly). Since we already have this post about Module, With and Block, I'd like not to talk too much about this topic in this post. Let's go on talking about explicit existence requirement.


Now let's deal with the question in title. To help readers understanding the topic better, I'll extend it to four related questions:

(1.1)

Why does

b = a;
f[a_] := b;
f[2]

evaluate to a rather than 2?

(1.2)

Why does

b = a;
f[a_] = b;
f[2]

evaluate to 2?

(1.3)

Why does

b = a;
f[a_] := Evaluate@b;
f[2]

evaluate to 2?

(1.4)

Why does

b = a;
f[a_] = Unevaluated@b;
f[2]

evaluate to a rather than 2?

"How to understand these? Trace doesn't seem to help much here. " Yeah it's not quite obvious in output of Trace, but we know that, the code f[a_] := b essentially stores something called downvalue in f, and we can use the function DownValues to display the downvalue as a Rule:

b = a; f[a_] := b; DownValues@f

enter image description here

And the effect of downvalue is the same as a usual Rule used in Replace, ReplaceAll, etc. In other words, ignoring the performance difference, code like f[2] can be replaced with Replace[…] in following manner:

f[x_] := x
f[3]
rule = DownValues@f
Clear[f]
Replace[f[3], rule]

enter image description here

f[x_] := x
f[3] + f[4]
rule = DownValues@f
Clear[f]
Replace[f[3], rule] + Replace[f[4], rule]

enter image description here

Remark

As to performance difference, you can clearly see it with

f[x_] := x
f[3] // RepeatedTiming
(* {2.26217*10^-7, 3} *)
rule = DownValues@f;
Clear[f]
Replace[f[3], rule] // RepeatedTiming
(* {5.72101*10^-7, 3} *)

Since it's not quite related to the topic discussed in this post, I'd like not to talk about it here.

So now the question becomes:

Why doesn't

Clear[f, a]; b = a; Replace[f[3], {HoldPattern[f[a_]] :> b}]

evaluate to 3? How can we explain this behavior of pattern matching in a simple and clear way?

Reading all the stuff above, probably you know what I'll say: it's because pattern matching only matches expression explicitly there. Though a a is stored in b in HoldPattern[f[a_]] :> b, it's not explicit. In other words, the a_ in left hand side of :> cannot notice the a hidden in b, so the replacement fails.

OK, then why does b = a; f[a_] = b; f[2] return 2? This is where automatic evaluation comes into play. We know that, the function Set (=) owns attribute HoldFirst (unlike SetDelayed (:=), which owns the attribute HoldAll):

Attributes@Set
(* {HoldFirst, Protected, SequenceHold} *)

In other words, the second argument of Set i.e. right hand side of = will automatically evaluate before it's controlled by Set:

b = a; Trace[f[a_] = b]

enter image description here

This automatic evaluation has stripped the shell (in this case, b) outside of a. Let's check the downvalue in this case:

b = a; f[a_] = b;
f // DownValues
(* {HoldPattern[f[a_]] :> a} *)

As we can see, the a is explicit in right hand side of the :> in this case, so the pattern matching will happen as expected.

The example (1.3) can be explained in a similar manner: though we've used SetDelayed in b = a; f[a_] := Evaluate@b; f[2], the Evaluate has enforced evaluation of second argument so the HoldAll attribute doesn't have any effect on right hand side of := in this case, and the a inside b becomes explicit. This can be seen with Trace and DownValues:

b = a;
Trace[f[a_] := Evaluate@b]
DownValues@f

enter image description here

Remark

Notice that, just like the With[…, Evaluate@…] example above, using SetDelayed doesn't show any advantage compared with Set in this case, because the localization is broken. (But still, SetDelayed has caused a tiny difference in this case, see this post for more info. )

On the contrary, though we've used Set in the example (1.4), the Unevaluated has stopped the automatic evaluation once, so, unlike the example (1.2), this time the shell isn't stripped, and the a inside b doesn't become explicit, so f[2] won't evaluate to 2:

b = a;
Trace[f[a_] = Unevaluated@b]
f // DownValues

enter image description here

Remark

These four examples show us that, when defining a function, one should not blindly rely on Set (=) or SetDelayed (:=) only. To choose between = and :=, you don't need any rule of thumb, just think about the following:

  1. Can the right hand side of =/:= be evaluated instantly?

  2. Should the right hand side of =/:= be evaluated instantly?


For better illustration let's see another three examples:

(2.1)

Why doesn't

2 /. (1 -> a)

evaluate to 2 a?

(2.2)

Why does

1 + 1 /. (2 -> a)

evaluate to a?

(2.3)

Why doesn't

Unevaluated[1 + 1] /. (2 -> a)

evaluate to a?

Again, this can be explained by explicit existence requirement. Being intelligent creatures, we human beings can realize 2 in example (2.1) is equal to 1 + 1, but Mathematica isn't matching in this manner, it only checks if 1 explicitly, clearly, visably, literally exists in the left hand side of /.. 1 won't match 2, just as abc won't match defg.

Remark

If pattern matching is kind of semantic matching then it'll really be a disaster: should 2 match 1 + 1 or 2 × 1 or 2 × 1 × 1…?

"OK, now I understand (2.1), then why does the matching succeed in (2.2)?" Because the automatic evaluation is there. ReplaceAll (/.) is a function without attribute like HoldAll, HoldFirst, etc:

enter image description here

In other words, the arguments of ReplaceAll will evaluate before going into ReplaceAll. Let's Trace:

1 + 1 /. (2 -> a) // Trace

enter image description here

As we can see, 1 + 1 has evaluated to 2, which matches the rule 2 -> a. But for

Unevaluated[1 + 1] /. (2 -> a)

the automatic evaluation of 1 + 1 in left hand side is stopped by Unevaluated, so ReplaceAll doesn't notice the 2 (in semantic sense), and the replacement doesn't happen.

Similarly,

Unevaluated[1 + 1] /. (1 + 1 -> a)

won't evaluate to a, and

Unevaluated[1 + 1] /. (HoldPattern[1 + 1] -> a)

will evaluate to a. The only thing that's worth noticing is, the HoldPattern cannot be replace by Unevaluated in this example, see this post for more info.

Remark

This isn't quite related to the topic discussed in this post, but do you know what will be the output of the following?:

a/. b_. c_ :> c^2

If you feel confused, read this.


As mentioned in the beginning of this post:

The explicit existence requirement is almost ubiquitous in Mathematica. You may seldom notice this, because you can freely use Mathematica without knowing it in most cases (mainly because of automatic evaluation, which is also ubiquitous).

To illustrate the ubiquitousnes of automatic evaluation, I'll add one more example. We know Thread is a useful function, it can be used in the following manner:

Clear[f, a, b, c]
lst = {a, b, c}; Thread[f[lst]]
(* {f[a], f[b], f[c]} *)

But,

lst = {a, b, c};
Thread[Unevaluated@f[lst]]
(* f[{a, b, c}] *)

Why?

Because Thread is also enslaved by the explicit existence requirement. Given that Thread doesn't have any Hold* attribute, its arguments always automatically evaluate unless we intentionally adjust evaluation order (as I've done with Unevaluated above), so we almost never have a chance to notice the influence of explicit existence requirement on Thread in our daily life.

Again, let's Trace:

lst = {a, b, c}; Thread[f[lst]] // Trace

enter image description here

As we can see, f[lst] already becomes f[{a, b, c}] before going into Thread, so Thread can notice the structure therein, but:

lst = {a, b, c}; Thread[Unevaluated@f[lst]] // Trace

enter image description here

In this example, what's going into Thread is just f[lst], the list {a, b, c} stored in lst is not explicit. In the eyes of Thread, the code is just the same as

Clear@lst; Thread[f[lst]]

So the transformation won't happen.


Time to end this post. Well, though I intended to talk about explicit existence requirement, discussions about evaluation order, pattern matching, etc. have involved in, but the various topics about core language are just related to each other, so it's a bit hard for me to discuss them separately. Explicit existence requirement isn't the only possible summary for the behavior discussed in this post, and there exist at least two alternatives:

  1. In document Blocks Compared with Modules, behavior of Module etc. is described as lexical scoping.

  2. In Leonid Shifrin's book Mathematica programming: an advanced introduction, the behavior of pattern matching is described as syntactic rather than semantic comparison of expressions.

These explanations are essentially equivalent to explicit existence requirement in my view. I prefer the explicit existence requirement because it's a straightforward explanation stems from the document of Manipulate. Finally, just for fun, let me fix the code in title in 10 ways:

(* 1 *)
Clear[f]
b = a;
f[a_] = b;
f[2]
(* 2 *)

(* 2 *)
Clear[f, b]
b[a_] := a;
f[a_] := b[a];
f[2]
(* 2 *)

(* 3 *)
Clear[f]
b = a;
(f[a_] := #) &@b;
f[2]
(* 2

(* 4 *)
Clear[f]
b = a;
SetDelayed @@ {f[a_], b};
f[2] *)
(* 2 *)


(* 5 *)
Clear[f]
b = a;
f[a_] := Evaluate@b;
f[2]
(* 2 *)


(* 6 *)
Clear[f]
b = a;
Unevaluated[f[a_] := b] /. OwnValues@b;
f[2]
(* 2 *)

(* 7 *)
Clear[f]
b = a;
Block[{SetDelayed}, f[a_] := b];
f[2]
(* 2 *)


(* 8 *)
Clear[f]
b = a;
Inactivate[f[a_] := b, SetDelayed] // Activate;
f[2]
(* 2 *)

(* 9 *)
Clear[f]
b = a;
Hold[f[a_] := b] /. Language`ExtendedDefinition[b][[1, -1, 1, -1]] // ReleaseHold;
f[2]
(* 2 *)

(* 10 *)
Clear[f]
b = a;
ClearAttributes[SetDelayed, HoldAll]
f[a_] := b;
SetAttributes[SetDelayed, HoldAll]
f[2]
(* 2 *)

Final Remark

More than once, I see someone trying to explain the topic using the concept global variables and local variables. This is not a good idea in my view, because it's replacing the original problem with a new problem that's almost equivalently complicated: how will you explain the meaning of global/local variable? To make these concepts really clear, your explanation will finally contain something similar to explicit existence requirement, this is called xzczd's tenth rule.

$\endgroup$
7
  • $\begingroup$ To the downvoter, I am interested in what's wrong with my answer, would you please elaborate. I'm not trying to complain here, I just want improve my answer if possible. $\endgroup$
    – xzczd
    Commented Apr 30 at 0:20
  • 3
    $\begingroup$ I think this is a good answer, but it doesn't touch on the fundamental concepts behind constructs like Module, With and Block: Module/With are lexical scoping constructs while Block is a dynamic scoping construct. The documentation has a good discussion about this. Other scoping constructs like Function, Table Manipulate and SetDelayed all follow this logic in some way or another. $\endgroup$ Commented Apr 30 at 8:22
  • $\begingroup$ @SjoerdSmit I tried to avoid discussing too much about With, Module and Block, but indeed, I should have mentioned the phrase lexical scoping (which is another equivalent description of explicit existence requirement in my view) at least. Edited. Thx for pointing out! $\endgroup$
    – xzczd
    Commented Apr 30 at 9:00
  • 4
    $\begingroup$ I downvoted purely on the length of the answer to be honest. Way too long to be a useful answer to such a basic question. But I would also down vote based on the content - you try to answer this question without talking about scoping and local/global variables then you are just avoiding the real answer. $\endgroup$
    – Jason B.
    Commented Apr 30 at 12:14
  • $\begingroup$ @JasonB. …Please see the Final Remark section in the end, I talked about local/global variables there. See also my reply to yarchik. I've added some guidance to the question, hopefully it's a bit easier to follow now. $\endgroup$
    – xzczd
    Commented Apr 30 at 13:05
5
$\begingroup$

I thinks it is pretty obvious why the first code returns a and the second 2. You can see it by comparing Definition[f] in both cases.

Clear[a, b, f]

b = a;
f[a_] := b;
f[2]

Definition[f]

Clear[a, b, f]

a

f[a_] := b

Compared with:

Clear[a, b, f]

b = a;
f[a_] = b;
f[2]

Definition[f]

Clear[a, b, f]

2

f[a_] = a
$\endgroup$
3
  • $\begingroup$ I don't think this answers the question. Please see the comments under Danier Huber's answer. $\endgroup$
    – xzczd
    Commented Apr 30 at 10:31
  • $\begingroup$ I think you do not understand difference between local and global variables. a in b=a is global variable while a in f[a_] is local variable. So f[a_] = a returns local value of a, i.e. f[2]=2. But f[a_] := b returns global b which was previously set to value a which is a global variable and this global variable a has nothing to do with local variable a in f[a_]. $\endgroup$ Commented Apr 30 at 11:07
  • 1
    $\begingroup$ This is exactly what I'm discussing in Final Remark section of my answer, see also my reply to yarchik. $\endgroup$
    – xzczd
    Commented Apr 30 at 11:14
4
$\begingroup$

To understand what is going on you need to look at "DownValues" to see what is actually stored:

b = a;
f[a_] := b;
DownValues[f]

{HoldPattern[f[a_]] :> b}

And;

b = a;
f[a_] = b;
DownValues[f]

HoldPattern[f[a_]] :> a}

In the first case, f[2] is evaluated to b and b is evaluated to a.

In the second case we have f[a_]-> a, there fore f[2] is evaluated to 2.

$\endgroup$
4
  • $\begingroup$ But in {HoldPattern[f[a_]] :> b}, isn't the a right there inside b? Why can't the clever Mathematica see it? $\endgroup$
    – xzczd
    Commented Apr 30 at 8:56
  • 1
    $\begingroup$ a_ is a dummy variable. $\endgroup$ Commented Apr 30 at 9:25
  • $\begingroup$ Then what's a "dummy variable"? How is it related to my question? $\endgroup$
    – xzczd
    Commented Apr 30 at 9:48
  • 3
    $\begingroup$ @xzczd It's the design philosophy of mma which enables more general capablilities of symbolic computation and broader expression power and provides more choices to explain your thoughts. $\endgroup$
    – wioiw
    Commented Apr 30 at 10:57

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.