Scoping of pattern variables [duplicate]

Question

I have run into a problematic behavior, and then came to find that it is a "possible issue" called out in the documentation, but I guess I am just baffled that this IS an issue, and am wondering why. The example from the documentation is:

In the presence of global variables, pattern variables may show unexpected behavior:

x=5;
f[x_]=x^2;
f[2]

The result of this is 25. I am utterly shocked that Mathematica doesn't recognize that within the definition of f, it should treat x as local. Every other programming language I have used recognizes this. And I swear that earlier versions of Mathematica recognized this, though I suppose I could have never tested it.

Why is this the behavior? Is there any way to fix the problem without resorting to a delayed assignment?

Edited to add: Many people are suggesting memoization and/or simply understanding the execution model of Mathematica. Unfortunately, the situations where I am decrying this behavior is not helped by memoization. Imagine a function f[x_,y_,z_,n_,l_,m_,p1_,p2_,p3_] that is defined over all real values of x, y, and z, and for some subset of integers n, l, and m, and applies to different systems based on real-valued parameters p1, p2, and p3. The function itself is complicated, and computationally intensive. Often this function is then combined into a composite function that still depends on the same variables and parameters, but which has multiple offsets (so g is f[x-x1,y-y1,z-z1,...] added together for multiple offsets). And then I need to integrate g over x, y, z to normalize it and then I need to ContourPlot3D the resulting function. In order to make the integration and plotting not take forever, I need Mathematica to do as much preprocessing simplification as possible, which means using Set instead of SetDelayed. The calculations are over all 3D space, so memoization doesn't help.

Further, the applications are part of a college class I teach that uses Mathematica as a tool, but isn't itself about Mathematica, so the goal is to give the students a just-in-time introduction to the Mathematica skills they need to solve the content-specific problems they are being tasked with. So, I give them code that they can copy-and-paste to do complicated stuff, and then expect them to modify calls to that code to apply the concepts to new problems. And here's the issue... if I have a parameter that all of the textbooks call De, then it makes sense to define the function with De as the name of the parameter in order to make calling that function more transparent. And it also makes sense for the students to plug in their values for De into a variable they name De. Many of these students have a tiny bit of programming experience before my class, and so expect the behavior to be similar to the languages they have used before. And in many, many cases that expectation is reasonable. But not in this case, which is why I asked my question. I cannot afford to devote significant time in class to the execution model of Mathematica when for nearly every other purpose a student approaching Mathematica as a procedural language is sufficient for my course.

So thank you all for your responses. And I apologize for venting.

Answer 1

Despite some thoughts here, you can localize a pattern variable without switching to SetDelayed:

ClearAll[f, x];
x = 1;
Module[
 {y = 0},
 f[x_] = x^(1 + 1) + y
];
??f
(*f[x$_]=x$^2*)

You can read more about scoping in Mathematica in this answer.

Answer 2

There are many people at this forum who are more knowledgeable about Mathematica than me, but I will attempt an answer anyway. (Believe me, I have done worse things in life.)

Why is this the behavior? Is there any way to fix the problem without resorting to a delayed assignment?

The delayed assignment is the way to fix the "problem," if there was ever a problem at all.

Suppose I have a computationally-intensive calculation that I want done once...

Remarkably, the very property of Wolfram Language that you are decrying is the property that allows one to memoize functions trivially--far more easily than in other programming languages. You can ensure that a function definition is called only once for a given value by following the delayed definition with an assignment. Here is an example:

ClearAll[f];
f[x_] := f[x] = (
    Print["Computing f[", x, "] for the first time, and the answer is"];
    x + 1
);

Now try calling f[1], f[1], f[2], and f[2]. Here is what you will see:

f[1]
Computing f[1] for the first time, and the answer is
2

f[1]
2

f[2]
Computing f[2] for the first time, and the answer is
3

f[2]
3

Notice that every time the function f is given a new value, it invokes the (delayed) definition. But when f is given a value that it has already seen before, it does not re-invoke the definition! It simply looks up the result in a giant lookup table (called DownValues).

Why? That is the "magic" of the second part of the function definition:

f[x_] := f[x] = (* Body of function *)

Mathematica works by repeatedly looking up the shape, or pattern, of an expression in DownValues, always looking for the most restrictive pattern that is applicable, until it can no longer resolve the expression further. So when Mathematica is asked to evaluate a function like f[1], it says to itself, "What is the most restrictive pattern that I can find in DownValues that matches f[1]?" Then Mathematica notices that it does not have an entry in DownValues for f[1], so it says to itself, "Well, I am going to have to evaluate the more general expression f[something] where the something is equal to 1."

But note that the delayed definition of f[x_] is actually

f[x] = (Print["Something"]; x + 1);

Since the return value of an assignment, or the Set command, is just the value of the assignment, Mathematica returns the value 2 while storing the key-value pair (f[1], 2) in DownValues.

Now, the next time Mathematica is asked to evaluate f[1], it again asks itself, "What is the most restrictive matching pattern for f[1] that I can find in DownValues?" But this time the answer is f[1], because f[1] is more specialised than f[something] and we told Mathematica to store the key-value pair (f[1], 2) in DownValues after the first invocation. Thus, in the case of the second call to f[1], Mathematica finds a direct match of the more restricted pattern f[1] in DownValues and simply returns the result 2 without going through the delayed definition and printing the string.

And so on. By keeping this simple execution model in mind, you can avoid a lot of confusion.