# Enforcing correct variable bindings and avoiding renamings for conflicting variables in nested scoping constructs

Using global variables the following turns an "expression" into a Function:

expr = 2 x;
Function[x, Evaluate[expr]]


Of course this doesn't work if for some reason a value is assigned to x. Thus, I want to turn expr into a Function iside a Module. But it seems like there is a scoping problem:

Module[{x, expr},
expr = 2 x;
Function[x, Evaluate[expr]]]

(* Function[x$, 2 x$1713] *)


Clearly, the x used in the first argument of Functionis not the same as the x in the second. I played around with Evaluate and even with Replace and Slot, also trying to create a function with an anonymous variable, but didn't succeed. The only way that I managed to get the right result is this one involving strings:

Module[{x, expr},
expr = 2 x;
"(" <> StringReplace[ToString@expr, ToString@x -> "#1"] <> ")&" // ToExpression
]


Is there a simpler, more straight-forward way to achieve this, without ever using any global variables?

-
I modified the title because the problem you faced in an instance of a more general and quite important issue. But if you don't like the new one, please feel free to roll back. –  Leonid Shifrin Mar 7 '13 at 0:00
That's totally fine with me. In fact I'm glad that this question turned out to be of this much interest and spawned such a nice and thorough answer. Thank you! –  einbandi Mar 7 '13 at 0:23
Related: (10086), (13757) –  Mr.Wizard Aug 22 '13 at 20:20

### Scoping constructs, lexical scoping and variable renamings

It pays off to understand a bit deeper how the scoping constructs work and what happens behind the scenes when you execute one. In addition to the documentation, this was discussed in part here, but let us present some summary.

When the lexical scoping construct Sc[vars, body] executes (where Sc can stand for constructs such as With,Module, Function), evaluation roughly happens via the following steps (or at least this represents our current understanding of it):

• First, the list of local variables vars is analyzed.
• The body is analyzed, and the presence of inner scoping constructs is tested. This test is performed verbatim, so that, to be detected, some inner scoping construct ScInner has to be present in body in the form ScInner[innerVars, innerBody]. If the inner scoping construct is dynamically generated at run-time (via ScInner @@ ... or otherwise), it is not detected by Sc during this stage.
• If some inner scoping constructs are found where some variables conflict with vars, then Sc renames them. It is important to stress that it is Sc that does these renamings in the inner scoping constructs. Indeed, those are inert during that stage (since Sc has HoldAll attribute and so body is kept unevaluated), so Sc is the only function in a position to do those renamings.
• The actual variable binding happens. The body is searched for instances of vars, and those instances are lexically bound to the variables.
• Depending on the nature of the scoping construct, further actions may be performed. Function does nothing, With performs the replacements of symbols (variables) with their values in body (according to bindings), while Module creates var$xxx variables (according to the bindings, both in the initialization and body), and then performs variables initializations. • The code in body is actually allowed to evaluate. ### How to fool scoping constructs From the above description, it is clear that, if one wants to avoid renamings for some inner lexical scoping construct ScInner[innerVars, innerBody] for whatever reason, one has to dynamically generated this code, so that it is not present inside Sc verbatim. Again, depending on the situation, one may or may not want to evaluate innerVars and innerBody. More often than not one wants to prevent such evaluation, so it is typical to use something like With[{y = boundToZ}, With @@ Hold[{y = z}, boundToZ]]  or With[{y = boundToZ}, Hold[{y = z}, boundToZ] /. Hold -> With]  or anything else that would prevent the innerVars or innerBody from unwanted early evaluation. Here is a more meaningful example. The following is a new scoping construct, which executes some code code with a variable var bind to a current value extracted from a Java iterator object: ClearAll[javaIterate]; SetAttributes[javaIterate, HoldAll]; javaIterate[var_Symbol, code_][iteratorInstance_?JavaObjectQ] := While[iteratorInstance@hasNext[], With @@ Hold[{var = iteratorInstance@next[]}, code] ];  The iteratorInstance is expected to be a Mathematica reference for Java object implementing Iterator interface. The variable var is bound to the current value of the iterator (extracted via iteratorInstance@next[]), using With. This is non-trivial, since we construct this With from pieces, and therefore generate lexical binding of this var to the occurrences of var in code, at every iteration of the While loop. In this case, the outer protecting scoping construct is actually SetDelayed. And we need the construct With @@ Hold[...] to prevent variable var renaming, which is exactly what we don't want here. However, there are cases, where we do want some or all of innerVars or innerBody to evaluate before the binding stage for the inner scoping constructs. The case at hand falls into this category. In such a case, perhaps the most straight-forward way is to use Sc @@ {innerVars, innerBody}, which is what acl did in his answer. ### The case at hand It is now clear why this solution works: Module[{x, expr}, expr = 2 x; Function @@ {x, expr} ]  Function[x$5494, 2 x$5494]  Since there wasn't a line Function[x,...] present verbatim, Module did not detect it. And since we do want the variable and body to evaluate before Function performs the variable bindings, the second version (Function @@ {...}) has been used. You will note that Evaluate is not needed because List is does not have the HoldAll attribute. This specific syntax is not the only approach. For example h[x, expr] /. h -> Function or ReplacePart[{x, expr}, 0 -> Function] would also work because there is not an explicit Function[x, . . .] in the code. It is instructive to realize that this version works too: Module[{x, expr}, expr = 2 x; Function[Evaluate[x], Evaluate[expr]] ]  while Function[...] is present here, the presence of extra Evaluate around x in Function made it impossible for Module to detect the conflict. Of course, there are many other ways one can achieve the same effect. It is now also clear why the following will not work: Module[{x, expr}, expr = 2 x; Function[x, z] /. z -> expr ]  Function[x, 2 x$151]


The point is, the substitution z -> expr happens only at the stage when the body of the Module is evaluated, while the binding stage happens earlier (as described above). During the binding stage, Module detects the name conflict all right, and of course renames. Only then is x converted into a newly created x\$151, and only after all that the code inside Module executes - by which time it is too late since the symbols inside Function and expr are different.

### The case of Block

Block is a natural approach when guarding against global values, but as Szabolcs comments it must be used with care. Block is not seen a scoping construct for the purposes of the automatic renaming described in the tutorial. You can also see some additional relevant discussion here. Because of that you will not get the "protection" that you may be accustomed to. Using Szabolcs's example:

f[z_] := Block[{x, expr}, expr = 2 x + z; Function[x, Evaluate[expr]]]

f[x]

Function[x, 3 x]


Note that this function will triple its argument rather than doubling and adding Globalx which may or may not be what you were expecting. Such injection is often very useful but at the same time if you are accustomed to the automatic renaming behavior (even if you are unaware of the mechanism) this may come as a surprise.

-
+1. The real reason why the version with Function[x, z] /. z -> expr does not work is that the one we are trying to fool here is not Function, but Module. It is Module that performs the renaming inside Function, and it uses verbatim search for Function[something] in the code of Module. This is why things like Function @@ ... work. I think it is important to realize this. –  Leonid Shifrin Mar 6 '13 at 20:50
@Leonid That is the point I was trying to make but I guess I didn't say it very well. I'll try again. –  Mr.Wizard Mar 6 '13 at 20:56
I think it is still not clear enough. Do you mind if I contribute to your answer? –  Leonid Shifrin Mar 6 '13 at 21:02
@acl See if you like the new title I just made for it. –  Leonid Shifrin Mar 6 '13 at 23:59
You've been Leonified! –  Rojo Mar 7 '13 at 5:05

I think this should be OK

Module[{x, expr},
expr = 2 x;
Function @@ {x, expr}
]


(because {x,2*expr} gets evaluated before Function replaces the List head)

-
Thanks! This is clean and easy. I knew I was missing something here... –  einbandi Mar 6 '13 at 18:42
I'd accept @MrW's/@Leonid's answer if I were you. much more thorough –  acl Mar 7 '13 at 0:00

You can also just replace Module with Block:

Block[{x, expr},
expr = 2 x;
Function[x, Evaluate[expr]]]

-
Perhaps it's good to mention that the usual caveat applies: if this is used in a function in a way similar to f[z_] := Block[{x, expr}, expr = 2 x + z; Function[x, Evaluate[expr]]], then f[x] will give an incorrect result. It's necessary to make sure that the function doesn't take symbolic arguments that are used together with the Block-variables. @acl's solution doesn't suffer from this problem. –  Szabolcs Mar 6 '13 at 19:33
Right. Although in some cases that may be also what you want to achieve when evaluating f[x] (i.e., treating x as the function variable). –  ecoxlinux Mar 6 '13 at 19:45

To inject expressions into a function, I use one of the following:

f1 = With[{expr = 2 x},
Function @@ {x, expr}];

f1[5]

(* ==> 10 *)

f2 = Function[x, body] /. body -> (2 x);

f2[5]

(* ==> 10 *)


Modification in light of the comments

The first approach works fine inside Module, the second doesn't, as mentioned by Szabolcs. To fix this, one can do a more comprehensive replacement like this:

Module[{x},
f3 = Function[x1, body] /. {x1 -> x, body -> (2 x)};
]

f3[3]

(* ==> 6 *)

-
Warning! Module[{x, expr, body}, expr = 2 x; Function[x, body] /. body -> expr ] won't make a correct Function. (it was also what I tried) –  Szabolcs Mar 6 '13 at 20:42
@Szabolcs Yes; I tried to address this in my answer below. –  Mr.Wizard Mar 6 '13 at 20:43
@Mr.Wizard I only got to reading it now :) –  Szabolcs Mar 6 '13 at 20:47
@Szabolcs It is Module that we are trying to fool here, not Function, thus this behavior - see my comment below Mr.Wizard's answer. –  Leonid Shifrin Mar 6 '13 at 20:51
@Szabolcs Thanks for pointing that out - I was in a hurry and overlooked how important the Module` part of the question was... –  Jens Mar 6 '13 at 23:26