# Variable scoping problem when mapping over delayed replacement

This is something that got me curious while I was playing around with Mathematica. Consider the following (contrived) example:

Clear[x,y];
{{1, 2}, {3, 4}, {5, 6}} /. {x_, y_} :> {x, #} & /@ {y, y^2, y^3}
(* {{{1, 2}, {3, 4}, {5, 6}}, {{1, 4}, {3, 16}, {5, 36}}, {{1, 8}, {3,
64}, {5, 216}}} *)


No problems there.

Now suppose y has been defined globally, so

y = 10;
{{1, 2}, {3, 4}, {5, 6}} /. {x_, y_} :> {x, #} & /@ {y, y^2, y^3}
(* {{{1, 10}, {3, 10}, {5, 10}}, {{1, 100}, {3, 100}, {5, 100}}, {{1,
1000}, {3, 1000}, {5, 1000}}} *)


The problem is that y gets evaluated to 10 before getting mapped to the delayed rule's rhs.

I realised I could do

{{1, 2}, {3, 4}, {5, 6}} /. {x_, y_} :> {x,
ReleaseHold[#]} & /@ Thread[Hold[{y, y^2, y^3}]]
(* {{{1, 2}, {3, 4}, {5, 6}}, {{1, 4}, {3, 16}, {5, 36}}, {{1, 8}, {3,
64}, {5, 216}}} *)


which looks kinda complicated, but works.

So, is there any better way to "shield" the y in the list being mapped over, from the global variable y?

I sort of randomly tried
Module[{y}, {{1, 2}, {3, 4}, {5, 6}} /. {x_, y_} :> {x, #} & /@ {y, y^2, y^3}]

but in this case the temporary variable's name(?) got substituted

(* {{{1, y$119596}, {3, y$119596}, {5, y$119596}}, {{1, y$119596^2}, {3,
y$119596^2}, {5, y$119596^2}}, {{1, y$119596^3}, {3, y$119596^3}, {5, y$119596^3}}} *)  - This is a precedence problem. Use {x_, y_} :> ({x, #} & /@ {y, y^2, y^3}) instead. – Leonid Shifrin May 16 '13 at 14:21 @LeonidShifrin For example, consider the example where I first ran across this problem: Graphics@BezierCurve[{{0, 0}, {1, 1}, {2, -1}, {3, 0}, {5, 2}, {6, -1}, {7, 3}}, SplineDegree -> 3] /. {x_, y_} :> {x, #} & /@ {y, y^2, y^3} – Aky May 16 '13 at 14:27 @LeonidShifrin I'd actually tried that, but if you look carefully, the output isn't right - unless I'm missing something. I realize you could take the transpose of the result (in this specific case), but I don't think that would always work(?) IIUC correctly, using parantheses that way amounts to modifying the pure function.. – Aky May 16 '13 at 14:38 Ok, I see. You want to generate rules programmatically. I think this is going to be tough no matter how you slice it. One thing that comes to mind: Cases[Hold[y, y^2, y^3], el_ :> RuleDelayed @@ Hold[{x_, y_}, {x, el}]], but this is not exactly simple. – Leonid Shifrin May 16 '13 at 14:43 As David Carraher says, being careful with names can help in situations like this. Consider using formal symbols in your RuleDelayed expression. – m_goldberg May 16 '13 at 16:33 ## 2 Answers You have already seen that there are a number of ways to skin this cat but I'd like to add some comments of my own. ## Other approaches When possible I prefer to avoid these situations in the first place, instead using something like: {{1, 2}, {3, 4}, {5, 6}} /. {x_, y_} :> Thread@{x, {y, y^2, y^3}} // Thread  {{{1, 2}, {3, 4}, {5, 6}}, {{1, 4}, {3, 16}, {5, 36}}, {{1, 8}, {3, 64}, {5, 216}}}  Here: 1. Pattern names x and y remain local 2. Pattern matching is done only once, rather than three times You can also avoid interactions with global symbols either by using Block or Formal Symbols. Formal Symbols have the Protected attribute and exist expressly for avoiding Imperial Global entanglements. Block requires foreknowledge of the names of the troublesome symbols, and must be updated any time they are to avoid introducing a bug. Example of Block usage: Block[{y}, {{1, 2}, {3, 4}, {5, 6}} /. {x_, y_} :> {x, #} & /@ {y, y^2, y^3} ]  ## Substitutions inside RuleDelayed Regarding substitutions into the right-hand side of RuleDelayed there is an important caveat: Mathematica transparently renames variables in nested scoping constructs, including RuleDelayed. Observe: Cases[Hold[y, y^2, y^3], foo_ :> {x_, y_} :> {x, foo}]  {{x$_, y$_} :> {x$, y}, {x$_, y$_} :> {x$, y^2}, {x$_, y$_} :> {x$, y^3}}


Note that the pattern names on the LHS have changed to x$ and y$ and the RHS does not match. For this reason the best way to make such substitutions is often using a pure function with Slot or SlotSequence (# and ##) parameters which does not cause such renaming to take place. However, pure functions by default will evaluate arguments so you must guard against this. One way is to use Hold and ReleaseHold as you did. Another is to use Unevaluated on each individual argument:

List @@ ({{1, 2}, {3, 4}, {5, 6}} /. {x_, y_} :> {x, #} & /@
Unevaluated /@ Hold[y, y^2, y^3])


You can also use an undocumented syntax of Function that combines the unnamed Slot parameters with the ability to specify function Attributes:

Function[Null, body, attributes]


Null can be implicit, therefore:

Function[, {{1, 2}, {3, 4}, {5, 6}} /. {x_, y_} :> {x, #}, HoldAll] /@
Unevaluated[{y, y^2, y^3}]


Note that in this case Unevaluated is used to prevent evaluation not in our Function (as above), but rather in Map (/@).

It is possible to wrestle Mathematica into making the substitution without renaming as described in the linked answer above, e.g.:

rls = Cases[Hold[y, y^2, y^3], foo_ :> RuleDelayed @@ Hold[{x_, y_}, {x, foo}]]

{{x_, y_} :> {x, y}, {x_, y_} :> {x, y^2}, {x_, y_} :> {x, y^3}}


Combined with the syntax expr /. {{rules1}, {rules2}, {rules3}} to apply multiple rule lists we may write:

{{1, 2}, {3, 4}, {5, 6}} /.
Cases[Hold[y, y^2, y^3], foo_ :> {RuleDelayed @@ Hold[{x_, y_}, {x, foo}]}]

{{{1, 2}, {3, 4}, {5, 6}}, {{1, 4}, {3, 16}, {5, 36}}, {{1, 8}, {3, 64}, {5, 216}}}

-
Thanks for the great answer... I'll have to study it carefully. – Aky May 18 '13 at 11:08

I'll just aggregrate here the various responses from the comments, plus my own humble thoughts and understanding (which may lack precision or contain mistakes, so feel free to improve or correct):

• Be careful about variable naming. (This is assuming that, generally speaking, you have good reason to have global variables spilling out in your program.)

• Use Block (but not Module) to restrict the troublesome variable to a local scope.

If I understand correctly, Module fails here because it works by renaming variables to avoid conflicts. So in our case, the y everywhere in our list gets renamed (to some y\$n where n is a random number with several digits) before it becomes local to the delayed rule. Variables within the rule are local to it, hence don't require renaming in the first place. (At least this seems to explains why Module[{y}, {1,2,3}/.y_:>y^2] "works" whereas Module[{y}, {1, 2, 3} /. y_ :> # & /@ y^2] fails.)

-

• Leonid's suggestion, which involves generating a delayed rule by Applying over a held expression

ruleslist = Cases[Hold[y, y^2, y^3], el_ :> RuleDelayed @@ Hold[{x_, y_}, {x, el}]] (* {{x_, y_} :> {x, y}, {x_, y_} :> {x, y^2}, {x_, y_} :> {x, y^3}} *)

{{1, 2}, {3, 4}, {5, 6}} /. # & /@ ruleslist (* {{{1, 2}, {3, 4}, {5, 6}}, {{1, 4}, {3, 16}, {5, 36}}, {{1, 8}, {3, 64}, {5, 216}}} *)

• Holding the list of expressions and then ReleaseHolding them once they've been mapped into the rhs of the delayed rule

{{1, 2}, {3, 4}, {5, 6}} /. {x_, y_} :> {x, ReleaseHold[#]} & /@ Thread[Hold[{y, y^2, y^3}]]

Edit: I forgot to mention m_goldberg's excellent suggestion of using formal symbols - however, Mr. Wizard's noted it in his answer.

-