Function incorrectly renaming variables in Rule Pattern

Question

Consider the simple pure Function containing a Rule with the named pattern q.

f = Function[{t}, q_ -> t]

When this function is called, it renames the pattern name to q$:

f[2]

>>> q$_ -> 3

This is quite strange, since there was clearly no variable collision in the function definition:

f
>>> Function[{t}, q_ -> t]

The documentation for Function includes:

The named formal parameters xi in Function[{x1,…},body] are treated as local, and are renamed xi$ when necessary to avoid confusion with actual arguments supplied to the function.

This doesn't seem to explain what's happening here, since an enclosed pattern q was renamed, instead of the "named formal parameters" t (which didn't need remaining anyway!).

Naturally passing symbols to the function shows the same behaviour:

f[a]
>>> q$_ -> a

however passing a symbol which happens to be the function variable (in this case t) prevents the renaming!

f[t]
>>> q_ -> t

Funnily enough, Function doesn't even rename the formal or pattern variables when they actually do collide (though of course, which symbol is which cannot be determined)!

f = Function[{t}, t_ -> t]
>>> Function[{t}, t_ -> t]

f[2]
>>> t_ -> t

What's going on here? What determines Function's variable renaming (and why does it include scoped expressions despite the doc's explanation)? Is this a bug?

Furthermore, how can I prevent or revert this renaming? I wish to display the enclosed rule q_ -> t verbatim, when the function is called.

I'm using version "11.2.0 for Mac OS X x86 (64-bit) (September 11, 2017)"

Answer 1

I think this is a product of Mathematica's lexical-scoping algorithm being efficient and just scoping anything that comes into comes into contact with a scoped variable (i.e. t in this case).

Here's some data that may or may not back that up. First, the case at hand

Function[{t},
  {
   q_ -> {q, t},
   z_ -> x,
   z_ :> x,
   t_ :> t
   }
  ][1]

{q$_ -> {q$, 1}, z_ -> x, z_ :> x, t_ :> t}

We see that for the other scoped symbols, nothing happens. Interestingly, Function is not doing value injection, since the second t isn't affected.

Compare that to a regular function definition

meh[t_] :=
  {
   q_ -> {q, t},
   z_ -> x,
   z_ :> x,
   t_ :> t
   };
meh[1]

(* RuleDelayed::rhs: Pattern t_ appears on the right-hand side of rule meh[t_]:>{q_->{q,t},z_->x,z_:>x,t_:>t}. *)

{q$_ -> {q$, 1}, z_ -> x, z_ :> x, Pattern[1, _] :> 1}

We see clearly that the same lexical scoping happens, but in this case the t value is injected everywhere.

Now trying this with With, which should do pure injection

With[{t = 1},
 {
  q_ -> {q, t},
  z_ -> x,
  z_ :> x,
  t_ :> t
  }
 ]

{q$_ -> {q$, 1}, z_ -> x, z_ :> x, t_ :> t}

we see that the t is actually untouched. My hypothesis is that the lexical scoping algorithm has a set of heads that it doesn't touch, and the LHS of Rule is one of them. I know Rule and RuleDelayed have a lot of special properties like SequenceHold. This might just be another one of those.

For the purest injection possible, we can use ReplaceAll

ReplaceAll[
 {
  q_ -> {q, t},
  z_ -> x,
  z_ :> x,
  t_ :> t
  },
 t :> 1
 ]

{q_ -> {q, 1}, z_ -> x, z_ :> x, Pattern[1, _] :> 1}

this does no scoping and just replaces t wherever it sees it.

Just to round things out, Block/Module behave exactly like With, which is unsurprising given that Block respects Hold and the scoping algorithm used in With is the same one as in Module (I think) but with different scoped names.

Block[{t = 1},
 {
  q_ -> {q, t},
  z_ -> x,
  z_ :> x,
  t_ :> t
  }
 ]

{q_ -> {q, 1}, z_ -> x, z_ :> x, t_ :> t}

Module[{t = 1},
 {
  q_ -> {q, t},
  z_ -> x,
  z_ :> x,
  t_ :> t,
  t_ -> t
  }
 ]

{q$_ -> {q$, 1}, z_ -> x, z_ :> x, t_ :> t, t_ -> t}

This is not a definitive answer by any means, but I think it provides some data in favor of it just being the way Mathematica does lexical scoping. I'm guessing that this is just an efficient way to do this, given that the internal MExpr object that represents Mathematica expressions (totally conjecture that there is one, but conjecture based on lots of hints and tips from WRI employees) can be treated as a tree and therefore can probably efficiently tell you what symbols are in and expression, but can't exactly tell you where. Easier to scope all of the symbols than to walk the tree and replace only the ones touched by the scoping construct explicitly.

One final bit of input, I think Function is smart when it's given t as an input and simply returns the function body without scoping. This is just an efficient short-cut for a common case. Here's my data for that statement

Function[
  {t},
  {
   q_ -> {q, t},
   z_ -> x,
   z_ :> x,
   t_ :> t
   },
  HoldAll
  ][t]

{q_ -> {q, t}, z_ -> x, z_ :> x, t_ :> t}

t = 5;
Function[
  {t},
  {
   q_ -> {q, t},
   z_ -> x,
   z_ :> x,
   t_ :> t
   },
  HoldAll
  ][t]

{q_ -> {q, 5}, z_ -> x, z_ :> x, t_ :> t}

as long as it sees t explicitly as input, it doesn't scope q.

Answer 2

It may not be obvious at first sight but Function[{t},q_->t] is a nest of automatic name scoping constructs: Function outside and Rule inside. The list of scoping constructs is given in Modularity and the Naming of Things, referred to by MarcoB above. See also Scoping Constructs, referred to in the documentation of Function.

In Function, renaming applies to formal parameters (the first argument of Function). In Rule, renaming applies to pattern names. Renaming is also known as α-conversion.

It may be more clear to rewrite your example as

In[2]:= Function[t,Rule[u_,x[t,u]]]@v
Out[2]= u$_->x[v,u$]

where x is any function, truly depending on t (in your example x[t_,u_]:=t).

The Wolfram language renames "more liberally than is strictly necessary". In my example, renaming would be necessary only if Unevaluated@u==v, depending on the value of v. The Wolfram language does not check this condition, it always renames as soon as a possibility of name conflict or collision exists (x depends on t), so it is lazy and excessively cautious but "faithful" to partial function "semantics".

The documentation is somehow laconic and euphemistic. When it says "By convention, x$ is never used as a global name.", this actually means that you can use x$ but it will never be renamed, at your own risk.

Coming back to the question, thus it is possible to avoid renaming like this:

In[9]:= Function[{t},q$_->t]@2
Out[9]= q$_->2

where q$ has not been renamed q$$. Indeed, renamed q$ is q$, renaming is idempotent.

This has two interests: it provides a neat way to deactivate renaming and (for the WL developers) it simplifies the programming of scoping constructs (and possibly improves the performance).

This is funny too:

In[10]:= Function[{t},q$$_->t]@2
Out[10]= q$$$_->2

Answer 3

In[28]:= q_ -> # &@2
Out[28]= q_ -> 2