10
$\begingroup$

At its heart, Mathematica is a dynamically scoped language. While the choice of dynamic scope for Mathematica is very much a defensible one, lexical scope is too useful to do without, and Mathematica tries to fake it. Unfortunately, the abstraction is leaky.

One of the leaks comes is visible in constructs like With and Function, where symbols naming formal parameters sometimes must be changed to avoid the possibility of clashes. An example of this is:

In[1]:= With[{fun = Function[x, Take[x, 2]]},
         gun[x_] := fun[x];

         gun[{1, 2, 3}]]
Out[1]= {1, 2}

This is exactly what we wanted to have happen, and we can see how Mathematica pulled it off:

In[2]:= DownValues[gun];
Out[2]= {HoldPattern[gun[x$_]] :> Function[x, Take[x, 2]][x$]}

The formal parameter x for gun was changed to x$, so it does not clash with the formal parameter of the anonymous function bound to fun by the With statement.

Now, say we do this sort of thing often enough that we abstract out the construction of that anonymous function:

In[3]:= makeFun[len_] :=
         Function[x,
          Take[x, len]];

We can see signs of potential danger already:

In[3]:= makeFun[2] 
Out[3]= Function[x$, Take[x$, 2]]

The x is already being renamed, and the renaming could clash with other renamings. We can test this:

In[4]:= With[{fun = makeFun[10]},
         hun[x_] := fun[x];

         hun[1]]
Function::flpar: Parameter specification 1 in Function[1,Take[1,10]] should be a symbol or a list of symbols. >>
Function::flpar: Parameter specification 1 in Function[1,Take[1,10]] should be a symbol or a list of symbols. >>
Out[4]= Function[1, Take[1, 10]][1]

Ouch. Checking DownValues indicates that the predicted name clash did indeed come to pass:

In[5]:= DownValues[hun]
Out[5]= {HoldPattern[hun[x$_]] :> Function[x$, Take[x$, 10]][x$]}

Is there any way (perhaps with the right suite of init.m tweaks) to keep this from happening? It really compromises the ability to treat functions as first class objects in a modular way.

FWIW, this is something I saw for the first time in Mathematica 10.2. I have vague memories of doing this sort of thing is the past without particular difficulty, but I could well be mistaken on that count.

$\endgroup$
10
$\begingroup$

One possible workaround is to "turn off" the name rewriting that Function does is makeFun, and provide a name for the formal parameter you know is unique. There are several ways of doing this, but this one is mine:

In[6]:= Module[{x, function},
          Attributes[function] = HoldAll;

          makeFun[len_] :=
           function[x,
            Take[x, len]] /. function -> Function];

This works:

In[7]:= With[{fun = makeFun[2]},
         pun[x_] := fun[x];

         pun[{1, 2, 3}]]
Out[7]= {1, 2}

In[8]:= DownValues[pun]
Out[8]= {HoldPattern[pun[x$_]] :> Function[x$217974, Take[x$217974, 2]][x$]}

The unique symbol created by Module[{x}, ...] is used as a formal parameter name, protecting us from accidental clashes (though if we really wanted to, we could probably engineer a name clash on purpose).

$\endgroup$
  • 1
    $\begingroup$ +1 This construction will be relatively safe, even in library code so long as the generated function is never loaded from a Save or DumpSave file. Another way to turn off name rewriting is like this makeFun[len_] := Module[{x}, Function@@Hold[x, Take[x, len]]]. Personally, I avoid named Function arguments like the plague. $\endgroup$ – WReach Sep 26 '15 at 4:05
  • 1
    $\begingroup$ @WReach I fully agree (also voted for this answer), but sometimes I just can't avoid named arguments in Function, particularly with nested closures (well, may be one can do that technically, using With etc, but it's really ugly). I am afraid that this poses a real threat to the correctness of my code where I use them, though. In particular, I heavily use them in Streaming` module. It may be worthwhile for me to inspect it and perhaps see if I can get rid of named arguments there. Of course, in my particular case, another option is to use the framework I described in my answer. $\endgroup$ – Leonid Shifrin Sep 26 '15 at 10:36
9
$\begingroup$

Here are a couple of other options:

Use system option "StrictLexicalScoping"

If you use

SetSystemOptions["StrictLexicalScoping" -> True]

Then your code runs fine (I changed the input to hun to avoid other errors):

With[{fun = makeFun[10]}, hun[x_] := fun[x]; hun[Range[15]]]

(* {1, 2, 3, 4, 5, 6, 7, 8, 9, 10} *)

Reasonably general top-level solution

A few years ago I wrote a tiny framework to deal with such problems (that was before Daniel did his work and made the "StrictLexicalScoping" option available). So, we can do

SetSystemOptions["StrictLexicalScoping" -> False];
ClearAll[hun];

to get back to the original state, and then load the code:

Import["https://gist.githubusercontent.com/lshifr/1683497/raw/AutoRenamings"]

Use runWithRenamings locally

And now:

runWithRenamings[
  With[{fun = makeFun[10]}, hun[x_] := fun[x];
  hun[Range[15]]]
]

(* {1, 2, 3, 4, 5, 6, 7, 8, 9, 10} *)

which is, wrap your code where definitions are made, into runWithRenamings. For the case at hand it produces this definition:

?hun

Global`hun

hun[x$2879$_]:=Function[x$,Take[x$,10]][x$2879$]

In general, runWithRenamings attempts to ensure that the renamed symbols produced via the renaming machanism will always be unique.

Dynamic environment based on runWithRenamings

You can make a dynamic environment that would catch the code using e.g. SetDelayed, which is a common enough use case:

ClearAll[withAutoRenamings];
SetAttributes[withAutoRenamings, HoldAll];
withAutoRenamings[code_] :=
  Internal`InheritedBlock[{SetDelayed},
    Module[{inSetDelayed},
      Unprotect[SetDelayed];
      s_SetDelayed /; ! TrueQ[inSetDelayed] := 
         Block[{inSetDelayed = True}, runWithRenamings[s]]; 
      Protect[SetDelayed];
      code]]

This will work also in cases like the following:

def[] := With[{fun = makeFun[10]}, hun[x_] := fun[x];hun[Range[15]]]

so that

withAutoRenamings @ def[]

(* {1, 2, 3, 4, 5, 6, 7, 8, 9, 10} *)

while runWithRenamings @ def[] would still produce errors.

Remarks

Unless you have strong reasons not to use "StrictLexicalScoping" option (such as, e.g. if this setting somehow breaks some legacy code, where things that "StrictLexicalScoping" -> True fixes, were used as features), you will be better off using it.

The top-level code I gave mostly as an illustration, and also there you may have a more direct control over things, but it is surely incomplete, and it also generally requires that you wrap the runWithRenamings environment around the problematic piece of code.

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.