Return to Answer

3. The system itself may not work properly in some cases. There are a number of such, but the most serious problem is arguably described here:

   f[x_] := g[Function[a, x]];
   g[fn_] := Module[{h}, h[a_] := fn[a]; h[0]];
   f[999]
   

3. The system itself may not work properly in some cases. There are a number of such, but the most serious problem is arguably described here:

   f[x_] := g[Function[a, x]];
   g[fn_] := Module[{h}, h[a_] := fn[a]; h[0]];
   f[999]
   

This last set of issues can be cured (thanks to Daniel Lichtblau), with a recently added system option "StrictLexicalScoping", which one has to set to True.

This last set of issues can be cured (thanks to Daniel Lichtblau), with a recently added system option "StrictLexicalScoping", which one has to set to True.

In Mathematica, things are different. The main reason is its "overtransparent" nature, where everything is an expression to the extent that the user is free to manipulate even scoping constructs with rules and patterns. This can be also seen in the way function calls are performed, and associated argument-passing: instead of the more standard mechanism where each function gets a stack frame where passed parameters are copied as local variables, in Mathematica functions all work like macros: they are essentially placeholders, and the parameters are injected verbatim into the body of the function, before it starts executing.

Because of this, the level of encapsulation and the more standard approach to implementing lexical scoping would fly in the face of such transparency, making lexical scoping constructs inaccessible for destructuring and pattern-matching. So, my guess is that it was a conscious design decision to keep things open and playing well with the core principles of the language. But then, you can't have local closed environments, so pretty much the only choice you have is to emulate it with one big environment and variable renaming.

1. Use the "StrictLexicalScoping" system option. For example:

    SetSystemOptions["StrictLexicalScoping" -> True];
    With[{x = a}, Hold[With[{a = b}, a + 2 x]]]
    With[{x = a$}, Hold[With[{a = b}, a + 2 x]]]
   
    (* 
     Hold[With[{a$1121066 = b}, a$1121066 + 2 {1, 2, 3, 4, 5, 6, 7, 8, 9, 10}]]
   
     Hold[With[{a$1121068 = b}, a$1121068 + 2 a$]]
    *)
   

2. Some years ago I wrote a tiny micro-framework to deal with renamings, which lives here. Here is how one can use it:

    SetSystemOptions["StrictLexicalScoping" -> False]
    Import["https://gist.githubusercontent.com/lshifr/1683497/raw/AutoRenamings"]
   

    runWithRenamings[With[{x = a}, Hold[With[{a = b}, a + 2 x]]]]
    runWithRenamings[With[{x = a$}, Hold[With[{a = b}, a + 2 x]]]]

    (*

     Hold[With[{a$1121073$ = b}, a$1121073$ + 2 {1, 2, 3, 4, 5, 6, 7, 8, 9, 10}]]

     Hold[With[{a$1121075$ = b}, a$1121075$ + 2 a$]]
    *)

The problem of colliding variable names at different scoping levels (which is the main reason for the renaming mechanism in Mathematica to exist), is solved in such a case simply by construction - the inner variables shadow the outer ones automatically, due to the way variable lookup is performed. This is because, in such a case, the environments are truly nested, and the variable lookup is done from inner to outer scopes. At any given time, only one scope is searched for a variable, and within single scope variables are always unique.

In Mathematica, things are different. The main reason is its "overtransparent" nature, where everything is an expression to the extent that the user is free to manipulate even scoping constructs with rules and patterns. This can be also seen in the way function calls are performed, and associated argument-passing: instead of the more standard mechanism where each function gets a stack frame where passed parameters are copied as local variables, in Mathematica functions all work like macros: they are essentially placeholders, and the parameters are injected verbatim into the body of the function, before it starts executing.

Because of this, the level of encapsulation and the more standard approach to implementing lexical scoping would fly in the face of such transparency, making lexical scoping constructs inaccessible for destructuring and pattern-matching. So, my guess is that it was a conscious design decision to keep things open and playing well with the core principles of the language. 

But then, you can't have local closed environments, so pretty much the only simple choice you have is to emulate it with one big environment (it might be possible to reconcile the open nature of Mathematica expressions with nested scopes somehow, but that seems a much more complex problem, and would probably require introducing new constructs and primitives into the language). Since you have one big environment, you can't really easily form a hierarchical lookup mechanism used in other languages. Therefore, you have to worry about variable collisions, and do something about that. Hence the variable renaming mechanism.

1. Use the "StrictLexicalScoping" system option. For example:

    SetSystemOptions["StrictLexicalScoping" -> True];
    With[{x = a}, Hold[With[{a = b}, a + 2 x]]]
    With[{x = a$}, Hold[With[{a = b}, a + 2 x]]]
   
    (* 
     Hold[With[{a$1121237 = b}, a$1121237 + 2 a]]
   
     Hold[With[{a$1121239 = b}, a$1121239 + 2 a$]]
    *)
   

2. Some years ago I wrote a tiny micro-framework to deal with renamings, which lives here. Here is how one can use it:

    SetSystemOptions["StrictLexicalScoping" -> False]
    Import["https://gist.githubusercontent.com/lshifr/1683497/raw/AutoRenamings"]
   

    runWithRenamings[With[{x = a}, Hold[With[{a = b}, a + 2 x]]]]
    runWithRenamings[With[{x = a$}, Hold[With[{a = b}, a + 2 x]]]]

    (*

     Hold[With[{a$1121244$ = b}, a$1121244$ + 2 a]]

     Hold[With[{a$1121075$ = b}, a$1121075$ + 2 a$]]
    *)