Why does the global variable exist here, where it was declared only in Module?

Question

Why does Mathematica choose the mechanism that makes the global variable a here while parsing?

In

f := Module[{a}, a;]

In

?a

Out

Global`a

I don't understand why Mathematica goes in the way that the global variable a is declared and not removed. Maybe there are its own purposes or intentions of why Mathematica is constructed in that way. Sometimes the way Mathematica works feels very confusing to me.

Thank you. :)

Answer 1

Why is this happening

The explanation was basically given by ciao in comments. You can also find a lot of information on this in this great answer of Mr.Wizard. I will perhaps try to view it from a somewhat different perspective.

To understand what happens, one should go back and consider what happens when we enter and execute some code. The steps are roughly the following

• The string code is parsed by the Mathematica parser
  • In the FrontEnd, top-level statements in the cell are passed to the kernel as separate evaluations
  • In packages, the code is parsed line by line, and complete expressions are evaluated as soon as they are formed
• At the time the parsing happens, all Mathematica cares about is that it is given a well-formed expression. There is no semantics attached to any expression yet, since evaluation happens only after parsing.
• During the parsing, new symbols are created if needed. This is the central point here. The decision to create a new symbol is made by the parser, based on the set of symbols in the current context, and those available in other contexts currently on the $ContextPath. Again, no semantics of any kind is involved in this process.
• Finally, parsed expression is allowed to evaluate, according to semantics of the symbols involved. Here, is is important to realize, that in general there is no correlation between the time code is parsed and the time it is evaluated. For example, Module in delayed definitions (e.g. functions) will only be evaluated when the function is called (which may happen at an unspecified moment in time, or not happen at all).

Since Module[{a}, a] is an expression like any other, it is parsed first, and the symbols a is created in the current working context (if it doesn't yet exist there or can not be found in other contexts currently on the $ContextPath), according to the general rule.

The fact that Module constructs symbols a$n as a part of its localization mechanism, is an evaluation - time effect. This happens much later than the construction of a during the parsing, and is a completely independent process. There is no way it can affect the parse-time artefacts like that.

Why I think this is the right behavior for Mathematica

I actually think that this is the right behavior. Why? Because in this way, the system stays completely true to its general principles, and does not create new rules and corner cases, which would make it yet harder to understand. The language is complex enough that we really don't need any additional complexity.

In fact, even though the lexical scoping as currently implemented in Mathematica has a number of issues (can be broken, is only an emulation, creates new symbols and is based on renaming rather than true localization), I am still a big fan of it. The reason is that it is very hackable, and when you realize it, you can bend it to do what you want, very easily. The whole fact that lexical scoping is done via symbol renaming, makes entire scoping completely available to the top-level user.

An example: a version of Module with an additional way to access local variables

Let me give an example, which I actually used in my code recently. We will construct a version of Module with a more fine-grained access to localized variables - it will give the user a way to access the localized variables by their string names inside Module, and also modify them:

ClearAll[varRegisteringModule];
SetAttributes[varRegisteringModule, HoldAll];
SyntaxInformation[varRegisteringModule] = {"LocalVariables" -> {"Solve", {1, 1}}};
varRegisteringModule[vars_, body_] :=
   Module[{getVars, strvars},
      SetAttributes[getVars, {HoldAll, Listable}];
      getVars[Set[v_Symbol, rhs_]] := Hold[v];
      getVars[v_Symbol] := Hold[v];
      (* String names for private variables. Must be evaluated before we enter Module *)        
      strvars = Function[v, ToString[Unevaluated[v]], {HoldAll, Listable}][vars];
      Module[vars,
        Block[{$private  = AssociationThread[strvars, getVars[vars]]},
 	    (* Evaluate inside Module, to get the vars bound to actual symbols produced by Module *)
  		      $private /: Set[$private[name_String], val_] /; KeyExistsQ[$private, name] :=
             Replace[ $private[name], Hold[var_] :> (var = val)];
          body
        ]
      ]
   ];

Here is how we can use it:

varRegisteringModule[{a, b, c},
  Print[$private["a"]];
  $private["a"] = 1;
  Print[a]
]

(*

  Hold[a$17084]

  1
*)

where you see that we can actually both access the exact name of the generated Module variable and also modify it, by using a new Module - local variable $private. Had I not put it into Block, and we could access these local variables even after Module evaluated.

Other use cases

Other class of examples comes from renaming system, which protects inner scoping constructs. More details on that can be found here and in the links therein. But the main point here is that we can always bend the system and fool that renaming mechanism, when we need to. It is a separate question how adequate is renaming scheme to implement lexical scoping, but having this sort of freedom to actually reprogram every aspect of the scoping on the top-level is very valuable and unique.

Finally, just the ability to build functions at run-time like so:

Block[{x}, Function @@ {x, f[x]}]

can be quite valuable, since it postpones function construction until run-time.

Why is that useful

Of course, you may ask when is this sort of access needed. Well, it becomes important for certain advanced applications, where you want to implement your own scoping scheme. But the main thing for me is that the system makes this sort of things possible at all, and actually quite transparent. Had Module acted as a black box, and there would be no way to make constructs like that.

The price for this freedom is, however, that things like scoping are delayed until run-time. But this means that they can in no way be coupled to events happening during the parsing of code / expressions, if the system wants to remain transparent and easy to understand. Thus, there is no way that those symbols constructed during the parsing stage could be removed by Module (and other lexical scoping constructs like With or Rule / RuleDelayed, Function, etc), because it is completely unknown in general when these constructs will actually be called, if at all, and that is completely uncorrelated with when they were parsed.

What to do about it

The best method to avoid these problems is to use packages, and place all implementations of your functions into the "`Private`" subsection of a package. In that case, those symbols will still be created, but they will be created in the "`Private`" sub-context, and won't in any way pollute your exported namespace or interact with anything important.

Actually, this is one good reason to always place your implementations in the "`Private`" sub-context, even if there are no explicit private functions / symbols you introduce yourself - because most of the lexical scoping constructs (including pattern variables in Set / SetDelayed), will produce such parasitic symbols, which you really don't want to leak into the public part of your package.