Take the 2-minute tour ×
Mathematica Stack Exchange is a question and answer site for users of Mathematica. It's 100% free, no registration required.

I have asked this question before on the Wolfram community forum (http://community.wolfram.com/groups/-/m/t/83845?p_p_auth=Gxoxe65B) but not with great success.

Module creates temporary variables of the form x$NNN which when no longer used are removed. However under certain conditions the temporary variables created by Module are not removed and remain in the global context. If this happens for a Module inside a frequently used function, this leads to memory leak situation where the memory usage of the kernel grows linearly with time. (Which is not good when doing numerical runs of several days).

In certain situations, this behaviour is explainable and probably a feature. Generally, speaking it usually involves some side effects which cause something outside the module to reference the temporary variable.

Some examples

1.

Module[{x}, x /. NDSolve[x'[t] == 1 && x[0] == 0, x, {t, 0, 1}][[1]]]

Result: x$NNN is not removed.

Reason: NDSolve stores some expressions referencing x$NNN in the system cache.

Solution: Clear system cache. (Presumably this will happen on its own as the kernel session progresses.)

2.

Module[{y, z, a, b}, a = y + z; b = 2*a];

Result: y\$NNN and z\$NNN are not removed from global context.

Reason: y\$NNN and z\$NNN appear in the module output and are thus referenced by Out.

Solution: Clear Out, or more systematically limit the History Length through $HistoryLength

3.

Module[{x}, f = x]

Result: x\$NNN is not removed from global context.

Reason: Definition of f is referencing x\$NNN

Solution: Clear f.

Behaviour that I do not understand

In addition to these well-understood cases, there some cases where I do not understand why mathematica is not removing the temporary variables after use:

4.

(from a different thread I found when looking for an answer)

a[b_] := Module[{x}, x := 1; x /; b]
a[True]

Result:a x$NNN is permanently placed in the global context every time a[True] is called.

Reason: Unknown. This appears to be a bug. Possibly related to how mathematica handles conditionals combined with Module (Clear and ClearSystemCache[] do not remove the x$NNN.)

5.

Module[{a, b},
a[i_] := b[i - 1];
b[i_] := a[i - 1]]

Result: a\$NNN and b\$NNN remain in global context.

Reason: Unknown. Bug? Related to circular references of temporary variables.

Edit: Some experimentation has lead me to simplify this example. It has confirmed to me that this must be related to circular referencing since the temporary variables disappear after clearing their definitions with

Clear["a\$NNN", "b\$NNN"]

It is essential that the variable names are input as a string, to avoid referencing them from the input history.

Interestingly enough both temporary variables remain if you clear only one of the definitions using:

Clear["a\$NNN"]

or

Clear["b\$NNN"]

This behaviour seems very buggy to me. At least a cannot think of a reason why this should happen by design.

Other cases

In addition to these cases I have seen this happen for more complicated functions involving Module, which I have not been able to trace to a simple cause.

It would be incredibly helpful for me to understand why this is happening (and how to avoid it). And related if it cannot be avoided: What is the best way of dealing with the stray temporary variables?

share|improve this question
    
Tangentially related: stackoverflow.com/a/1640675/590388 –  Alexey Popkov Nov 15 '13 at 16:07
1  
Your example 4 AFAIK was first published in this thread: "Tempvar zombies littering my context!" –  Alexey Popkov Nov 15 '13 at 16:13
    
@AlexeyPopkov: I believe so yes. –  mmeent Nov 20 '13 at 16:34
1  
This thread should be linked here too (see the comments there): Reducing memory usage in an extended Mathematica session –  Szabolcs Nov 20 '13 at 17:21

2 Answers 2

4 makes sense to me. But first I need to explain how I believe conditions on the rhs work.

In a definition like f[x_]:=rhs[x], when Mathematica sees f[something] it immediately replaces it with rhs[something] and continues its evaluation. This is not as trivial as it sounds. This is what makes g[0]:=0; g[n_]:=g[n-1]; g[10000] be an iteration and not a recursion, for example.

When the rhs is a condition, in one of its many forms (Condition or RuleCondition possibly nested inside Module|Block|With|CompoundExpression), then the condition must be resolved before returning its lhs.

So, let's say we define

a[b_] := Module[{x}, x := 1; x /; b]

The x is part of the signature of the definition (it is something to be evaluated before knowing if the definition fit or not). I'm not too bothered by the fact that a symbol x$ is created. I can't see how that would be problematic since you wouldn't create 999999999999 unique definitions and expect them not to take up some space.

Now, when we evaluate a[True], Mathematica checks if the definition fits. So, it creates x$3945723, assigns it the value 1, checks if True is True and since it is, it returns the lhs of the condition AS IS. Meaning, it returns x$3945723. That symbol is then evaluated to 2, already from outside the Module, which didn't clear the local because it was part of the result.

Again, this is what allows something like g[10000] to be an iteration and not a recursion with g defined by

g[0] = 0;
g[y_] := Module[{x = y}, g[x - 1] /; True];

In many cases, in which the result is invariant, Mathematica actually clears the temporary variable, but THAT is what strikes to me as special bonus smart behaviour to me, and not the other way around.

How to fix it? Well, either play around and find a form in which Mathematica can be smart, or you be the smart one and force it to return the evaluated value. The combo Block with Condition evaluates the lhs of the condition before replacement. So you could change your definition to

a[b_] := Module[{x}, x := 1; Block[{}, x /; True] /; b]

PD: guess what message you would get by replacing Module with Block in the definition of g and running g[10000] again

share|improve this answer
    
Your explanation raises 2 questions: 1) If you are right and Condition is evaluated outside of the Module, isn't it a bug itself? 2) This explanation contradicts the workaround you suggest: if Condition is evaluated outside of Module, then it will be evaluated even with Block on the lhs (and this happens if one removes the /;True from Block). –  Alexey Popkov Nov 16 '13 at 5:44
2  
@AlexeyPopkov 1) The Condition isn't evaluated outside of the Module. Condition is HoldAll. All it does, when immediately inside a Module on the rhs of a rule, is to check if it matches and, if it matches, then do the replacement unevaluated. Try Hold[2]/. 2:>Module[{x=2}, x/;True] –  Rojo Nov 16 '13 at 5:49
    
@AlexeyPopkov I am not sure why that design choice was made but I wouldn't know why the opposite design choice should have been made, so I don't see it as a bug. –  Rojo Nov 16 '13 at 5:51
    
Both questions are answered, thank you. Hold[2] /. 2 :> Block[{x = 3}, x /; True] gives Hold[3] but Hold[2] /. 2 :> Module[{x = 3}, x /; True] gives Hold[x$853]. It means that Module performs literally lexical scoping and no more, and Block performs dynamic scoping. I think such enlightening example must be in the Documentation! –  Alexey Popkov Nov 16 '13 at 6:01

Your fourth example

I'll try to elaborate your fourth example which AFAIK was first published in this thread: "Tempvar zombies littering my context!"

First of all, I switch off the history tracking:

$HistoryLength = 0;

I demonstrate the expected behavior. I define a function

a[b_] := Module[{c, d}, c := 1; d := 9; d];

Let us see which symbols exist now in the current context:

Names[$Context<>"*"]

{"a", "b", "c", "d"}

This behavior is expected, it is by design.

Now I put a Condition into Module:

a[b_] := Module[{c, d}, c := 1; d := 9; d /; b === 1]; 

Now

 Names[$Context<>"*"] 

{"a", "b", "c", "c\$", "d", "d\$"}

It seems that the d$ variable is due to Condition but the c$ variable is unexpected. If I try to Remove it I get:

Remove[c$]

Remove::relex: Cannot remove lexical symbol c$ except automatically (when c is removed). >>

Let us see what happens if the Condition fires:

a[1]; Names[$Context <> "*"]

{"a", "b", "c", "c\$", "d", "d\$", "d\$574"}

New symbol d$574 is generated. Checking its Definition:

Definition@d$574

Attributes[d$574]={Temporary}

d$574:=9

It has an OwnValue defined!

Let us check what happens if we call a[1] multiple times:

Table[a[1], {1000}]; Length@Names[$Context <> "*"]

1007

We get 1000 new Temporary variables each having its own OwnValue! What is the memory leak!


One workaround I see is do not use Condition inside a Module in the discussed way. Instead, you can put Condition in the left-hand side of the function definition:

a[b_] /; b === 1 := Module[{c, d}, c := 1; d := 9; d];
a[1]
Names[$Context <> "*"]

{"a", "b", "c", "d"}

Another workaround with explanation why this behavior should not be considered as a bug provided in the Rojo's answer.

share|improve this answer

Your Answer

 
discard

By posting your answer, you agree to the privacy policy and terms of service.

Not the answer you're looking for? Browse other questions tagged or ask your own question.