Take the 2-minute tour ×
Mathematica Stack Exchange is a question and answer site for users of Mathematica. It's 100% free, no registration required.

Consider this code:

MemoryInUse[]
T = Table[RandomComplex[], {i, 1, 6000}, {j, 1, 6000}];
MemoryInUse[]
T += T\[ConjugateTranspose];
MemoryInUse[]
{Es, Ys} = Eigensystem[T];
MemoryInUse[]
T = Table[RandomComplex[], {i, 1, 6000}, {j, 1, 6000}];
MemoryInUse[]
T += T\[ConjugateTranspose];
MemoryInUse[]
{Es, Ys} = Eigensystem[T];
MemoryInUse[]
$HistoryLength = 0;
MemoryInUse[]
Clear[T]
MemoryInUse[]
Clear[Es, Ys]
MemoryInUse[]
ClearSystemCache[]
MemoryInUse[]

It gives me the following results:

15808208

880820520

1456822832

4919500424

5783503032

6359505096

9822181440

9822182648

9822182112

9822181384

9822162952

Clearly, the memory clears negligibly on any of ClearSystemCache, Clear and zeroing $HistoryLength$. Repeating its execution leads to swapping, after start of which I hurry up to kill MathKernel before my X or WM or anything else are OOM-killed.

So what are the working ways to release the memory?

share|improve this question
1  
You need to put $HistoryLength = 0 before your calculations. –  ybeltukov Dec 14 '13 at 22:18
1  
@ybeltukov but shouldn't zeroing it clear all the history already present? –  Ruslan Dec 14 '13 at 22:23
3  
It is just a global variable. It not clear the history. You can still access to the history by ByteCount@Out[12] or ByteCount@%10. It can be cleared by Unprotect[In, Out]; Clear[In, Out]; but it is not a good idea. –  ybeltukov Dec 14 '13 at 22:32
    
@ybeltukov Wow, you're right. This could make an answer. –  Ruslan Dec 15 '13 at 7:44
    
Done! P.S. RandomComplex[{0, 1 + I}, {6000, 6000}] is faster. –  ybeltukov Dec 15 '13 at 13:39
add comment

2 Answers 2

up vote 8 down vote accepted

$HistoryLength is just a global variable. It not clear the history. You can still access the history by

ByteCount@Out[12]
ByteCount@%10

It can be cleared by

Unprotect[In, Out]
Clear[In, Out]

However, it would be better if you set $HistoryLength=0 before your resource-intensive code.

P.S. It would be great to have $HistoryMemoryLimit or something like this.

share|improve this answer
add comment

Here is a method I've prepared for clearing memory between function iterations. It's a working model although I haven't needed to use it yet. It fully quits memory between each iteration. Each section needs to be in a separate cell to work. The second cell runs a four-cell loop for as many iterations as specified. The functionToEvaluate[n] should be user-specified, and would be limited to not doing things that would break the loop.

Note, this code modifies a $FrontEnd option which persists across sessions, so if you experiment with it you may want to check the default value True is set when you're done. The last section does that.

(* Setup section *)
numberOfIterations = 4;
initialOptionState = Options[$FrontEnd,
    "ClearEvaluationQueueOnKernelQuit"][[1, -1]];
SetOptions[$FrontEnd,"ClearEvaluationQueueOnKernelQuit" -> False]
(* initialise temporary log file *)
Put[{initialOptionState, numberOfIterations, MaxMemoryUsed[]}, 
 FileNameJoin@Append[FileNameSplit@$TemporaryDirectory, "log.m"]]
(* The log keeps a record of the initial option status, 
    numberOfIterations and memory use. *)
ReadList[FileNameJoin@Append[FileNameSplit@$TemporaryDirectory, "log.m"]]

enter image description here

(* Evaluate only this cell to launch the iteration procedure *)
nb = EvaluationNotebook[];
SelectionMove[nb, Next, Cell];
Do[FrontEndExecute[FrontEndToken[nb, "SelectNextLine"]], {2}];
SelectionEvaluate[nb]

enter image description here

Quit[];

enter image description here

iteration = Length[log = ReadList[
     FileNameJoin@Append[FileNameSplit@$TemporaryDirectory, "log.m"]]];
numberOfIterations = log[[1, 2]];
PutAppend[{DateList[], MemoryInUse[], MaxMemoryUsed[]}, 
     FileNameJoin@Append[FileNameSplit@$TemporaryDirectory, "log.m"]];
(* Define or Get the function to be iterated *)
functionToEvaluate[n_] :=(*Print[n]*)n^2
functionToEvaluate[iteration];

enter image description here

If[iteration < numberOfIterations,
 nb = EvaluationNotebook[];
 SelectionMove[nb, Previous, Cell, 4];
 SelectionEvaluate[nb],
 initialOptionState = log[[1, 1]];
 SetOptions[$FrontEnd, 
  "ClearEvaluationQueueOnKernelQuit" -> initialOptionState];
 Print[iteration, " interations completed."]]

enter image description here

Print["This cell does not get evaluated by the loop."]

enter image description here

(* Viewing the log *)
Grid[ReadList[FileNameJoin@
   Append[FileNameSplit@$TemporaryDirectory, "log.m"]]]

enter image description here

(* Check option has been reset to former value *)
Options[$FrontEnd, "ClearEvaluationQueueOnKernelQuit"]

enter image description here

(* Use this to reset option, if required (for instance if loop is interrupted). *)
SetOptions[$FrontEnd, "ClearEvaluationQueueOnKernelQuit" -> True]

I have noticed the option's state sometimes get set counter to my expectations, most probably due to the quitting and resetting, so sometimes the loop only runs once. I haven't completely finished figuring that out yet, but if you have suggestions I'd be interested to hear. In practice, once it's looping it can be left to run. Code improvement suggestions welcome.

Another approach to running fresh kernels would be to use one kernel to run another repeatedly, as demonstrated here.

share|improve this answer
add comment

Your Answer

 
discard

By posting your answer, you agree to the privacy policy and terms of service.

Not the answer you're looking for? Browse other questions tagged or ask your own question.