# How to un-eat memory?

Consider this code:

MemoryInUse[]
T = Table[RandomComplex[], {i, 1, 6000}, {j, 1, 6000}];
MemoryInUse[]
T += T\[ConjugateTranspose];
MemoryInUse[]
{Es, Ys} = Eigensystem[T];
MemoryInUse[]
T = Table[RandomComplex[], {i, 1, 6000}, {j, 1, 6000}];
MemoryInUse[]
T += T\[ConjugateTranspose];
MemoryInUse[]
{Es, Ys} = Eigensystem[T];
MemoryInUse[]
$HistoryLength = 0; MemoryInUse[] Clear[T] MemoryInUse[] Clear[Es, Ys] MemoryInUse[] ClearSystemCache[] MemoryInUse[]  It gives me the following results: 15808208 880820520 1456822832 4919500424 5783503032 6359505096 9822181440 9822182648 9822182112 9822181384 9822162952 Clearly, the memory clears negligibly on any of ClearSystemCache, Clear and zeroing $HistoryLength$. Repeating its execution leads to swapping, after start of which I hurry up to kill MathKernel before my X or WM or anything else are OOM-killed. So what are the working ways to release the memory? - You need to put $HistoryLength = 0 before your calculations. –  ybeltukov Dec 14 '13 at 22:18
@ybeltukov but shouldn't zeroing it clear all the history already present? –  Ruslan Dec 14 '13 at 22:23
It is just a global variable. It not clear the history. You can still access to the history by ByteCount@Out[12] or ByteCount@%10. It can be cleared by Unprotect[In, Out]; Clear[In, Out]; but it is not a good idea. –  ybeltukov Dec 14 '13 at 22:32
@ybeltukov Wow, you're right. This could make an answer. –  Ruslan Dec 15 '13 at 7:44
Done! P.S. RandomComplex[{0, 1 + I}, {6000, 6000}] is faster. –  ybeltukov Dec 15 '13 at 13:39

$HistoryLength is just a global variable. It not clear the history. You can still access the history by ByteCount@Out[12] ByteCount@%10  It can be cleared by Unprotect[In, Out] Clear[In, Out]  However, it would be better if you set $HistoryLength=0 before your resource-intensive code.

P.S. It would be great to have $HistoryMemoryLimit or something like this. - Here is a method I've prepared for clearing memory between function iterations. It's a working model although I haven't needed to use it yet. It fully quits memory between each iteration. Each section needs to be in a separate cell to work. The second cell runs a four-cell loop for as many iterations as specified. The functionToEvaluate[n] should be user-specified, and would be limited to not doing things that would break the loop. Note, this code modifies a $FrontEnd option which persists across sessions, so if you experiment with it you may want to check the default value True is set when you're done. The last section does that.

(* Setup section *)
numberOfIterations = 4;
initialOptionState = Options[$FrontEnd, "ClearEvaluationQueueOnKernelQuit"][[1, -1]]; SetOptions[$FrontEnd,"ClearEvaluationQueueOnKernelQuit" -> False]
(* initialise temporary log file *)
Put[{initialOptionState, numberOfIterations, MaxMemoryUsed[]},
FileNameJoin@Append[FileNameSplit@$TemporaryDirectory, "log.m"]] (* The log keeps a record of the initial option status, numberOfIterations and memory use. *) ReadList[FileNameJoin@Append[FileNameSplit@$TemporaryDirectory, "log.m"]]


(* Evaluate only this cell to launch the iteration procedure *)
nb = EvaluationNotebook[];
SelectionMove[nb, Next, Cell];
Do[FrontEndExecute[FrontEndToken[nb, "SelectNextLine"]], {2}];
SelectionEvaluate[nb]


Quit[];


iteration = Length[log = ReadList[
FileNameJoin@Append[FileNameSplit@$TemporaryDirectory, "log.m"]]]; numberOfIterations = log[[1, 2]]; PutAppend[{DateList[], MemoryInUse[], MaxMemoryUsed[]}, FileNameJoin@Append[FileNameSplit@$TemporaryDirectory, "log.m"]];
(* Define or Get the function to be iterated *)
functionToEvaluate[n_] :=(*Print[n]*)n^2
functionToEvaluate[iteration];


If[iteration < numberOfIterations,
nb = EvaluationNotebook[];
SelectionMove[nb, Previous, Cell, 4];
SelectionEvaluate[nb],
initialOptionState = log[[1, 1]];
SetOptions[$FrontEnd, "ClearEvaluationQueueOnKernelQuit" -> initialOptionState]; Print[iteration, " interations completed."]]  Print["This cell does not get evaluated by the loop."]  (* Viewing the log *) Grid[ReadList[FileNameJoin@ Append[FileNameSplit@$TemporaryDirectory, "log.m"]]]


(* Check option has been reset to former value *)
Options[$FrontEnd, "ClearEvaluationQueueOnKernelQuit"]  (* Use this to reset option, if required (for instance if loop is interrupted). *) SetOptions[$FrontEnd, "ClearEvaluationQueueOnKernelQuit" -> True]


I have noticed the option's state sometimes get set counter to my expectations, most probably due to the quitting and resetting, so sometimes the loop only runs once. I haven't completely finished figuring that out yet, but if you have suggestions I'd be interested to hear. In practice, once it's looping it can be left to run. Code improvement suggestions welcome.

Another approach to running fresh kernels would be to use one kernel to run another repeatedly, as demonstrated here.

-