# Aborting evaluation when the memory exceeds a certain limit

Sometimes when generating a lot of data the memory usage shoots beyond its physical realm and my laptop freezes. There is no way to stop Mathematica when this happens. Task manager (or its equivalent in other OSes) won't even start.

I was wondering if it's possible to set a threshold somewhere in Mathematica to stop it automatically after a certain amount of memory.

-
If it is due to lots of intermediary calculations, setting $HistoryLength =0 could lead to using less memory. – Andrew Mar 11 '12 at 8:12 add comment ## 4 Answers Since one may not always accurately predict when MemoryContrained is needed, I recommend setting up a watch-dog task. Belisarius described how to do this here in answer to my question. I will reproduce it below as answers that are merely links are discouraged. In Mathematica 8 you could start a memory watchdog, something along the lines of: maxMemAllowed = 1.3 1024^3; (*1.3 GB*); intervalBetweenTests = 1; (*seconds*) iAmAliveSignal = 0; Dynamic[iAmAliveSignal] RunScheduledTask[ If[MemoryInUse[] > maxMemAllowed , Quit[], iAmAliveSignal++], intervalBetweenTests];  Remember to run RemoveScheduledTask[ScheduledTasks[]];  to disable it. - Are you actually using this in practice? – Szabolcs Mar 11 '12 at 7:09 @Szabolcs no, I don't have version 8. :-/ If belisarius cares to post his own answer I will delete mine. – Mr.Wizard Mar 11 '12 at 7:46 I didn't realize this was v8 functionality. You might want to try ?Internal*Periodical*. I'm not sure if those Internal functions exist in v7, but they seem to provide very similar (the same?) functionality. I can't recall how to use them, but it should be easy to figure out. – Szabolcs Mar 11 '12 at 8:19 Ha! Nice answer! – belisarius May 19 '12 at 22:43 add comment You can make use of either TimeConstrained or MemoryConstrained to terminate evaluation when it runs out of time or memory respectively. For example, if you have a function that has a reasonable memory footprint, but takes time to evaluate, you can abort evaluation after a certain amount of time (in seconds) has elapsed, as: TimeConstrained[Eigenvalues@RandomReal[1, {5000, 5000}], 1] Out[1]=$Aborted


On the other hand, if memory is the one that blows up first, you can wrap it in MemoryConstrained to abort when a certain amount of bytes have been consumed:

MemoryConstrained[Range /@ Range[100], 1000]
Out[2]= $Aborted  - add comment You can put your calculation inside TimeConstrained. However in your case, probably the better idea is to restrict the used memory. That's done with MemoryConstrained. If you don't want to figure out the available memory yourself, see here for how to do it automatically. For example this terminates a calculation if the calculation needs more than 1 GB of memory: MemoryConstrained[Simplify[complicatedExpression],1024^3]  This only constrains the simplification, but gives back the original expression if the evaluation fails: MemoryConstrained[Simplify[#],1024^3,#]&[complicatedExpression]  This stops evaluation after one hour: TimeConstrained[Simplify[complicatedExpression],3600]  - add comment In addition to Mr.Wizard's answer. In many cases it is very practical to stop the evaluation when the actual amount of free physical memory in your system becomes less than specified threshold. You can get the amount of free physical memory very efficiently via NETLink call to GlobalMemoryStatusEx function of kernel32.dll (which is available both under 32 bit and 64 bit Windows systems). Here I define getFreePhysMem[] function which returns the current amount of free physical memory: Needs["NETLink"]; getFreePhysMem::internalError = "globalMemoryStatusEx[memorystatusex] has not returned True."; If[$OperatingSystem === "Windows",
globalMemoryStatusEx =
Symbol["DefineDLLFunction"][
"[StructLayout(LayoutKind.Sequential, CharSet=CharSet.Auto)]
public class MEMORYSTATUSEX
{public uint dwLength;
public ulong ullTotalPhys;
public ulong ullAvailPhys;
public ulong ullTotalPageFile;
public ulong ullAvailPageFile;
public ulong ullTotalVirtual;
public ulong ullAvailVirtual;
public ulong ullAvailExtendedVirtual;
public MEMORYSTATUSEX()
{this.dwLength = (uint)
Marshal.SizeOf(typeof( MEMORYSTATUSEX ));}}
[return: MarshalAs(UnmanagedType.Bool)]
[DllImport(\"kernel32.dll\", CharSet=CharSet.Auto,     \
SetLastError=true)]
public static extern bool GlobalMemoryStatusEx([In, Out]  \
MEMORYSTATUSEX lpBuffer);"];
memorystatusex =
memorystatusex = memorystatusex <> "+MEMORYSTATUSEX";
memorystatusex = Symbol["NETNew"][memorystatusex];
getFreePhysMem[] :=
If[TrueQ[globalMemoryStatusEx[memorystatusex]],
memorystatusex@ullAvailPhys,
Message[getFreePhysMem::internalError]; Abort[]; \$Failed]];


Timings:

Do[getFreePhysMem[], {100}] // AbsoluteTiming
(*=> {0.0312500, Null}*)


More information on this function: "Calling kernel.dll from Mathematica. 1", "Calling kernel.dll from Mathematica. 2".

-
@Mr.Wizard In such a way we should call it getFreePhysMem[] for clarity. Good idea, I'll correct my post. –  Alexey Popkov Mar 15 '12 at 17:45
QAlexey Thanks for this detailed answer. It Sure gives me enough code to ponder about :). –  Lou Mar 15 '12 at 21:31
@Lou There is another method with JavaLink. You may be also interested in this discussion: "Self-restarting MathKernel - is it possible in Mathematica?". –  Alexey Popkov Mar 15 '12 at 23:49
Could this be expanded to OS X? –  shrx Jun 12 '13 at 18:41
@shrx I don't use OS X but you could try the JavaLink solution. –  Alexey Popkov Jun 12 '13 at 20:19