# Aborting evaluation when the memory exceeds a certain limit

Sometimes when generating a lot of data the memory usage shoots beyond its physical realm and my laptop freezes. There is no way to stop Mathematica when this happens. Task manager (or its equivalent in other OSes) won't even start.

I was wondering if it's possible to set a threshold somewhere in Mathematica to stop it automatically after a certain amount of memory.

-
If it is due to lots of intermediary calculations, setting $HistoryLength =0 could lead to using less memory. – Andrew Mar 11 '12 at 8:12 ## 4 Answers Since one may not always accurately predict when MemoryContrained is needed, I recommend setting up a watch-dog task. Belisarius described how to do this here in answer to my question. I will reproduce it below as answers that are merely links are discouraged. In Mathematica 8 you could start a memory watchdog, something along the lines of: maxMemAllowed = 1.3 1024^3; (*1.3 GB*); intervalBetweenTests = 1; (*seconds*) iAmAliveSignal = 0; Dynamic[iAmAliveSignal] RunScheduledTask[ If[MemoryInUse[] > maxMemAllowed , Quit[], iAmAliveSignal++], intervalBetweenTests];  Remember to run RemoveScheduledTask[ScheduledTasks[]];  to disable it. - Are you actually using this in practice? – Szabolcs Mar 11 '12 at 7:09 @Szabolcs no, I don't have version 8. :-/ If belisarius cares to post his own answer I will delete mine. – Mr.Wizard Mar 11 '12 at 7:46 I didn't realize this was v8 functionality. You might want to try ?Internal*Periodical*. I'm not sure if those Internal functions exist in v7, but they seem to provide very similar (the same?) functionality. I can't recall how to use them, but it should be easy to figure out. – Szabolcs Mar 11 '12 at 8:19 Ha! Nice answer! – Dr. belisarius May 19 '12 at 22:43 In addition to Mr.Wizard's answer. In many cases it is very practical to stop the evaluation when the actual amount of free physical memory in your system becomes less than specified threshold. You can get the amount of free physical memory very efficiently via NETLink call to GlobalMemoryStatusEx function of kernel32.dll (which is available both under 32 bit and 64 bit Windows systems). Here I define getFreePhysMem[] function which returns the current amount of free physical memory: Needs["NETLink"]; getFreePhysMem::internalError = "globalMemoryStatusEx[memorystatusex] has not returned True."; If[$OperatingSystem === "Windows",
memorystatusex = Symbol["LoadedNETTypes"][];
globalMemoryStatusEx =
Symbol["DefineDLLFunction"][
"[StructLayout(LayoutKind.Sequential, CharSet=CharSet.Auto)]
public class MEMORYSTATUSEX
{public uint dwLength;
public uint dwMemoryLoad;
public ulong ullTotalPhys;
public ulong ullAvailPhys;
public ulong ullTotalPageFile;
public ulong ullAvailPageFile;
public ulong ullTotalVirtual;
public ulong ullAvailVirtual;
public ulong ullAvailExtendedVirtual;
public MEMORYSTATUSEX()
{this.dwLength = (uint)
Marshal.SizeOf(typeof( MEMORYSTATUSEX ));}}
[return: MarshalAs(UnmanagedType.Bool)]
[DllImport(\"kernel32.dll\", CharSet=CharSet.Auto,     \
SetLastError=true)]
public static extern bool GlobalMemoryStatusEx([In, Out]  \
MEMORYSTATUSEX lpBuffer);"];
memorystatusex =
Complement[Symbol["LoadedNETTypes"][], memorystatusex][[1, 1]];
memorystatusex = memorystatusex <> "+MEMORYSTATUSEX";
memorystatusex = Symbol["NETNew"][memorystatusex];
getFreePhysMem[] :=
If[TrueQ[globalMemoryStatusEx[memorystatusex]],
memorystatusex@ullAvailPhys,
Message[getFreePhysMem::internalError]; Abort[]; $Failed]];  Timings: Do[getFreePhysMem[], {100}] // AbsoluteTiming (*=> {0.0312500, Null}*)  More information on this function: "Calling kernel.dll from Mathematica. 1", "Calling kernel.dll from Mathematica. 2". - QAlexey Thanks for this detailed answer. It Sure gives me enough code to ponder about :). – Lou Mar 15 '12 at 21:31 @Lou There is another method with JavaLink. You may be also interested in this discussion: "Self-restarting MathKernel - is it possible in Mathematica?". – Alexey Popkov Mar 15 '12 at 23:49 Could this be expanded to OS X? – shrx Jun 12 '13 at 18:41 @shrx I don't use OS X but you could try the JavaLink solution. – Alexey Popkov Jun 12 '13 at 20:19 You can make use of either TimeConstrained or MemoryConstrained to terminate evaluation when it runs out of time or memory respectively. For example, if you have a function that has a reasonable memory footprint, but takes time to evaluate, you can abort evaluation after a certain amount of time (in seconds) has elapsed, as: TimeConstrained[Eigenvalues@RandomReal[1, {5000, 5000}], 1] Out[1]=$Aborted


On the other hand, if memory is the one that blows up first, you can wrap it in MemoryConstrained to abort when a certain amount of bytes have been consumed:

MemoryConstrained[Range /@ Range[100], 1000]
Out[2]= \$Aborted

-
For some reason TimeConstrained either doesn't work for me or I don't understand how it works ... For example, I want something like: TimeConstrained[ For[i = 1, i < 100000, i++, Print[i]] , 1] to actually stop evaluating after one second. Instead it takes about ten seconds, even tho it only prints to about 1800 before aborting ... Is there a way to actually make THE ENTIRE process take one second? – Solarmew Jan 6 at 19:19
@Solarmew TimeConstrained is for kernel evaluations, whereas in your example, the "time consuming" part or what you see as a long pause is because the front end is trying to finish printing and displaying all the numbers that were processed in that 1 second. Try this, for example: TimeConstrained[For[i = 1, i < 100000, i++, Pause[10]], 1] — this aborts in 1 second. I don't think it is possible to do what you want (i.e. the printing to also stop), but printing output to console in a For loop is not a good practice anyway. – R. M. Jan 7 at 14:56
@Solarmew Instead, if you would like to see the progress, try modifying your code to something like the following: max = 10000000; Dynamic@ProgressIndicator[i/max] and then in a different cell: TimeConstrained[For[i = 1, i < max, i++, Null], 1]. You should see the progress bar increase and then stop after 1 second. – R. M. Jan 7 at 14:57

You can put your calculation inside TimeConstrained. However in your case, probably the better idea is to restrict the used memory. That's done with MemoryConstrained.

If you don't want to figure out the available memory yourself, see here for how to do it automatically.

For example this terminates a calculation if the calculation needs more than 1 GB of memory:

MemoryConstrained[Simplify[complicatedExpression],1024^3]


This only constrains the simplification, but gives back the original expression if the evaluation fails:

MemoryConstrained[Simplify[#],1024^3,#]&[complicatedExpression]


This stops evaluation after one hour:

TimeConstrained[Simplify[complicatedExpression],3600]

-