# Aborting evaluation when the memory exceeds a certain limit

Sometimes when generating a lot of data the memory usage shoots beyond its physical realm and my laptop freezes. There is no way to stop Mathematica when this happens. Task manager (or its equivalent in other OSes) won't even start.

I was wondering if it's possible to set a threshold somewhere in Mathematica to stop it automatically after a certain amount of memory.

-
If it is due to lots of intermediary calculations, setting $HistoryLength =0 could lead to using less memory. – Andrew Mar 11 '12 at 8:12 ## 4 Answers Since one may not always accurately predict when MemoryContrained is needed, I recommend setting up a watch-dog task. Belisarius described how to do this here in answer to my question. I will reproduce it below as answers that are merely links are discouraged. In Mathematica 8 you could start a memory watchdog, something along the lines of: maxMemAllowed = 1.3 1024^3; (*1.3 GB*); intervalBetweenTests = 1; (*seconds*) iAmAliveSignal = 0; Dynamic[iAmAliveSignal] RunScheduledTask[ If[MemoryInUse[] > maxMemAllowed , Quit[], iAmAliveSignal++], intervalBetweenTests];  Remember to run RemoveScheduledTask[ScheduledTasks[]];  to disable it. - Are you actually using this in practice? – Szabolcs Mar 11 '12 at 7:09 @Szabolcs no, I don't have version 8. :-/ If belisarius cares to post his own answer I will delete mine. – Mr.Wizard Mar 11 '12 at 7:46 I didn't realize this was v8 functionality. You might want to try ?Internal*Periodical*. I'm not sure if those Internal functions exist in v7, but they seem to provide very similar (the same?) functionality. I can't recall how to use them, but it should be easy to figure out. – Szabolcs Mar 11 '12 at 8:19 Ha! Nice answer! – belisarius May 19 '12 at 22:43 You can make use of either TimeConstrained or MemoryConstrained to terminate evaluation when it runs out of time or memory respectively. For example, if you have a function that has a reasonable memory footprint, but takes time to evaluate, you can abort evaluation after a certain amount of time (in seconds) has elapsed, as: TimeConstrained[Eigenvalues@RandomReal[1, {5000, 5000}], 1] Out[1]=$Aborted


On the other hand, if memory is the one that blows up first, you can wrap it in MemoryConstrained to abort when a certain amount of bytes have been consumed:

MemoryConstrained[Range /@ Range[100], 1000]
Out[2]= $Aborted  - You can put your calculation inside TimeConstrained. However in your case, probably the better idea is to restrict the used memory. That's done with MemoryConstrained. If you don't want to figure out the available memory yourself, see here for how to do it automatically. For example this terminates a calculation if the calculation needs more than 1 GB of memory: MemoryConstrained[Simplify[complicatedExpression],1024^3]  This only constrains the simplification, but gives back the original expression if the evaluation fails: MemoryConstrained[Simplify[#],1024^3,#]&[complicatedExpression]  This stops evaluation after one hour: TimeConstrained[Simplify[complicatedExpression],3600]  - In addition to Mr.Wizard's answer. In many cases it is very practical to stop the evaluation when the actual amount of free physical memory in your system becomes less than specified threshold. You can get the amount of free physical memory very efficiently via NETLink call to GlobalMemoryStatusEx function of kernel32.dll. Here I define a getFreePhysMem[] function which returns the current amount of free physical memory: Needs["NETLink"]; getFreePhysMem::internalError = "globalMemoryStatusEx[memorystatusex] has not returned True."; If[$OperatingSystem === "Windows" && $SystemWordLength === 32, memorystatusex = Symbol["LoadedNETTypes"][]; globalMemoryStatusEx = Symbol["DefineDLLFunction"][ "[StructLayout(LayoutKind.Sequential, CharSet=CharSet.Auto)] public class MEMORYSTATUSEX {public uint dwLength; public uint dwMemoryLoad; public ulong ullTotalPhys; public ulong ullAvailPhys; public ulong ullTotalPageFile; public ulong ullAvailPageFile; public ulong ullTotalVirtual; public ulong ullAvailVirtual; public ulong ullAvailExtendedVirtual; public MEMORYSTATUSEX() {this.dwLength = (uint) Marshal.SizeOf(typeof( MEMORYSTATUSEX ));}} [return: MarshalAs(UnmanagedType.Bool)] [DllImport(\"kernel32.dll\", CharSet=CharSet.Auto, \ SetLastError=true)] public static extern bool GlobalMemoryStatusEx([In, Out] \ MEMORYSTATUSEX lpBuffer);"]; memorystatusex = Complement[Symbol["LoadedNETTypes"][], memorystatusex][[1, 1]]; memorystatusex = memorystatusex <> "+MEMORYSTATUSEX"; memorystatusex = Symbol["NETNew"][memorystatusex]; getFreePhysMem[] := If[TrueQ[globalMemoryStatusEx[memorystatusex]], memorystatusex@ullAvailPhys, Message[getFreePhysMem::internalError]; Abort[];$Failed]];


Timings:

Do[getFreePhysMem[], {100}] // AbsoluteTiming
(*=> {0.0312500, Null}*)


More information on this function: "Calling kernel.dll from Mathematica. 1", "Calling kernel.dll from Mathematica. 2".

-
Alexey, good to see you posting again. +1 for the method. Shouldn't the function be called with freePhysicalMemory[] to adhere to convention, and for ease of handling? – Mr.Wizard Mar 15 '12 at 17:24
@Mr.Wizard In such a way we should call it getFreePhysMem[] for clarity. Good idea, I'll correct my post. – Alexey Popkov Mar 15 '12 at 17:45
QAlexey Thanks for this detailed answer. It Sure gives me enough code to ponder about :). – Lou Mar 15 '12 at 21:31
@Lou There is another method with JavaLink. You may be also interested in this discussion: "Self-restarting MathKernel - is it possible in Mathematica?". – Alexey Popkov Mar 15 '12 at 23:49