3
$\begingroup$

Summary:

The evaluation of an Export[...] expression fails with a "Not enough memory available" error, but the value returned by MaxMemoryUsed[] is well below the available memory reported by the free -b command.


I have a simple script (not a notebook) of the following form:

$HistoryLength = 0;  (* probably meaningless here *)

Get["dump_of_grid.mx"];  (* loads GRID *)

Export["grid.png", GRID];  (* error occurs when evaluating this expression *)

Print["MaxMemoryUsed: " <> ToString[MaxMemoryUsed[]] <> " bytes"];
Run["free -b"];

Basically, with the Get expression, the script loads from a DumpSave-generated file an item called GRID, and then it immediately attempts to export GRID to a *.png file.

More specifically, the script outputs the following:

Rasterize::bigraster: Not enough memory available to rasterize Notebook expression.
MaxMemoryUsed: 487726296 bytes
             total       used       free     shared    buffers     cached
Mem:    33710690304 18287972352 15422717952  583143424  201375744 10360836096
-/+ buffers/cache: 7725760512 25984929792
Swap:   68499271680 4590067712 63909203968

So MaxMemoryUsed reports 487726296 bytes, while the external command free -b says that there are still 15422717952 bytes free.

In light of this, why would Mathematica fail with a "Not enough memory" error?


(It could be that Mathematica first computes how much memory the rasterization will require, and fails if the resulting number is greater than the available memory; if this is what happens, then the value returned by MaxMemoryUsed[] is not a fair estimate of how much memory Mathematica needs. Is there some other way to get this estimate?)

$\endgroup$
  • 2
    $\begingroup$ it is just speculating, but it might well be that mathematica does what you expect: it tries to allocate memory for the rasterization, OS says nope, not enought memory and mma reports that it couldn't do what you wanted. As the name indicates MaxMemoryUsed implies that reports only how much memory actualy was used, not how much it tried to get. Of course it would be good to know how much it tried to get and didn't succeed, but unfortunately I think such functionality is not available. $\endgroup$ – Albert Retey May 28 '16 at 17:37
  • 2
    $\begingroup$ What's in your GRID? Sometimes Mathematica tries to interpolate the data points and the resulted plots may be quite large. If that's the problem, then you may want to consider making a smaller plot by resampling, turn off interpolation, or using Image[data...] to construct the plots, etc. $\endgroup$ – xslittlegrass May 28 '16 at 17:41
  • 6
    $\begingroup$ Another thing you might keep in mind: Mathematica is generally not very economical with respect to memory consumption, which is especially true for the import and export functionality. It also seems worth mentioning that exporting graphics is to a large part done by the frontend and special code running in extra processes, so the memory consumption of those is not even reported by the kernel. Considering all that Mathematica might be the wrong tool for what you try to do. Can you give some background about what exactly you try to achieve? $\endgroup$ – Albert Retey May 28 '16 at 17:42
  • $\begingroup$ I had a similar problem and temporarily created a 1 terabyte swap file. Despite this, Mathematica gave me the same error, long before it has used that much memory. I think there might be some built-in limit re how much memory Mathematica can use. $\endgroup$ – barrycarter Sep 25 '18 at 0:13

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Browse other questions tagged or ask your own question.