Take the 2-minute tour ×
Mathematica Stack Exchange is a question and answer site for users of Mathematica. It's 100% free, no registration required.

I have a simple Mathematica program which writes some plots to image files for later conversion into a movie. Unfortunately, the program leaks so much memory that it quickly exhausts all 12G of RAM on my machine. The only way out is to quit the kernel(s).

I can't see why this program shouldn't use a bounded amount of memory. I've read through Debugging memory leaks which unfortunately hasn't helped - the only "heavy" symbol is 'data', whose size is fixed. I don't see what's growing!

Note that the same problem occurs regardless of whether the loop is Map or Do, Parallel- or not. The problem also occurs in both Mathematica 8 and 9, both under Linux.

Help?

(* number of frames *)
n = 1000;
(* just some bogus data *)
makeData[_] := Table[{{x, y} = RandomReal[{-3, 3}, {2}], {y, -x}}, {200}];
data = Array[makeData, n];
(* Export the frames *)
ParallelMap[
  Export[
    "movie-" <> IntegerString[#, 10, 4] <> ".png",
    ListVectorPlot[data[[#]]]] &,
  Range[1, n]];

Edit:
Some more information: after running this code (the non-parallel version), I checked the memory usage of the processes involved. The percentages are out of 12GB, so both the frontend and the kernel are consuming quite a bit of memory, while PNG.exe is almost nonexistent.

  PID USER      PR  NI  VIRT  RES  SHR S %CPU %MEM    TIME+  COMMAND            
24575 gredner   20   0  4400  740  588 S    0  0.0   0:00.00 Mathematica        
24646 gredner   20   0 3029m 2.3g  22m S    0 20.5   1:43.02 Mathematica        
24655 gredner   20   0 3384m 1.6g  14m S    0 13.7   3:31.47 MathKernel         
24984 gredner   20   0  387m 7364 2464 S    0  0.1   0:20.92 PNG.exe            
share|improve this question
    
Have you tried using ParallelDo rather than ParallelMap? I know it shouldn't make a difference, but some of these things are more involved than they seem. Also try just plain Do. Maybe your system is launching like, infinity kernels all of which are running 100 computations, so you may need to specify things like Method -> "EvaluationsPerKernel" -> 1 –  amr May 1 '13 at 19:08
    
I've now tried ParallelDo with the same outcome. –  GRedner May 1 '13 at 19:13
2  
What Mathematica version are you using? I know from my own experience doing this exact sort of thing (exporting images) that there is an inexplicable memory issue in my version 8 which didn't occur at all in my V9 trial from the same exact copy-pasted code. –  amr May 1 '13 at 20:55
3  
Have you checked if the MathKernel itself leaks memory or the spawned PNG.exe processes? –  Thies Heidecke May 1 '13 at 21:57
1  
It could also be the front end that leaks, check that process as well. –  Szabolcs May 1 '13 at 23:13
show 9 more comments

2 Answers

Extended comment

I ran this code several times, with different values for n, and using Map or Do as well as ParallelMap. Each time a plot was created and exported, Mathematica's memory in use increased by about 1.7MB. We can call this a 'leak', but it seems to be by design.

So, with n as 1000, and allowing for some memory for the front end and the kernel to start with, 1000 plots uses up some but not all the available memory. (Hence my original answer - your code runs fine as it stands.)

When n reached 4000, my computer started to run out of memory — paging continuously, or 'thrashing'. Although it says 12GB total memory, the operating system needs some to work with as well.

I was unable to get to 10000 plots, for which my calculations suggest I'd need over 15GB of free memory devoted to Mathematica.

ParallelMap speeds up the process of generating the plots considerably, but it seems that each kernel uses free memory at the same rate as a single kernel, so the end result is that the computer starts to thrash sooner rather than later.

Anyway, my opinion is that there should be a way to make Mathematica forget about each plot once you've written it to disk.

share|improve this answer
    
Can you try increasing the number of frames (say increasing it by a factor of 10), re-running the experiment, and looking at memory usage again? –  GRedner May 3 '13 at 15:02
    
@GRedner might be able to some time, but my kernels are having a break for the weekend. (As am I...) –  cormullion May 3 '13 at 16:25
add comment

Another glorified comment

The leak is not in Export. It is in ListVectorPlot.

Compare

tt = Table[{y, -x}, {x, -3, 3, 0.2}, {y, -3, 3, 0.2}];
before = MemoryInUse[]; ListVectorPlot[tt]; after = MemoryInUse[];
after - before

with

before = MemoryInUse[]; Identity[tt]; after = MemoryInUse[];
after - before
share|improve this answer
add comment

Your Answer

 
discard

By posting your answer, you agree to the privacy policy and terms of service.

Not the answer you're looking for? Browse other questions tagged or ask your own question.