Why does this simple program leak memory?

Question

I have a simple Mathematica program which writes some plots to image files for later conversion into a movie. Unfortunately, the program leaks so much memory that it quickly exhausts all 12G of RAM on my machine. The only way out is to quit the kernel(s).

I can't see why this program shouldn't use a bounded amount of memory. I've read through Debugging memory leaks which unfortunately hasn't helped - the only "heavy" symbol is 'data', whose size is fixed. I don't see what's growing!

Note that the same problem occurs regardless of whether the loop is Map or Do, Parallel- or not. The problem also occurs in both Mathematica 8 and 9, both under Linux.

Help?

(* number of frames *)
n = 1000;
(* just some bogus data *)
makeData[_] := Table[{{x, y} = RandomReal[{-3, 3}, {2}], {y, -x}}, {200}];
data = Array[makeData, n];
(* Export the frames *)
ParallelMap[
  Export[
    "movie-" <> IntegerString[#, 10, 4] <> ".png",
    ListVectorPlot[data[[#]]]] &,
  Range[1, n]];

Edit:
Some more information: after running this code (the non-parallel version), I checked the memory usage of the processes involved. The percentages are out of 12GB, so both the frontend and the kernel are consuming quite a bit of memory, while PNG.exe is almost nonexistent.

  PID USER      PR  NI  VIRT  RES  SHR S %CPU %MEM    TIME+  COMMAND            
24575 gredner   20   0  4400  740  588 S    0  0.0   0:00.00 Mathematica        
24646 gredner   20   0 3029m 2.3g  22m S    0 20.5   1:43.02 Mathematica        
24655 gredner   20   0 3384m 1.6g  14m S    0 13.7   3:31.47 MathKernel         
24984 gredner   20   0  387m 7364 2464 S    0  0.1   0:20.92 PNG.exe            

Answer 1

Extended comment

I ran this code several times, with different values for n, and using Map or Do as well as ParallelMap. Each time a plot was created and exported, Mathematica's memory in use increased by about 1.7MB. We can call this a 'leak', but it seems to be by design.

So, with n as 1000, and allowing for some memory for the front end and the kernel to start with, 1000 plots uses up some but not all the available memory. (Hence my original answer - your code runs fine as it stands.)

When n reached 4000, my computer started to run out of memory — paging continuously, or 'thrashing'. Although it says 12GB total memory, the operating system needs some to work with as well.

I was unable to get to 10000 plots, for which my calculations suggest I'd need over 15GB of free memory devoted to Mathematica.

ParallelMap speeds up the process of generating the plots considerably, but it seems that each kernel uses free memory at the same rate as a single kernel, so the end result is that the computer starts to thrash sooner rather than later.

Anyway, my opinion is that there should be a way to make Mathematica forget about each plot once you've written it to disk.

Answer 2

Another glorified comment

The leak is not in Export. It is in ListVectorPlot.

Compare

tt = Table[{y, -x}, {x, -3, 3, 0.2}, {y, -3, 3, 0.2}];
before = MemoryInUse[]; ListVectorPlot[tt]; after = MemoryInUse[];
after - before

with

before = MemoryInUse[]; Identity[tt]; after = MemoryInUse[];
after - before