5
$\begingroup$

Bug fixed in version 10.2.0


I'm running a large computation in parallel and the memory usage of the parallel kernels is increasing with every iteration of the calculation. Neither $HistoryLength=0, ClearSystemCache[], nor CleanSlate[] can solve the issue. I narrowed the problem down to using NIntegrate on the parallel kernel. Here is a simplified version of my code:

$HistoryLength = 0;
    ClearMemory := Block[{},
        Unprotect[In, Out];
        Clear[In, Out];
        Protect[In, Out];
        ClearSystemCache[];
       ];
    HGt[a_] := NIntegrate[t^a, {t, 0, 1}];
    calct[aa_] := Block[{$HistoryLength = 0, res},
        res = HGt[aa];
        ClearMemory;
        res
       ];
    CloseKernels[];
    LaunchKernels[1];
    ParallelEvaluate[$HistoryLength = 0];
DistributeDefinitions[calct];
ClearMemory;
jobs = Map[ParallelSubmit[Print[MemoryInUse[]]; calct[#]] &, Range[8]];
WaitAll[jobs]

As output I get:

(kernel 1) 17279624
(kernel 1) 19220336
(kernel 1) 19239064
(kernel 1) 19240160
(kernel 1) 19240896
(kernel 1) 19241632
(kernel 1) 19242368
(kernel 1) 21322872
{0.5, 0.333333, 0.25, 0.2, 0.166667, 0.142857, 0.125, 0.111111}

If I use the same code with NIntegrate replaced by NSum the kernel doesn't leak memory:

(kernel 1) 17279624
(kernel 1) 17277056
(kernel 1) 17277392
(kernel 1) 17277392
(kernel 1) 17277392
(kernel 1) 17277392
(kernel 1) 17277392
(kernel 1) 17277392

Why is the parallel kernel leaking memory and how can I fix it?

Just closing and relaunching the kernels from time to time seems to be no solution, because I'm also experiencing this problem.

I'm using Mathematica Version 10.0 for Linux x86 (64-bit) (September 9, 2014).

Thanks a lot for your help!

$\endgroup$
  • 6
    $\begingroup$ Fixed in version 10.2. $\endgroup$ – ilian Jul 14 '15 at 17:01
7
$\begingroup$

(Reposting my comment as an answer, just to take the question off the unanswered list)

This is a bug and has been fixed in version 10.2. I am not aware of any workarounds that may be applicable in earlier versions.

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.