4
$\begingroup$

I was writing a API with a lot of http requests, when I noticed Mathematica was slowly using more and more memory, and after some debugging I think the culprit is URLRead.

The following code should reproduce the problem,

$HistoryLength = 0;

mem = N@MemoryInUse[];
Do[URLRead["http://wolfram.com", {"StatusCode", "Headers"}];, {i,100}];
N@MemoryInUse[] - mem

215 488.

While a similar block of code with RandomReal instead produces no significant increase of memory usage. Moreover, ClearSystemCache will only reduce the memory after the first run on a fresh kernel.


The questions then: Can anybody reproduce these results? Is this expected behaviour? And more importantly, since I plan to continuously run my API, is there a way to reclaim this memory?


Tested on MMA 11.3.0.0 on macOS 10.13.3

$\endgroup$
  • $\begingroup$ But it is 0.2 MB, is it significant? $\endgroup$ – Kuba Jun 18 '18 at 10:40
  • $\begingroup$ @Kuba In the case of my API this amounts to about 0.7 MB per 5 minutes, and since I plan to continuously run the code for days, this will become significant. $\endgroup$ – user19218 Jun 18 '18 at 10:55
  • $\begingroup$ I can also see an average 10.7 kbytes per call to URLRead, growing linearly. Mma 11.3.0 Win7 64. Using a local server and Table[ { Do[URLRead[url, {"StatusCode", "Headers"}];, {i, 10}]; 10 k, N@MemoryInUse[] }, {k, 0, 20} ] $\endgroup$ – rhermans Jun 18 '18 at 11:01
  • 5
    $\begingroup$ I checked on 12.0 development build I currently work on, and, at least on Mac OS, I don't have this issue. So hopefully it has been fixed already. $\endgroup$ – Leonid Shifrin Jun 18 '18 at 11:29

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Browse other questions tagged or ask your own question.