4
$\begingroup$

I was writing a API with a lot of http requests, when I noticed Mathematica was slowly using more and more memory, and after some debugging I think the culprit is URLRead.

The following code should reproduce the problem,

$HistoryLength = 0;

mem = N@MemoryInUse[];
Do[URLRead["http://wolfram.com", {"StatusCode", "Headers"}];, {i,100}];
N@MemoryInUse[] - mem

215 488.

While a similar block of code with RandomReal instead produces no significant increase of memory usage. Moreover, ClearSystemCache will only reduce the memory after the first run on a fresh kernel.

The questions then: Can anybody reproduce these results? Is this expected behaviour? And more importantly, since I plan to continuously run my API, is there a way to reclaim this memory?

Tested on MMA 11.3.0.0 on macOS 10.13.3

Improve this question
$\endgroup$
5
  • $\begingroup$ But it is 0.2 MB, is it significant? $\endgroup$
    – Kuba
    Commented Jun 18, 2018 at 10:40
  • $\begingroup$ @Kuba In the case of my API this amounts to about 0.7 MB per 5 minutes, and since I plan to continuously run the code for days, this will become significant. $\endgroup$
    – user19218
    Commented Jun 18, 2018 at 10:55
  • $\begingroup$ I can also see an average 10.7 kbytes per call to URLRead, growing linearly. Mma 11.3.0 Win7 64. Using a local server and Table[ { Do[URLRead[url, {"StatusCode", "Headers"}];, {i, 10}]; 10 k, N@MemoryInUse[] }, {k, 0, 20} ] $\endgroup$
    – rhermans
    Commented Jun 18, 2018 at 11:01
  • 5
    $\begingroup$ I checked on 12.0 development build I currently work on, and, at least on Mac OS, I don't have this issue. So hopefully it has been fixed already. $\endgroup$
    – Leonid Shifrin
    Commented Jun 18, 2018 at 11:29
  • $\begingroup$ If I run the code on Mac OS 10.15.6 MMA 12.1.1.0 I get 1.59103x10^7 So the problem is still there and seems even worse? $\endgroup$
    – ulysseus
    Commented Aug 11, 2020 at 0:55

0

Reset to default

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Browse other questions tagged or ask your own question.