4
$\begingroup$

Consider MemoryInUse. After kernel initiation, I have noticed that its value keeps increasing, even when not evaluating any cell, as seen

ml = Table[MemoryInUse[], {t, 0, 10000}];
ListPlot[ml, PlotRange -> {Automatic, {Mean@ml - 100000, Mean@ml + 100000}}, Frame -> True]

enter image description here

Any idea why this happens? Where could I read more about memory management within Mathematica notebooks? Any way to reset it?

Edit: Also, I have noticed that by re-evaluating a cell with the code MemoryInUse[], the value increases every time I evaluate it.

$\endgroup$
1
  • $\begingroup$ One could argue that the "edit" is not a clarification of the problem but moving the goalpost. The MemoryInUse keeps increasing every time you do something that uses new memory. Among other things , the Session History or lingering variables. $\endgroup$
    – rhermans
    Jan 23, 2023 at 16:02

1 Answer 1

4
$\begingroup$

Explanation

At least on my system, the slope Mean@Differences@ml is equal to $16$ bytes, which is equal to ByteCount[MemoryInUse[]].

It looks like the MemoryInUse increases exactly by the ByteCount amount it takes to store the output of MemoryInUse, as the Table executes and stores each value.

I don't find that surprising at all, as you fill the table with data that is stored in memory, that increases the memory used.

Solution

To understand this better, unsurprisingly, the best option would be to read the documentation about Memory Management. It's unclear what you mean with "Reset" the memory in use, but if yu mean minimizing the amount of MemoryInUse[] could be a question on its own.

If you want to avoid that increase during the table execution, you will need to either pre-allocate the data in memory or not use memory at all, by writing on disk.

Pre-allocation

You can pre-allocate the Table and update its content like this

ml = ConstantArray[0,10000];
Do[ ml[[t]]=MemoryInUse[], {t, 10000}];

enter image description here

Write to disk

Alternatively, you could store the values on disk instead of memory.

Block[
    {
        fname = FileNameJoin[{$TemporaryDirectory, "testfile"}],
        s
    },
    s = OpenWrite[fname];
    Do[ Write[s, MemoryInUse[] ], 1000];
    Close[fname];
    ListPlot[ ReadList[fname] ]
]

enter image description here

$\endgroup$
8
  • $\begingroup$ That makes sense. However, without the Table object, by simply re-evaluating a cell with the line MemoryInUse[], I get an increasing value every time. Do you get this as well? What causes this? I have added this comment as an edit to the original question. By reset, I meant minimising this change. I am not too familiar with memory usage within Mathematica, but happy to read more about it. $\endgroup$
    – sam wolfe
    Jan 23, 2023 at 15:52
  • $\begingroup$ Are you leaving the results in the stack? Try $HistoryLength = 0. $HistoryLength specifies the number of previous lines of input and output to keep in a Wolfram System session. Can't test any further without detailed explanation of what you are doing and what is the observation. $\endgroup$
    – rhermans
    Jan 23, 2023 at 15:58
  • 1
    $\begingroup$ @samwolfe You are not just re-evaluating it. In order to collect the result, values have to be stored somewhere. So, every item allocates another block of memory... $\endgroup$
    – kirma
    Jan 23, 2023 at 16:01
  • 1
    $\begingroup$ @samwolfe If you want to see something not accumulating memory usage on the kernel side on every evaluation, look at Do[Echo[MemoryInUse[]], 100]. That doesn't append entries to a list. $\endgroup$
    – kirma
    Jan 23, 2023 at 16:05
  • 1
    $\begingroup$ @kirma not on the kernel, but will use memory on the front end. $\endgroup$
    – rhermans
    Jan 23, 2023 at 16:06

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.