I found a really weird issue using NDSolve`ProcessEquations inside module.
We start by setting HistoryLength to 0. After this we define a function which takes in an integer and creates a Table of stateData Objects of the desired length and returns a list containing a string and the list of stateData objects. To show how much memory the local objects use I make use of the Print[] function.
$HistoryLength = 0;
Clear[prepareEquations]
prepareEquations[iMax_] :=
Module[{stateDataList, output},
stateDataList =
Table[
First @
NDSolve`ProcessEquations[{y''[t] + y[t] == 0, y[0] == 1, y'[0] == 0}, {y, y'}, t],
{i, 1, iMax}];
Print["ByteCount stateDataList :", ByteCount[stateDataList]];
output = {"hello", stateDataList};
Print["ByteCount output :", ByteCount[output]];
output
];
First we want to see how much memory we use before calling the functions.
MemoryInUse[]
30105504
Alright, so about 30.1 mb. Now we call the function with an input value of 5000. The function will at the same time print how much memory the local objects use.
finalList = prepareEquations[5000];
ByteCount stateDataList :87590968
ByteCount output :87591064
So the output takes about 87.6 mb. So we would assume that the total memory used after calling this function would be about 30.1+87.6=117.7
mb. But instead we have
MemoryInUse[]
164795040
Which is about 164.8 mb. Hence there seems to vanish about 164.8-117.7=47.1
mb.
If i look into my global context, finalList takes up as much memory as the ByteCount of output. If I change the output to not return the stateDataList by for example output={"hello",1}
, no memory is leaked. Where does the extra memory go when I return a list of stateData objects?
Notice that local variables y$1235
, y$125
, ..., and so on is leaked to the global context. I removed them by
Remove /@ Select[Names["Global`*"],StringMatchQ[#1, "y$" ~~ DigitCharacter ..] &];
and the memory used after this was
MemoryInUse[]
164518400
so they don't seems to have a major impact. If i look into my global context it looks like this
Reverse@SortBy[Map[{#, ByteCount@ToExpression[#]} &, Names["Global`*"]], Last] //TableForm
{"finalList", 87591064}, {"y", 0}, {"t", 0}, {"stateDataList$", 0}, {"stateDataList", 0}, {"prepareEquations", 0}, {"output$", 0}, {"output", 0}, {"iMax", 0}, {"i", 0}
So the memory leaked doesn't show up here. Can someone help me figure this out?
Module
and thePrint
s.ClearSystemCache
has no effect. UsingRemove
onfinalList
gives me back most of the memory. Note that the data you generate has a functionExperimental'NumericalFunction
in it. Well.. yeah, undocumented functions.. $\endgroup$ – Jacob Akkerboom Oct 9 '13 at 12:18Experimental'NumericalFunction
? $\endgroup$ – jaclea Oct 9 '13 at 13:29finalList[[2,1]]//FullForm
I believe is where I found it. I still don't know where the memory went. $\endgroup$ – Jacob Akkerboom Oct 9 '13 at 13:38