# How can I find the source of a memory leak?

I found a really weird issue using NDSolveProcessEquations inside module.

We start by setting HistoryLength to 0. After this we define a function which takes in an integer and creates a Table of stateData Objects of the desired length and returns a list containing a string and the list of stateData objects. To show how much memory the local objects use I make use of the Print[] function.

$HistoryLength = 0; Clear[prepareEquations] prepareEquations[iMax_] := Module[{stateDataList, output}, stateDataList = Table[ First @ NDSolveProcessEquations[{y''[t] + y[t] == 0, y[0] == 1, y'[0] == 0}, {y, y'}, t], {i, 1, iMax}]; Print["ByteCount stateDataList :", ByteCount[stateDataList]]; output = {"hello", stateDataList}; Print["ByteCount output :", ByteCount[output]]; output ];  First we want to see how much memory we use before calling the functions. MemoryInUse[]  30105504  Alright, so about 30.1 mb. Now we call the function with an input value of 5000. The function will at the same time print how much memory the local objects use. finalList = prepareEquations[5000]; ByteCount stateDataList :87590968 ByteCount output :87591064  So the output takes about 87.6 mb. So we would assume that the total memory used after calling this function would be about 30.1+87.6=117.7 mb. But instead we have MemoryInUse[]  164795040  Which is about 164.8 mb. Hence there seems to vanish about 164.8-117.7=47.1 mb. If i look into my global context, finalList takes up as much memory as the ByteCount of output. If I change the output to not return the stateDataList by for example output={"hello",1}, no memory is leaked. Where does the extra memory go when I return a list of stateData objects? Notice that local variables y$1235, y$125, ..., and so on is leaked to the global context. I removed them by Remove /@ Select[Names["Global*"],StringMatchQ[#1, "y$" ~~ DigitCharacter ..] &];


and the memory used after this was

MemoryInUse[]

164518400


so they don't seems to have a major impact. If i look into my global context it looks like this

Reverse@SortBy[Map[{#, ByteCount@ToExpression[#]} &, Names["Global*"]], Last] //TableForm

{"finalList", 87591064},
{"y", 0},
{"t", 0},
{"stateDataList$", 0}, {"stateDataList", 0}, {"prepareEquations", 0}, {"output$", 0},
{"output", 0},
{"iMax", 0},
{"i", 0}


So the memory leaked doesn't show up here. Can someone help me figure this out?

-
I get a very similar result, even if I remove the Module and the Prints. ClearSystemCache has no effect. Using Remove on finalList gives me back most of the memory. Note that the data you generate has a function Experimental'NumericalFunction in it. Well.. yeah, undocumented functions.. –  Jacob Akkerboom Oct 9 '13 at 12:18
Yes, I also tried to Remove the finalList and got back most of the memory. Did you manage to find where the memory was hidden? I guess that there are temporary variables which may take up memory here. When I look into the NDSolve context there is a lot of temporary variables but none of them take up signficiant space. Where did you find the Experimental'NumericalFunction ? –  jaclea Oct 9 '13 at 13:29
finalList[[2,1]]//FullForm` I believe is where I found it. I still don't know where the memory went. –  Jacob Akkerboom Oct 9 '13 at 13:38
I still haven't found a solution to the problem above. Suggestions, anyone? –  jaclea Feb 28 at 11:46
Maybe you can try to make a more minimal example. Did you manage to find the Experimental'NumericalFunction? If you can update the question a little bit, I will once again be an "active" question and other people might see it. –  Jacob Akkerboom Feb 28 at 12:06