Where is my memory? -- LibraryLink never returns the memory

Question

Bug introduced in 8.0 or earlier and persisting through 10.3

---

This is an example LibaryLink code from Mathematica documentation, which takes an integer and return an integer list:

 $HistoryLength = 0;    
 Needs["CCompilerDriver`"];

 MMASrc = "
   #include \"WolframLibrary.h\"
   #include \"WolframCompileLibrary.h\"

   DLLEXPORT int demo_I_T(WolframLibraryData libData, 
              mint Argc, MArgument *Args, MArgument Res) {
      MTensor T0;
      mint i, I0, dims[1];
      int err = LIBRARY_NO_ERROR;

      I0 = MArgument_getInteger(Args[0]);
      dims[0] = I0;

      err = libData->MTensor_new(MType_Integer, 1, dims, &T0);
      for ( i = 1; i <= I0 && !err; i++) {
          err = libData->MTensor_setInteger( T0, &i, i*2);
      }
      MArgument_setMTensor(Res, T0);
      return err;
  }
  ";

CreateLibrary[MMASrc, "mylib", "TargetDirectory" -> "/tmp"];   
libFunc = LibraryFunctionLoad["/tmp/mylib.dylib", "demo_I_T", {Integer}, {Integer, 1}];

The library function never returns the memory :

Table[Table[libFunc[100], {40000}]; Pause[0.1]; MemoryInUse[]/1024.^2, {10}]
(* {66.3692, 100.548, 134.727, 168.906, 203.085, 237.264, 271.443, 305.622, 339.8, 373.979} *)

compared to Mathematica function:

mmaFunc[i_] := 2*Range[i]

Table[Table[mmaFunc[100], {40000}]; Pause[0.1]; MemoryInUse[]/1024.^2, {10}]
(* {373.984, 373.984, 373.984, 373.984, 373.984, 373.984, 373.984, 373.984, 373.984, 373.984} *)

Why and how to fix it?

---

A different facet of the same bug is described here.

Answer 1

This is not an issue with LibraryLink as such. It seems to be an issue with how the compiled function interpreter calls a library link function. It's a bug and submitted as such. If you replace your code with

Table[Table[libFunc[100];, {40000}]; Pause[0.1]; 
 MemoryInUse[]/1024.^2, {10}]

there is no memory leak. Note the ; after the call to libFunc[100].