I recognised that for some reason when I use Compile and use a C-compiler the memory consumption of Mathematica's MathKernel is very large even after I reset all the stored data or variables with Clear["Global`*"].

I thought the aforementioned command would reset just everything that is stored inside the kernel. Why is Mathematica still occupying a huge amount of memory. Is the overhead of Compile so big and what can I do to improve it?


My problem seems to be related to the Parallelization option within Compile. If enter my arguments as follows


the huge overhead during the evaluation of cfun is removed after calling cfun and only the output of cfun occupies memory (no matter the value of the Parallelization option within compile).

However, if I call Parallelization->True together with RuntimeAttributes -> {Listable} and insert the arguments as follows


then even for n=1 a huge overhead is produced that eventually occupies more memory than the actual output of cfun. The goal why I wanted to write the arguments as lists and use the listable attribute was to use effective parallelization when increasing n to higher integers like n=3.

So for my program, with Parallelization->False and the latter technique of entering the arguments, my program occupies eventually only 4.9% of my memory. Setting Parallelization->True with the same length of output it consumes 19.3% of my memory.

This is a remarkable huge overhead when calling the Parallelization option. Is there any way to improve it, or is this option simply like a black box?


I think this thread might be slightly related to my problem:

Listable compiled function crashes the kernel with "no more memory available"

  • $\begingroup$ If you genuinely want to reset everything in the kernel I would use Quit. $\endgroup$ – user6014 Feb 8 '18 at 15:34
  • $\begingroup$ Quit quits the kernel yes. But I want to find out what is causing the huge memory consumption. Are there any commands to see what Mathematica stores and where it is stored? Is it necessary for Compile to store something somewhere which I am not aware of? Or should Compile only occupy as much memory as the output of the compiled function requires? $\endgroup$ – Display Name Feb 8 '18 at 15:42
  • $\begingroup$ Ok, so if I use $HistoryLength = 0 and then clear my compiled function cfun then all the memory seems to be accessible again. Within my Compile I use a Block environment with many local variables. Is there a way to just clear everything within my compiled function after using it, but keeping the function itself available? (if I use clear I need to re-input the function again if I want to use it again) $\endgroup$ – Display Name Feb 8 '18 at 16:04
  • $\begingroup$ See my Edit in the OP. I figured out that my problem is related to the Parallelization option within compiled. $\endgroup$ – Display Name Feb 9 '18 at 8:20
  • $\begingroup$ Without a simple example to test, it is hard to say what happens on your machine. $\endgroup$ – halirutan Feb 25 '18 at 5:52

Browse other questions tagged or ask your own question.