I'm working on a PC with 10 processor cores (20 threads), so it can support 20 concurrent computations. I often run simulations in the background on my PC such that some fraction of these, say 12, are occupied. I would like to write a function myLaunchKernels[] that launches (n) kernels, where (n) is the number of unoccupied processor threads [give or take two for system processes]. So in my dummy example, it would launch 8 kernels minus however many designated for system processes. [This may seem trivial and for me (where I can usually remember how many are in use) it often is, but it might be useful to others using distributed computing or with HPC applications with more cores and concurrent jobs.]

This essentially just requires an ability to - in Mathematica - do what one can do manually by looking at Task Manager in Windows and counting how many processes are active and consuming appreciable CPU power. If this numberActiveProcesses function returned the 12 from the above example, it would be trivial to feed this into LaunchKernels after subtracting a couple for system usage.

It would be similar to the standard $ProcessorCount; is there a similar (maybe undocumented) function for detecting the number of threads in use at the moment?

  • $\begingroup$ On HPC clusters, it should be possible to query the number of available cores from the job scheduler. Actually, normally one requests a certain number of cores. I do not think that the concept of "free cores" is very clearly defined. But on a cluster you are always expected to be a good citizen and use a certain number of cores, even if technically you could use more. $\endgroup$ – Szabolcs Nov 1 '18 at 19:47

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Browse other questions tagged or ask your own question.