# Effective use of idle cores when using parallelization

Assume I have 10 cores in a CPU. And I run ParallelMap on them:

LaunchKernels[10];
ParallelMap[F1,L1];


where L1 is a list of the dimension {10,1} and F1 is whatever function. Thus, all 10 cores start working hard computing F1, each core computes F1 for a pointed element in L1. The issue is that each of the cores finish its work in their own time, and when some are being idle with finished tasks, the others continue computing.

Now, assume, I have to compute several sequent ParallelMaps commands:

LaunchKernels[10];
ParallelMap[F1,L1];
ParallelMap[F2,L2];
...


With such the order of commands, Mathematica implements the lines step by step and cannot run the second ParallelMap until all elements in the first become computed. (Please, see UPD below)

Are there ways to load the idle cores with the second task, not waiting for the ending of the first one? I can assign 5 cores for “task 1” and 5 for “task 2”, but I am sure there are more efficient and elegant ways.

P.S. I am pretty new in Mathematica, and might not know the propper nomenclature and key words. Please, refer me to the topics I duplicate or comment here.

UPD After @Domen 's response below, I have to specify the condition: I would like to have access to the output of the finished calculations made with ParallelMap[F1,L1], while ParallelMap[F1,L1] is still being run.

You can join all your "jobs" into one list, then call ParallelMap once.

joinJobs[specs__] := Catenate@Table[{s[[1]], #} & /@ s[[2]], {s, {specs}}]

L1 = Array[a, 2];
L2 = Array[b, 4];
L3 = Array[c, 3];

jobs = joinJobs[{F1, L1}, {F2, L2}, {F3, L3}]
(* {{F1, a[1]}, {F1, a[2]}, {F2, b[1]}, {F2, b[2]}, {F2, b[3]},
{F2, b[4]}, {F3, c[1]}, {F3, c[2]}, {F3, c[3]}} *)

ParallelMap[#[[1]][#[[2]]] &, jobs]
(* {F1[a[1]], F1[a[2]], F2[b[1]], F2[b[2]], F2[b[3]], F2[b[4]], F3[c[1]],
F3[c[2]], F3[c[3]]} *)

• Your response helped me to specify the issue I concerned of: I would like to load the CPU's and keep tasks distinguished to get the output timely when calculations of each task is finished. What you propose should definitely work, thank you ! However, I would like to have access to the output of the finished calculations made with ParallelMap[F1,L1], while ParallelMap[F2,L2] is still being run. Sounds insignificant, but my ParalellMap’s takes about a few weeks each, and I prefer to work with intermediate results, not waiting for the end of the full computations. Thank you ! Commented Mar 7 at 12:57
• Why don't you write the results to a file? Commented Mar 7 at 14:37
• 1) Correct me, you suggested gathering all the functions in a pool and apply ParallelMap to them. Whether we can interrupt ParallelMap and withdraw an intermediate result ? If you have a solution I will appreciate it if you share it. I do utilize Save[...] after each ParallelMap. 2) I am working with a cluster of 8 nodes (PC's) with 20 cores per node. I am not sure, one can safely address the nodes memory while they are in the work process. 3) I do believe one can do it simpler. There has to be a Mathematica command to run the next command if a core is idle after the previous. Commented Mar 7 at 16:50
• In fact, the approach you suggested, might be described as following: Parallelize of the stack {ParallelMap[F1,L1],ParallelMap[F2,L2]}. I did not manage to handle with the syntaxis # \$@ yet, pardon. It is likely, I will need ParallelSubmit, it sounds close to what I think of. But it is not clear how to handle it. Commented Mar 7 at 17:07