Is there a way to tell ParallelMap to use just some specific kernels (as is possible for ParallelEvaluate)?

Background: I have 6 local and 16 remote kernels. During some analysis I call ParalleMap twice. One ParallelMap is quicker using all the kernels, but one is actually slower (probably bandwidth limited). This is why I would like to run one ParallelMap on all the kernels and the other only on the local kernels. I don't want to close the kernels in between if possible. I also tried setting the parallelization method to "CoarsestGrained" or "FinestGrained", but that didn't help much.


Could somebody shed light on why the following does not work?

num = 10;
list = Range[num];
function = Labeled[Framed[#], $KernelID] &;
ParallelMap[function, list]
aKer = Kernels[];
Block[{Parallel`Protected`$kernels = aKer[[{2, 3}]]},
   ParallelMap[function, list]

Mathematica graphics

The second map is still run on 4 of the Kernels instead of just two. I traced ParallelMap to



Parallel`Developer`ParallelDispatch[Parallel`Parallel`Private`cmds_] :=

so overriding Parallel`Protected`$kernels should in theory work.


2 Answers 2


Let me give a different approach. One downside of WalkingRandomly's approach is, that he distributes the whole list over all subkernels. The he uses Part in each subcall to select the data to use. I will make this differently:

  • I divide the data into chunks and define each chunk as subdata on every subkernel you want to use in the current call
  • then I can simply call Map[f,subdata] on each wanted subkernel with ParallelEvaluate

The chunkenize works whether Length[data] is divisible by the number of used kernels or not.

chunkenize[data_, nkernels_] := 
 Partition[data, UpTo[Ceiling[Length[data] / nkernels]]]

MyParallelMap[f_, data_, kernels_] := 
 Module[{chunks = chunkenize[data, Length[kernels]]},
    ParallelEvaluate[subdata = #1, kernels[[First[#2]]]] &, chunks];
   ParallelEvaluate[Map[f, subdata], kernels]

Trying it gives

data = Range[20];
f[x_] := {$KernelID, x^2}
kernels = LaunchKernels[];

MyParallelMap[f, data, kernels]

Or if you like



Also I would really like to know why overriding Parallel`Protected`$kernels does not work.

When you trace the output of a simple ParallelMap call, you can investigate what happens. What I did is, I created a full trace output and checked then, on what positions the subkernels like KernelObject[1, "local"] appear.

In detail this meant to check the FullForm of a subkernel because then you see that it has the form


then I launched some kernels and trace the output. Using Position you can find all positions which match a sub-kernel

kernels = LaunchKernels[];
trace = Trace[ParallelMap[$KernelID &, Range[100]]];
pos = Position[trace, Parallel`Kernels`kernel, Infinity];

If you now inspect a bit the positions where the sub-kernels arise, you first find what you found: Parallel`Protected`$kernels. But soon you see

Part[trace,Sequence@@Drop[pos[[10]], -4]]

This brings us to the following solution:

  $KernelCount = 2,
  Parallel`Protected`$sortedkernels = Take[kernels, 2]
 ParallelMap[$KernelID &, Range[100]]

Since I didn't find this that quick, I had time to do some more spelunking. You may have noted, that I set $KernelCount in the Block. This is, because the value of it is used for the partitionizer.

  • $\begingroup$ Yep. That's a better way to do it :) $\endgroup$ Nov 27, 2012 at 0:54
  • 3
    $\begingroup$ @WalkingRandomly Nevertheless you had already the basic idea. And keeping in mind that you are a newbie who made 215 reps yesterday and writes his answers with clear code-boxes and images, I can just repeat: I'm very glad to have you here. $\endgroup$
    – halirutan
    Nov 27, 2012 at 0:58
  • $\begingroup$ thanks :) I find it interesting seeing how other people solve the same problem. I'm off to bed now. 1am here $\endgroup$ Nov 27, 2012 at 1:05
  • $\begingroup$ Actually..one more thing. It would have never have occured to me to use a Block[] within a Module. In fact, I never use Block[]. May I ask why you use it here? Why not just do MyParallelMap[f_, data_, kernels_] := Module[{chunks = chunkenize[data, Length[kernels]], subdata}, MapIndexed[ParallelEvaluate[subdata = #1, kernels[[First[#2]]]] &, chunks]; DistributeDefinitions[f]; ParallelEvaluate[Map[f, subdata], kernels]] $\endgroup$ Nov 27, 2012 at 1:11
  • $\begingroup$ @WalkingRandomly In this case it works without Block because ParallelEvaluate has the HoldFirst attribute. I just like to make sure (even for the reader) that subdata is a new, local variable which should not interfere with any globally defined subdata. The difference between Block and Module is the scoping type: First uses dynamic scoping the latter lexical scopying $\endgroup$
    – halirutan
    Nov 27, 2012 at 1:21

I don't think that you can do this with ParallelMap directly (well, I can't find anything in the documentation at least) but you could emulate such behaviour using ParallelEvaluate. Consider the following example and its output on my quad core laptop

list = Range[num];
function = Labeled[Framed[#], $KernelID] &;
ParallelMap[function, list]

Result of evaluating the above code

The goal of this function is just to frame the input with the $KernelID of the kernel that did the framing printed underneath. Imagine that kernels {3,4} were broken and so I only wanted to work with kernels {1,2}

num = 10;
list = Range[num];
function = Labeled[Framed[#], $KernelID] &;
wantedkernels = {1, 2};
chunk = num/Length[wantedkernels];
DistributeDefinitions[chunk, list, function];

   list[[1 + ($KernelID - 1)*chunk ;; ($KernelID*chunk)]]],
  wantedkernels] // Flatten 

The result of the above code

Performance might suffer from the fact that each kernel gets a fixed amount of work (so no load balancing etc) but perhaps this is enough for your needs?

  • $\begingroup$ One pitfall here is that kernel IDs don't necessarily start at 1. Just run LaunchKernels[];CloseKernels[];LaunchKernels[]. Also losing the load balancing is a major blow. But I appreciated the general idea and will try it out (+1). $\endgroup$
    – Ajasja
    Nov 27, 2012 at 13:58
  • $\begingroup$ Agreed. I only intended to show the basic principal. halirutan's answer uses the same basic idea as mine but is more robust and probably more efficient. Even his solution,howerver, would need more work before it became something that would be fit for general usage. $\endgroup$ Nov 27, 2012 at 14:09

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.