ParallelDo only parallelizes by the innermost variable. Is it possible to distribute the calculation over all the points (all the variable) separately? let me demonstrate:

CloseKernels[]; LaunchKernels[6];
ParallelDo[Print@Row[{i, j, k}, ", "], {i, 2}, {j, 10}, {k, 10}]

This only runs on two kernels, because i only has two values.

enter image description here

Method "FinestGrained" is of no help either.

As a workaround I can do

WorkUnits = Flatten[Table[{i, j, k}, {i, 2}, {j, 5}, {k, 5}], 2];
ParallelDo[With[{i = w[[1]], j = w[[2]], k = w[[3]]},
  Print@Row[{i, j, k}, ", "]
 , {w, WorkUnits}]

enter image description here

But this seems very clumsy.

Is there a more elegant way to parallelize the calculation over all variables, not just the first in a ParallelDo?

  • $\begingroup$ I think there was a similar question about ParallelTable, which has the same behaviour, but I can't find it ... I tend to use a workaround similar to yours: make a flat list of "work items" and then ParallelMap a function over them. $\endgroup$
    – Szabolcs
    Jan 20, 2014 at 18:00
  • $\begingroup$ @Szabolcs Aha, Thanks. Probably a seperate Q, but, do you have a more general way of {i = w[[1]], j = w[[2]], k = w[[3]]}. I remember seeing that somewhere, but can't find it. $\endgroup$
    – Ajasja
    Jan 20, 2014 at 18:21
  • $\begingroup$ When I did parallel calculations like this, I had a function defined for the task at hand. Something like Clear[f]; f[{i_,j_,k_}] := ...; ParallelMap[f, workItems]. It's admittedly not the most concise way if your function is actually quite small, but at that time the important things for me was keeping the code relatively organized and getting the bloody thing done finally (the parallel stuff ran for several hours). So I didn't have much time/energy left to think about the most elegant solution. $\endgroup$
    – Szabolcs
    Jan 20, 2014 at 19:21
  • $\begingroup$ Instead of Flatten@Table I tend to use Tuples[{Range[2],Range[5],Range[5]}] but again this is just a personal preference and it's not actually better than a flattened table. Well, actually Table has the nasty property that it does not generate packed arrays if you have more than a single iterator, while Tuple does generate a packed array. But I think in most practical applications of this type that won't matter much. $\endgroup$
    – Szabolcs
    Jan 20, 2014 at 19:23
  • $\begingroup$ In fact I think that the question I remembered was about why Table doesn't keep arrays packed when there are multiple iterators, and not about parallelization ... the answer to that was that when there are multiple iterators then the result may be ragged, as in Table[{i, j}, {i, 5}, {j, i}]. But I'm digressing and making the comment thread too long. $\endgroup$
    – Szabolcs
    Jan 20, 2014 at 19:28

1 Answer 1


I'm not sure if it would be considered more elegant, but here's an idea:

Since your goal is to have each evaluation go to a separate kernel, one-by-one, we can submit evaluations manually:

   Print@Row[{i, j, k}, ", "]
  {i, 2}, {j, 5}, {k, 5}

One inconvenience with ParallelSubmit is that the easiest way to use it is to submit each evaluation separately (i.e. use the finest possible granularity). If the evaluations are small, then this is not efficient: Mathematica will spend most of its time submitting evaluations and retrieving results, not actually performing the computations. In this case you were asking specifically for the finest granularity, so it might be a good solution for your problem. It's good to be aware of this overhead problem though.

Your workaround doesn't suffer from this problem. Here's a slightly different way to do the same:

workItems = Tuples[{Range[2], Range[5], Range[5]}];

Clear[f] (* make sure there are no leftover definitions from prior uses of f *)
f[{i_, j_, k_}] := Print@Row[{i, j, k}, ", "]

ParallelMap[f, workItems, Method -> ...]

You are still free to quickly set the granularity to a coarser one if needed and reduce the communication overhead. With ParallelSubmit this would require enough extra work to outweigh the other conveniences.

  • $\begingroup$ Defining a function is not a bad idea, and I can get around the {i = w[[1]], j = w[[2]], k = w[[3]]} boring to type and error prone construct:) $\endgroup$
    – Ajasja
    Jan 20, 2014 at 19:50

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.