Take the 2-minute tour ×
Mathematica Stack Exchange is a question and answer site for users of Mathematica. It's 100% free, no registration required.

I just attempted to run code that had nested ParallelMap[] functions. It generates the error message:

ParallelMap::subpar: Parallel computations cannot be nested; proceeding with sequential evaluation.

I can represent a simplified version of the code's structure with the following:

f[list_] := ParallelMap[# + 0.1 &, list]
dataLists = RandomReal[{0, 1}, {5, 100, 3}];
Dimensions[dataLists]
ParallelMap[f[#] &, dataLists];

When ParallelMap calls f[datLists] I have extra kernels available for use, but it appears that whatever manages the distribution of processing jobs can't reach them.

I can appreciate that nesting parallel computations presents some complications, but not having the ability to nest them doesn't seem very Mathematica like.

As my code has many functions at the same (2nd) level in the hierarchy as f[list_], this constrains my possible solutions quite a bit and makes optimizing performance by using parallel computation time consuming. I think I can either parallelize at the top level or parallelize all the functions at the second level, but obviously not both.

Also, this 2nd level of functions tend to greater complexity than the simple example above. They look more like:

If[#[[1]] == 0, 0, 1 - NProbability[x < #[[1]], x \[Distributed] #[[2]]]] &, 
    Transpose[{listOfReals, myDistributions}]

Just speculative brainstorming, but maybe (probably) I haven't thought of all the possible ways to attack a problem like this.

  • Rather than mapping or parallel mapping, could I recast this problem in other ways better suited to the single available level of parallelization?
  • Could one flatten the computations then cull out the answers afterwards?
  • Could one turn this into a matrix operation of some kind?

As CUDA and OpenCL don't look like a ready or easy solution for me (see: CUDA and GPU hardware and compatibility questions) I'd hoped to get everything I could out of Mathematica's parallel powers.

share|improve this question
1  
Could you just Quiet the message and get sequential evaluation automatically at all levels except the outermost? –  Andrew Moylan Feb 27 '12 at 22:31
add comment

1 Answer

up vote 9 down vote accepted

This limitation exists, and as far as I can see we have to live with it for now. It does indeed make it impractical to use Parallel operations inside functions---the functions won't work in parallel kernels.

I recommend you parallelize at the top level unless you have a specific reason to do it at lower levels. The longer the a "unit calculation" that is run in a subkernel, the less parallelization overhead will matter. This overhead can be significant if you are running the subkernels on different machines from your main kernel (as you seem to be based on your Lightweight Grid question). Could you give some indication of how long it takes for your functions to finish, and how long the list dataLists is? If dataLists is long enough compared to the number of processors you have available, then probably parallelization at the highest level will be most efficient.

Alternatively you could just use Off[ParallelMap::subpar].

share|improve this answer
2  
For a ParallelMap equivalent that works both in the master kernels and subkernels without producing messages, one might try: maybeParallelMap := If[System`Parallel`$SubKernel, Map, ParallelMap]. Multiple levels of parallelization aren't easily handled within the framework of the Parallel` package, not least since subkernels can't (officially) communicate with one another except via the master kernel. –  Oleksandr R. Feb 27 '12 at 22:42
    
@OleksandrR. You raise an interesting point. In my situation my processes run discretely at both the top and secondary levels. By this I mean the top level parallelizations have no passing of information between them. At the 2nd level, again no passing of information between any set of process running within a top level process. Certainly each of the discrete top level process will receive and aggregate information returned from sub level process, but this really just gives us the most elementary way of parallel processing. –  Jagra Feb 28 '12 at 1:22
    
I have an extensive program. 30+ custom functions & +300 lines of functional code. 3 top level data sets of {12000, 2} dimensions. Code generates a subset of permutations of each. Lots of processing - up to 90 min. to run on 2 core iMac. 4 types of calculations take the most time: Fitting parameters for custom distributions; NExpectation[] at ea. time step (using the above distributions); NSolve[]to find the intersection of pairs of PDFs; & NProbability[]. All these map over lengthy lists, I might get the most performance boost from parallelizing these at level 2 rather then just level 1. –  Jagra Feb 28 '12 at 1:41
1  
@Jagra if you have a lot of independent computations and simply want better control over the parallelization granularity then I would suggest trying something with ParallelSubmit. This can be wrapped around any part of the overall computation at any level. –  Oleksandr R. Feb 28 '12 at 1:55
    
@OleksandrR. -- I'll definitely take a look at ParallelSubmit tomorrow. Thanks. –  Jagra Feb 28 '12 at 4:31
add comment

Your Answer

 
discard

By posting your answer, you agree to the privacy policy and terms of service.

Not the answer you're looking for? Browse other questions tagged or ask your own question.