I just attempted to run code that had nested ParallelMap[]
functions. It generates the error message:
ParallelMap::subpar: Parallel computations cannot be nested; proceeding with sequential evaluation.
I can represent a simplified version of the code's structure with the following:
f[list_] := ParallelMap[# + 0.1 &, list]
dataLists = RandomReal[{0, 1}, {5, 100, 3}];
Dimensions[dataLists]
ParallelMap[f[#] &, dataLists];
When ParallelMap
calls f[datLists]
I have extra kernels available for use, but it appears that whatever manages the distribution of processing jobs can't reach them.
I can appreciate that nesting parallel computations presents some complications, but not having the ability to nest them doesn't seem very Mathematica like.
As my code has many functions at the same (2nd) level in the hierarchy as f[list_]
, this constrains my possible solutions quite a bit and makes optimizing performance by using parallel computation time consuming. I think I can either parallelize at the top level or parallelize all the functions at the second level, but obviously not both.
Also, this 2nd level of functions tend to greater complexity than the simple example above. They look more like:
If[#[[1]] == 0, 0, 1 - NProbability[x < #[[1]], x \[Distributed] #[[2]]]] &,
Transpose[{listOfReals, myDistributions}]
Just speculative brainstorming, but maybe (probably) I haven't thought of all the possible ways to attack a problem like this.
- Rather than mapping or parallel mapping, could I recast this problem in other ways better suited to the single available level of parallelization?
- Could one flatten the computations then cull out the answers afterwards?
- Could one turn this into a matrix operation of some kind?
As CUDA and OpenCL don't look like a ready or easy solution for me (see: CUDA and GPU hardware and compatibility questions) I'd hoped to get everything I could out of Mathematica's parallel powers.