# how to nest parallelized computations on a cluster?

I am lucky to have access to a computational cluster and I could submit my job to many-many cores. As I need to evaluate a numerical integral within a numerical integral, I would like to know if I can set things up (e.g. during the compilation of the inner integral as compiled function) in a way that would run the inner function on 8 cores (on a single node) and distribute the outer one across nodes in the cluster?

tmp = Compile[
{{z,_Real},{zz,_Real}},
NIntegrate[1 - \[Xi]u[zzz]/\[Xi]c[zzz], {zzz, z, zz},Parallelization -> True]
]
g[z_] := NSolve[
T'[z]/(1 - T'[z]) - (k SurvivalFunction[H,z]/(z PDF[Hstar, z]))
*NIntegrate[(1 - g) Exp[tmp[z, zz]] PDF[H, zz]/SurvivalFunction[H, z],
{zz, z, \[Infinity]}], g, Reals]
N[g[2000000], 10]


Simple LaunchKernel and Parallelize did not seem to give me this control. (If it made sense in the first place...)

Separately, I would evaluate my function (g) for many points (a List), but as all for all points g uses the integrals over the right tail of the same distribution, actually it sounds like a waste of time to calculate them separately, even if I do it within a single call like ParallelTable. Would Mathematica notice that it could use some calculations across the evaluations? Or shall I explicitly separate the useful object first? Or can I call some clever Fold? Could any of this be parallelized? (Not trivial if I try to be cheap and ask Mathematica to solve for a function only implicitly defined in an expression.)

EDIT: Actually the code as given does not run, but gives the error

CompiledFunction::cfsa: "Argument zz at position 2 should be a !(\"machine-size real number\")."

Probably because of the outer integral having upper limit of infinity?

-
I'm afraid this code doesn't make much sense conceptually. Parallelization is an option of Compile, not NIntegrate, and NIntegrate isn't compilable anyway. Compile with Parallelization -> True is not a magic bullet for parallelization either; it only does anything for Listable functions. ParallelTable may in fact be useful for calculating g at many points, but any scheme for nested parallelization must be constructed explicitly. –  Oleksandr R. May 19 '12 at 21:23

## 1 Answer

I asked a similar question a while back. Bottom line, Mathematica's grid computing functionality does not directly support nesting of parallel processing.

To optimize parallelization you need to determine which level of processing you can best apply it to.

-
thanks. Does it mean I should anything differently? E.g. launch different numbers of kernels at different times? Or just launch the maximum at the beginning, and if Mathematica wants to spread the inner function across all, so be it? Or a clever way around that would be to compile that without parallelization? (By the way, I still don't get compilation did not work here...) –  László May 19 '12 at 14:16
I'd launch all available kernels at the beginning. You could try compiling without the parallelization, then using DistributeDefinitions[] of the compiled function before parallelizing it. I haven't studied why your compile doesn't work, but you may want to provide simplified version of it in another question. Also its customary to vote up helpful answers and select good answers to your questions. Welcome to Mathematica SE. –  Jagra May 19 '12 at 15:36
Jagra, thanks, I would vote it up if I had the reputation (I knew SE before :). –  László May 19 '12 at 21:12