# Does nesting NIntegrate inside ParallelMap hinder NIntegrate's built-in parallel execution?

I have a function fun[a_,b_] and I want to NIntegrate that function with respect to a for each of about 8 values of b that are stored in bValList. I am considering using

ParallelMap[
NIntegrate[fun[a,#],{a,0,1}]&,
bValList]


I've read that NIntegrate does some things in parallel already. Assuming I have sufficiently many kernels available, would nesting NIntegrate inside ParallelMap prevent NIntegrate from its usual automatic parallel execution?

• Do you have a reference to NIntegrate's parallel capabilities? Sep 2, 2017 at 9:34
• @MariusLadegårdMeyer I mentioned that in multiple answers. I noticed it when looking at CPU usage while running a long integration. You can try it. Sep 2, 2017 at 10:31
• For example, try Do[ NIntegrate[1/x Cos[Log[x]/x], {x, 0, 10}, PrecisionGoal -> 12, MaxRecursion -> 30], {100} ] and watch the CPU usage. The answer is the same to the OP. You can try it on your own. Sep 2, 2017 at 10:32
• Please do not accept my answer for now. I hope someone will explain the strange results. Sep 2, 2017 at 10:54

NIntegrate's parallel capabilities are not documented. It is something I noticed by chance. This means that you should experiment to try to get an answer to your question (and keep in mind that not all integrals may behave the same).

For example,

int[] := NIntegrate[1/x Cos[Log[x]/x], {x, 0, 10}, PrecisionGoal -> 12, MaxRecursion -> 30]

Do[int[], {100}] // AbsoluteTiming
(* {13.0918, Null} *)


During this computation I can see 50% CPU usage on a 4-core, 8-thread CPU. Taking the CPU's "hyperthreading" capabilities into account, this is essentially full utilization of all 4 cores.

Let us now run the same calculation in a single subkernel, and observe the CPU usage again.

LaunchKernels[]
(* {"KernelObject"[1, "local"], "KernelObject"[2, "local"],
"KernelObject"[3, "local"], "KernelObject"[4, "local"]} *)

k = First[%]
(* "KernelObject"[1, "local"] *)

DistributeDefinitions[int]
(* int *)

ParallelEvaluate[Do[int[], {100}], k] // AbsoluteTiming
(* {12.9261, Null} *)


This time I see that only a single core is being used. This indicates that NIntegrate won't work in parallel on a subkernel. However, the timing for the calculation is the same! I do not know why.

Let us now try to run the same Do in parallel, using all four subkernels at the same time.

ParallelDo[int[], {100}] // AbsoluteTiming
(* {3.57534, Null} *)


I see full CPU utilization again, and now the timing is only about 1/4 of the original.

So it seems that NIntegrate does use parallelization very efficiently, and it is still worth running it in an explicit parallel loop (like ParallelDo).

I do not understand why this is. Further investigation would be useful, but I am going to stop here.

• I observe the same with your examples using Mathematica 11.1.1. But when I run them in version 8.0.4 I get the same timings but your first example (evaluation in the master kernel) runs only in one core! So "parallelization" in the master kernel of version 11.1.1 is very suspicious: it doesn't give an advantage in timing but seem to take all CPU cores for nothing! Sep 2, 2017 at 10:56
• @Szabolcs It may be related to similar restrictions on the parallelization of LinearSolve/LAPACK, which I assume "LevinRule" uses in your example. I do not know the internals and cannot really say how wild this guess is. Sep 2, 2017 at 14:58
• @MichaelE2 It certainly is (e.g. MKLThreadNumber system option). What's weird here is that when NIntegrate uses all 4 cores, it still takes just as much time as when it uses only one core. Sep 2, 2017 at 15:00
• @MichaelE2 It is indeed MKLThreadNumber. The problem can be reproduced like this too: 1. check that MKLThreadNumber is 4 at startup, time NIntegrate (13 s). 2. set it to 1, check that NIntegrate uses only one core, but notice that it still takes 13 seconds. BTW this might not happen with all integrands. Sep 2, 2017 at 15:16
• Another wild guess: Cost of overhead = savings of parallelization. The default Levin system is 23 x 23, which size LinearSolve does not seem to parallelize. With Method -> {"LevinRule", "Points" -> 256}, which yields a 513 x 513 system, I see a 33% savings of Do[int[], {10}] over ParallelEvaluate[..] (134 vs 200 sec.). I suspect there's other parallelization going on, too, which makes guesswork difficult. -- Concerning the thread number, one can verify it is limited for subkernels with ParallelEvaluate[SystemOptions["ParallelOptions" -> "MKLThreadNumber"]]. Sep 2, 2017 at 16:25