Mma 10: Half the parallel power (Macs)?

Here is a comparison of the parallel kernels launched under Mathematica under v9 and v10, on the same identical current 2014 R2-D2 Mac Pro ...

[ Update: Valerio has commented that the same issue arises on the Macbook Air.]

$ProcessorCount  12 Issuing:  LaunchKernels[]  ... launches 12 kernels, and actually uses them ... notice that the ParallelTable is 12 times the speed of Table[] for this construct: In[5]:= Table[Pause[1]; f[i], {i, 12}] // AbsoluteTiming  Out[5]= {12.003106, {f[1], f[2], f[3], f[4], f[5], f[6], f[7], f[8], f[9], f[10], f[11], f[12]}} In[6]:= ParallelTable[Pause[1]; f[i], {i, 12}] // AbsoluteTiming  Out[6]= {1.010648, {f[1], f[2], f[3], f[4], f[5], f[6], f[7], f[8], f[9], f[10], f[11], f[12]}} So, to perform the same operation, the parallel result under v9 is 12 times the speed of the single kernel result. Under v10 -- half my potential processing power has gone $ProcessorCount


6

... down from 12 - even though I am running on the identical machine. Now, I know that my Mac Pro actually has 6 processors, and each runs 2 threads ... and under v9, that yielded 12 processor kernels for Mma 9 ... but under v10, it is only yielding 6 kernels ... ON THE SAME MACHINE. And this has real effects ... it effectively reduces by 50% the maximum potential power of my Mac:

 LaunchKernels[]


... launches 6 kernels (not 12 kernels as under v9).

Compare the performance:

 In[3]:= ParallelTable[Pause[1]; f[i], {i, 12}] // AbsoluteTiming


Out[3]= {2.009933, {f[1], f[2], f[3], f[4], f[5], f[6], f[7], f[8], f[9], f[10], f[11], f[12]}}

So, under the new v10, I am getting half the parallel performance here and half the kernels that I got under v9. Even more perplexing is that this worked fine in an earlier pre-release version of v10.

I am very confused. Anyone have any ideas how I can get my missing kernels back? Or why a decision may have been made to hobble the performance of the Mac Pro under v10?

Just noticed that if I go to:

• Evaluation Menu -> Parallel Kernel Configuration

... the automatic setting for:

• Number of kernels to use: is set to: Automatic (which Mma sets to 6)

If I change this to:

• Manual setting

and set it to 12 ... then it seems to use 12.

But I am still confused as to why, if Mathematica 10 can actually support 12 kernels on the machine, ... why would Wolfram set it to use only half of them by default, when v9 supported all of them by default?

Szabolcs suggests below that Mathematica may not practically use more kernels than physical cores, even if your processor supports virtual cores ... so there is no real difference. In reply, here is a quick timing test of a real-world application (kernel density estimation) from the mathStatica benchmarking test suite. The task is to plot 12 kernel density estimates, corresponding to 12 different bandwidths.

bandwidths = {.2, .35, .45, .55, .65, 1, 1.5, 2, 2.2, 2.5, 3, 3.2};


Here are the results running under:

• v9 (default: 12 kernels): 3.38 seconds
• v10 (default: 6 kernels): 9.53 seconds
• v10 (manual overide to 12 kernels): 7.46 seconds

I don't know what has changed to cause such a performance hit under v10 ... but even so, that is not the point. The point is that the v10 default kernel setting fails to take advantage of the power of the Mac Pro ... and results in worse performance in a typical parallel-processing application.

More extensive real-world test:

Update: 1 August 2014

I have now had the opportunity to run the full mathStatica (primarily symbolic) benchmark suite under both:

• the default v10 parallel setting (6 kernels)
• the manual override v10 setting (12 kernels)

Here are the results:

The results fall into 2 categories:

• For problems that have more than 6 separate components to them: ... For such problems, using 12 kernels is ALWAYS unambiguously faster, and significantly so.

• For problems that have 6 or less separate components: ...For instance, Examples 7 and 9 can only be broken down into 2 symbolic components, so the benefits of parallelism max out with 2 kernels. In these cases, the 6 automatic kernels case is sometimes marginally faster than the 12 kernel case (presumably due to running overheads etc) ... but the difference is tiny, and essentially unnoticeable.

In summary: for problems that CAN benefit from more than 6 kernels, the default Mma 10 (automatic) setting of 6 kernels on a Mac Pro appears to be sub-optimal, and fails to take advantage of the full capability of the machine. This problem is new to v10, and does not occur under v9.

-
+1. This is a very well-researched post and even the example is pretty. That said, I would suggest to entertain the possibility that the cause of the performance hit is not directly due to the number of kernels launched or even the number of recognized cores in v10 but that it could be due to some other reason. It may also be worthwhile to track what MMA is doing in Activity Monitor. –  heropup Jul 11 '14 at 14:53
Something else that is also worth asking is whether we see a similar performance hit between v9 and v10 on Windows. –  heropup Jul 11 '14 at 16:36
What functions are run in the benchmark? –  rcollyer Jul 11 '14 at 18:05
@rcollyer mathStatica's NPKDEPlot function uses ParallelTable[Plot[ Funky ] ] to produce each separate curve on a separate kernel, where Funky is a Compiled application of (Map of Total of some Ifs and Buts). –  wolfies Jul 11 '14 at 18:28
I have the same issue on my MacBook Air: by default Mathematica 10 uses 2 instead of the 4 cores that Mathematica 9 would use. –  Valerio Jul 31 '14 at 14:17

I believe that this is an intentional and beneficial change in v10.

Mathematica 9 was not able to correctly detect the number of physical cores, and it launched as many kernels as the number of virtual cores (which is double when using HyperThreading).

Mathematica 10 can now detect the number of physical cores correctly and will launch only as many kernels as necessary to achieve optimal performance for most applications.

The benchmark you used is flawed because it doesn't use the CPU (it uses Pause). If you try a real benchmark, you'll find that the majority of applications doesn't benefit from launching more kernels than the number of physical cores. In fact in many cases performance will suffer because Mathematica's way of parallelization does incur a non-negligible overhead with each additional kernel.

Launching more kernels than physical cores is not worth it because the performance benefits of HyperThreading are minor while there is a parallelization overhead introduced with each additional kernel. Not to mention to memory overhead of each additional kernel.

-
@wolfies If you find that your application really does benefit from launching more kernels than the number of cores you have, you always have the option to do that. This will depend on the particular application. It is not true that "half the parallel power is gone". The default is different, and IMO more sensible. You still have the possibility of manual override. I often limited the number of kernels in v9 manually, now you'll have to increase them in v10 manually. Nothing has changed for the worse, as your post would imply. –  Szabolcs Jul 11 '14 at 14:49
@wolfies The decision of how many kernels to use is up to you, the user and there's no default that applies to all situations. You show an example where more kernels help. This is not always the case. Try e.g. dict = {"a", "b", "c"}; strings = Table[StringJoin@RandomChoice[dict, 10], {4000}]; Parallelize@Outer[HammingDistance, strings, strings]; It won't help at all. It might actually hurt a bit. You asked why v10 launches fewer kernels by default---I explained it. You already know how to override that. What is the remaining question? –  Szabolcs Jul 11 '14 at 15:21
The gain from SMT can be up to 100%, so it isn't always minor, although this large benefit isn't available for most loads. (The information on Wikipedia is over a decade out of date and not applicable to current processors. HT on the P4 was something of a hack to mask the low single-thread performance of that design, but the situation is much improved now.) Nonetheless I agree totally with your answer and think the current default is going to be better than the previous one in almost all cases. What I do wonder about is CPU affinity, as if this is not right, there will be performance issues. –  Oleksandr R. Jul 11 '14 at 15:57
@heropup Sounds like a good idea. Why don't you write another answer (community wiki if you like, to make contributions easier) and start collecting some benchmarks? –  Szabolcs Jul 11 '14 at 16:46
@Szabolcs I don't agree that this change is beneficial. I have been using all the available virtual cores with very good efficiency, it is just a matter of reducing the overhead. As a concrete example, to run my turboGL code (www.turbogl.org - set the parameter nst to nst=10^6) on my MacBook Air with Intel i7 it takes 69s with Mathematica 9 and 123s with Mathematica 10 (default configuration). –  Valerio Jul 31 '14 at 14:29