# Optimal parallelization on Apple Silicon

I have the 2021 13 inch Macbook Pro with the M1 Pro chip (with $ProcessorCount equal to 10), currently running Mathematica 13.0.0. I have had a good experience with serial computations on this chip, but my experience with parallelization has been mixed so far. For instance, I ran a couple of independent RAM-heavy symbolic manipulations of uneven difficulty using ParallelTable, and this ended up taking more time than a serial computation using Table. I believe the issue is that one of the difficult computations ended up on one of the "efficiency" cores and this ended up acting as a bottleneck. A search revealed possibly related issues with machine learning performance reported in this question. However, no final answer seems to have been posted. What are the recommendations for using functions such as Parallelize and ParallelTable and other parallelization tools on Apple Silicon chips optimally? I tried to explore this question using the following benchmark:  TimingTab = Table[t0 = AbsoluteTime[]; LaunchKernels[k]; ParallelTable[{$KernelID, i, Sum[EllipticPi[i/100000., Sin[ j/50.], (\[Pi] j)/4000.], {j,1,1000}]}, {i,1,1000}];
CloseKernels[];
tout = AbsoluteTime[] - t0;
Pause[20];
tout
,{k, 1, 16}];


This is just a simple example of the type of computation I might try to break up using ParallelTable. The Pause[20] is there to allow for the computer to cool slightly and to allow for a fair "adiabatic" comparison. The results can be plotted as

 ListPlot[TimingTab, AxesLabel -> {"Kernels", "Total Evaluation Time [s]"}, Filling -> Axis]
ListPlot[Table[TimingTab[[1]]/(TimingTab[[i]] i), {i, 1, 16}], AxesLabel -> {"Kernels", "Efficiency"}, Filling -> Axis]


There is a distinct minimum at 8 kernels and no speedup (modulo noise) seems to be achieved by launching more kernels. One can also see the obvious break in efficiency at 8 cores. This seems puzzling given the 6+4 setup of the chip. According to the specs the chip should allow for 10 threads. Does the laptop perhaps reserve two threads for the OS?

There also seems to be a possible drop in efficiency after 6 kernels. Perhaps this has to do with employing also the non-performance cores? (But does Mathematica even know to pick performance cores before non-performance ones?) Of course, this example may still not explore the issues I had with some RAM heavy-hitters as mentioned earlier.

EDIT: I was unable to reproduce the original issues with parallelization and higher RAM use through simple examples (e.g. FullSimplify of long randomly generated symbolic expressions). I still believe that the original issues that led me to this question stemmed from edge cases where one nearly exhausts the RAM, and from the fact that the Macbook manages the RAM saturation differently from the Intel PC I used previously. When the RAM is used moderately (up to ~80% of capacity), the scaling stays essentially the same as in the plots above and 6-8 kernels seem to give the best/most frugal performance.

I do not have the time/resources to investigate the edge cases (the FullSimplify saturation of RAM can take hours for each expression). I will accept the answer of Henrik Schumacher with some essential tips for parallelization, but feel free to add additional comments if you had similar experiences or if you have further tips.

• Is it fair to include the time that it takes to start and shut down the kernels in your benchmark? What is the core load before you LaunchKernels? The front-end does have a kernel already running. Jul 13, 2022 at 16:06
• @rhermans I think in practice yes, but I am not sure. It seems to me that the laptop kills inactive kernels, so you have to make sure that an appropriate number is launched every time. This is a small overhead and should not influence the argument too much I believe.
– Void
Jul 13, 2022 at 16:09
• @rhermans As to the second part of the question, I have only one Local kernel for the Mathematica front-end. I am not sure how to check the load from other non-Mathematica processes apart from the fact that 2-4% of the CPU power are shown as in use by the Activity monitor before executing the snippets I show in the OP.
– Void
Jul 13, 2022 at 16:20
• My results (M1 Max Pro, V13.1, 8+2 cores): i.sstatic.net/Gspbz.png — Biggest efficiency drop was 7/8 (not 8/9 as in yours); abs. min. timing was k=11. Don't really know much about internal process management, but system & background processes are running on some processor. They are running according to ActivityMonitor. Jul 13, 2022 at 17:21
• @MichaelE2 Interesting, your absolute time seems to saturate at 6 kernels essentially, and the break in efficiency is much less obvious. This makes me even more puzzled...
– Void
Jul 13, 2022 at 22:05

There are several reasons why parallelization does not scale. Memory bandwidth bottlenecks are one of them (see my comment above). However, I don't think that this problem is really memory bound. I think the issue here is actually Amdahl's law: As rhermans already pointed out, you also time the launch and shut down of the kernels. This is basically serial code and for some reason, the overhead of Mathematica's Parallel constructs is super heavy.

Moreover, depending on the parallelization strategy, there might also be some syncronization overhead during the loop:

1. The cores need to be coordinated by a master kernel, and the latter has to spend a share of its compute power for that. The more inhomogenous the compute powers and jobs are, the more coordination work is need. On an M1 it might be ideal to use one of the efficiency cores as master and let only the performance core do the heavy lifting. Not sure how Mathematica does it.

2. When parallel jobs share a ressource (like writing into a common array) there might be additional waiting times due to write locks (and unfortunately also due to false sharing). (I don't think that this is an issue here.)

To minimize this overhead, one should try

1. to launch the desired number of kernels only once per session, and

2. try to make the jobs evenly sized, and

3. use Method -> "CoarsestGrained" for the cheapest of all parallelization strategies: Just assign roughly the same number of jobs to each core and let them loose.

See here: This is the effect of just using Method -> "CoarsestGrained" on a M1 Max:

CloseKernels[];
TimingTab = Table[
tout = AbsoluteTiming[
LaunchKernels[k];
ParallelTable[
{$KernelID, i,Sum[EllipticPi[i/100000., Sin[j/50.], (\[Pi] j)/4000.], {j, 1,1000}]}, {i, 1, 1000}, Method -> "CoarsestGrained"]; CloseKernels[]; ][[1]]; tout, {k, 1, 8}]; TimingTab Range[Length[TimingTab]]  {73.4668, 75.8862, 77.7967, 79.2004, 80.7508, 83.1171, 85.8353, 91.505} And now without timing the one-time setup costs: CloseKernels[]; TimingTab = Table[ LaunchKernels[k]; tout = AbsoluteTiming[ ParallelTable[ {$KernelID, i,Sum[EllipticPi[i/100000., Sin[j/50.], (\[Pi] j)/4000.], {j,1, 1000}]},
{i, 1, 1000},
Method -> "CoarsestGrained"];
][[1]];
CloseKernels[];
tout, {k, 1, 8}];

TimingTab Range[Length[TimingTab]]


{73.7155, 74.323, 75.9786, 75.1525, 75.9343, 76.3173, 77.8758, 81.1627}

Still not perfect, but a strong indicator, that the setup cost ought not to be neglected.

• Thanks for the answer. I gain the impression that Mathematica maybe does not decide which kernel runs on which core. I think it possibly creates independent processes and lets the OS decide what to do with those. Also - I believe that CoarsestGrained might be a good choice for the batch of elliptic integrals, but not a good choice for a batch of unevenly difficult tasks with execution times that differ at least on the order of seconds (such as a FullSimplify of long symbolic expressions of varying size). There I assume the Method->FinestGrained will actually typically be faster.
– Void
Jul 14, 2022 at 16:48
• "I gain the impression that Mathematica maybe does not decide which kernel runs on which core. I think it possibly creates independent processes and lets the OS decide what to do with those." -- That is likely to be true. I've read somewhere, e.g., that macos even deactivates some processor affinity settings of OpenMP. Let's hope this is for the better of the programmer; after all Apple should know better than the programmer what is going to work well on the new processors. Jul 14, 2022 at 17:46
• "Also - I believe that CoarsestGrained might be a good choice for the batch of elliptic integrals, but not a good choice for a batch of unevenly difficult tasks with execution times that differ at least on the order of seconds" -- Absolutely. That's why I concatenated the three advice points 1., 2., and 3. by an "and". ;) Jul 14, 2022 at 17:48