# Does Eigenvalues evaluate in a parallelized way?

I use mathematica on a computer with linux operating system. The computer has 2 cpus and each cpu has 4 cores, so there are totally 8 cores available.

Now I got confused with whether the evaluation of Eigenvalues is parallelized or not. I present two kinds of codes below

one use ParallelDo

ParallelDo[
Eigenvalues[# +
ConjugateTranspose[#] &[Table[RandomReal[], {i, 5000}, {j,5000}]]
];
,{4}]//AbsoluteTiming


the time it takes is 135.12254 second. And use Top command in linux during the evaluation, it shows like below:

It show that 4 cores are trying their best, that's quite reasonable.

the other use Do only

Do[Eigenvalues[# + ConjugateTranspose[#] &[Table[RandomReal[], {i, 5000}, {j,5000}]]];,{4}]//AbsoluteTiming


this time it takes 109.004395 second. It evaluates even faster!!. And use Top, It shows like below:

Almost 800%, it means that this time, the evaluation uses all the cores? I don't know why it shows one 800% instead of eight 100%s. What is the difference?

If Eigenvalues really evaluates in a parallelized way automatically, why Parallelize[Eigenvalues[matrix]] gives error message saying that " Eigenvalues can't be parallized;proceeding with sequential evaluation"??

• Hit H in top, that should show all user threads – ssch Jun 1 '13 at 13:44
• @ssch yeah! it shows all. I never hit H before. So 800% is the same as eight 100%s in essence, right? – matheorem Jun 1 '13 at 13:52
• @matheorem Because Eigenvalues doesn't use Mathematica's parallelization. Parallelize can only deal with high level Mathematica constructs while Eigenvalues is implemented in a low level language for dense numerical matrices. Mathematica is just calling this low level multithreaeded implementation. – Szabolcs Jun 1 '13 at 14:58
• The reason why top displays 100% four times is because there are four processes (namely subkernels) doing calculation using a single thread each. I guess the idea is that if you explicitly parallelize, you don't want Mathematica to interfere with your parallelization strategy. – celtschk Jun 1 '13 at 15:53
• However it has a natural followup question: Which Mathematica functions use thread-level parallelism? – celtschk Jun 1 '13 at 19:18

Eigenvalues (and some other builtin functions) may internally use a very efficient multithreaded implementation (depending on what sort of matrix it's applied to, i.e. sparse or dense, exact or inexact, etc.)

It is important to understand that this does not use Mathematica's parallelization framework and it is not user-controllable, apart from setting how many threads to use. You can read more about this here.

How many threads to use by these functions is controlled by the "MKLThreads" system option:

SystemOptions["MKLThreads"]


Update: In recent versions of Mathematica the option is SystemOptions["ParallelOptions" -> "MKLThreadNumber"].

This is set to the number of cores of your CPU on the main kernel, however, it is set to 1 on all subkernels. This explains the CPU usage patterns you see. 800% means that the process is using 8 cores.

Generally, when using a function that is already internally multithreaded, manual Parallelizeation will either not do anything or will slow things down.

• There are a few other controls available via environment variables. But these are mainly about CPU affinity, whether the threads are persistent across invocations, whether the workload is partitioned statically or dynamically, &c. Most users probably don't want or care to change these. – Oleksandr R. Jun 24 '14 at 8:26
• @FredSimons Thank you, corrected! – Szabolcs Feb 17 '16 at 10:08

I don't think you are Parallelize-ing the Eigenvalues. What your code is doing is generating 4 5000X5000 random Hermitian matrix and find the eigenvalues. You can check with a smaller order (say 5X5), like

m = # + ConjugateTranspose[#] &[Table[RandomReal[], {i, 5}, {j, 5}]];
ParallelDo[Eigenvalues[m];, {4}] // AbsoluteTiming
Do[Eigenvalues[m];, {4}] // AbsoluteTiming
Table[Eigenvalues[m], {4}] // TableForm


I define the matrix in m to make sure ParallelDo and Do are using the same matrix. For my laptop execution time for ParallelDo and Do are 0.011845 and 0.000189 (Do wins again). Last Table command will show you what exactly is happening. You can check each loop produce a complete set of eigenvalues. It is not like all the loops together produce the complete set.

Now regarding your query - why the parallelization takes more time than series execution - well, I don't know that well. Only thing I can say it is the effect of interdependence of kernels which sometimes also depends on how you set your loops. From my experience I would suggest that it is always good to do a test drive with both series and parallel to see what suits better with your code.