# Are built-in Mathematica functions already parallelized?

I've been noticing something strange since updating to Mathematica 8, and that is that occaisionally I'll see that the MathKernel is using up to 800% CPU in my Activity Monitor on OS X (I have 8 cores). I have no Parallel calls whatsoever, and this is in a single kernel, not across multiple kernels. My code is pretty much only Interpolates, Maps, Do loops, and plotting routines.

I'm curious if some of the built-in Mathematica routines are in fact already parallel, and if so, which ones?

• occaisionally - As in: same function different cpu usage? Or does one function always give 800% cpu? I think the latter can be answered by mathematica users, the former might be independent of mathematica? or a Software bug somewhere Mar 29, 2012 at 17:40
• As a side question, why is Sort not multithreaded, are there not multithreaded implementations? May 28, 2012 at 17:10

A lot of functions are internally multi-threaded (image processing, numerical functions, etc.). For instance:

In[1]:= a = Image[RandomInteger[{0, 255}, {10000, 10000}], "Byte"];

In[2]:= SystemOptions["ParallelOptions"]

Out[2]= {"ParallelOptions" -> {"AbortPause" -> 2., "BusyWait" -> 0.01,
"RecoveryMode" -> "ReQueue", "RelaunchFailedKernels" -> False}}

In[3]:= ImageResize[a, {3723, 3231},
Resampling -> "Lanczos"]; // AbsoluteTiming

Out[3]= {1.2428834, Null}

In[4]:= SetSystemOptions[

Out[4]= "ParallelOptions" -> {"AbortPause" -> 2., "BusyWait" -> 0.01,
"RecoveryMode" -> "ReQueue", "RelaunchFailedKernels" -> False}

In[5]:= ImageResize[a, {3723, 3231},
Resampling -> "Lanczos"]; // AbsoluteTiming

Out[5]= {2.7461943, Null}


## Functions calling optimized libraries

Mathematica surely gets benefit from multi-threaded libraries (such as MKL) too:

In[1]:= a = RandomReal[{1, 2}, {5000, 5000}];

In[2]:= b = RandomReal[1, {5000}];

In[4]:= LinearSolve[a, b]; // AbsoluteTiming

Out[4]= {4.9585104, Null}

In[6]:= LinearSolve[a, b]; // AbsoluteTiming

Out[6]= {8.5545926, Null}


Although, the same function may not get multi-threaded depending on the type of input.

## Compiled function

CompiledFunctions and any other functions that automatically use Compile can be multi-threaded too, using Parallelization option to Compile.

## Caution

1. Measuring timing with AbsoluteTiming for multi-threaded functions could be inaccurate sometimes.

2. The performance gain is usually not direct proportion to the number of threads. It depends on a lot of different factors.

3. Increasing number of threads (by using SetSystemOptions ) more than what your CPU support (either physical or logical cores) is not a good idea.

• Thanks for the very informative answer. Is there any list available of the routines that are multi-threaded? The reason I ask is that if I do decide to use Parallel calls, having each processor execute 8 threads would result in 64 threads total, which I presume is much less efficient. Mar 29, 2012 at 18:42
• I am not aware of such a list, but I will post if I find one. Also, it depends on the situation. For instance LinearSolve doesn't get multi-threaded if it computes symbolically. And yes, Parallel call is less efficient for obvious reason, sometimes necessary. Mar 29, 2012 at 18:43
• You said, "Measuring timing with AbsoluteTiming for multi-threaded functions could be inaccurate sometimes.". Do you mean by any change that Timing is inaccurate and AbsoluteTiming is accurate? I thought AbsoluteTiming returned wall time, while Timing returned the sum of the amounts of time used by each core. Or is this platform dependent? May 28, 2012 at 15:39

Some, yes. For example, many image processing functions are multi-threaded. One way to see whether a function is leveraging multi-threading is to perform the same kind of tests as for ImageResize in Yu-Sung Chang's answer. You will witness speed-up for dozens of functions, including for example (list really not exhaustive): Binarize, Dilation, Erosion, ImageAdjust, ImageDeconvolve, MeanShiftFilter, MedianFilter, MorphologicalBranchPoints, MorphologicalGraph, Pruning, Radon, TotalVariationFilter, WienerFilter, etc.

• Can you give any data to back up this assertion? (I believe you, but data is needed.) Mar 30, 2012 at 18:25
• @rcollyer Fair enough... One way to see whether a function is leveraging multi-threading is to perform the same kind of tests as for ImageResize in Yu-Sung Chang's answer. You will witness speed-up for dozens of functions, including for example (list really not exhaustive): Binarize, Dilation, Erosion, ImageAdjust, ImageDeconvolve, MeanShiftFilter, MedianFilter, MorphologicalBranchPoints, MorphologicalGraph, Pruning, Radon, TotalVariationFilter, WienerFilter, etc. Mar 30, 2012 at 18:50

Some matrix operations seem to use more than one core. For instance,

Eigensystem[RandomReal[{-1, 1}, {2000, 2000}]]


uses two cores on my machine. This also works for sparse matrices:

ClearAll[tmparray];
tmparray = SparseArray[
{
{i_, j_} /; Abs[i - j] \[LessEqual] 2 \[Rule] RandomReal[]
},
{100000, 100000}
];

Eigenvalues[
tmparray,
-40
]


Probably this is done by the libraries that Mathematica calls.