Are built-in Mathematica functions already parallelized?

Question

I've been noticing something strange since updating to Mathematica 8, and that is that occaisionally I'll see that the MathKernel is using up to 800% CPU in my Activity Monitor on OS X (I have 8 cores). I have no Parallel calls whatsoever, and this is in a single kernel, not across multiple kernels. My code is pretty much only Interpolates, Maps, Do loops, and plotting routines.

I'm curious if some of the built-in Mathematica routines are in fact already parallel, and if so, which ones?

Answer 1

Natively multi-threaded functions

A lot of functions are internally multi-threaded (image processing, numerical functions, etc.). For instance:

In[1]:= a = Image[RandomInteger[{0, 255}, {10000, 10000}], "Byte"];

In[2]:= SystemOptions["ParallelOptions"]

Out[2]= {"ParallelOptions" -> {"AbortPause" -> 2., "BusyWait" -> 0.01,
    "MathLinkTimeout" -> 15., "ParallelThreadNumber" -> 4, 
   "RecoveryMode" -> "ReQueue", "RelaunchFailedKernels" -> False}}

In[3]:= ImageResize[a, {3723, 3231}, 
   Resampling -> "Lanczos"]; // AbsoluteTiming

Out[3]= {1.2428834, Null}

In[4]:= SetSystemOptions[
 "ParallelOptions" -> {"ParallelThreadNumber" -> 1}]

Out[4]= "ParallelOptions" -> {"AbortPause" -> 2., "BusyWait" -> 0.01, 
  "MathLinkTimeout" -> 15., "ParallelThreadNumber" -> 1, 
  "RecoveryMode" -> "ReQueue", "RelaunchFailedKernels" -> False}

In[5]:= ImageResize[a, {3723, 3231}, 
   Resampling -> "Lanczos"]; // AbsoluteTiming

Out[5]= {2.7461943, Null}

Functions calling optimized libraries

Mathematica surely gets benefit from multi-threaded libraries (such as MKL) too:

In[1]:= a = RandomReal[{1, 2}, {5000, 5000}];

In[2]:= b = RandomReal[1, {5000}];

In[3]:= SystemOptions["MKLThreads"]

Out[3]= {"MKLThreads" -> 4}

In[4]:= LinearSolve[a, b]; // AbsoluteTiming

Out[4]= {4.9585104, Null}

In[5]:= SetSystemOptions["MKLThreads" -> 1]

Out[5]= "MKLThreads" -> 1

In[6]:= LinearSolve[a, b]; // AbsoluteTiming

Out[6]= {8.5545926, Null}

Although, the same function may not get multi-threaded depending on the type of input.

Compiled function

CompiledFunctions and any other functions that automatically use Compile can be multi-threaded too, using Parallelization option to Compile.

Caution

1. Measuring timing with AbsoluteTiming for multi-threaded functions could be inaccurate sometimes.

2. The performance gain is usually not direct proportion to the number of threads. It depends on a lot of different factors.

3. Increasing number of threads (by using SetSystemOptions ) more than what your CPU support (either physical or logical cores) is not a good idea.

Answer 2

Some, yes. For example, many image processing functions are multi-threaded. One way to see whether a function is leveraging multi-threading is to perform the same kind of tests as for ImageResize in Yu-Sung Chang's answer. You will witness speed-up for dozens of functions, including for example (list really not exhaustive): Binarize, Dilation, Erosion, ImageAdjust, ImageDeconvolve, MeanShiftFilter, MedianFilter, MorphologicalBranchPoints, MorphologicalGraph, Pruning, Radon, TotalVariationFilter, WienerFilter, etc.

Answer 3

Some matrix operations seem to use more than one core. For instance,

Eigensystem[RandomReal[{-1, 1}, {2000, 2000}]]

uses two cores on my machine. This also works for sparse matrices:

ClearAll[tmparray];
tmparray = SparseArray[
   {
    {i_, j_} /; Abs[i - j] \[LessEqual] 2 \[Rule] RandomReal[]
    },
   {100000, 100000}
   ];

Eigenvalues[
 tmparray,
 -40
 ]

Probably this is done by the libraries that Mathematica calls.