Why is ParallelMap way slower than Map

Question

I recently tried to use ParallelMap instead of Map and to my surprise encountered that ParallelMap seems to be slower in general than Map, which does not make sense to me.

Here is a simple test case, that shows the behavior on my system (tested it on Linux 64 Bit QuadCore i7 and on MacOS DualCore Core2Duo, both running Mathematica 9.0.1):

LaunchKernels[];
f[x_] := Sin[x] + Cos[x] + Tan[x];
test = Table[i, {i, 100000}];

Timing results are the following (and they are consistent, additional kernels have been started before):

Map[f[#] &, test]; // AbsoluteTiming

{0.198230, Null}

ParallelMap[f[#] &, test]; // AbsoluteTiming

{0.650516, Null}

What am I missing here?

Answer 1

I think this may be a duplicate of: How to avoid unpacking from Language`ExtendedFullDefinition

In Mathematica parallelism is only useful when processing takes longer than data transfer, otherwise the overhead of that transfer will make the parallel operation slower than the plain one. It should be somewhat faster than your original use of ParallelMap, but still not as fast as the plain usage to add Method -> "CoarsestGrained":

"CoarsestGrained": break the computation into as many pieces as there are available kernels

ParallelMap[f, test, Method -> "CoarsestGrained"]

Note that you do not need to embed f in a Function (f[#] &) to use it.

Answer 2

I think that ParallelMap has bad implementation of the data distribution between kernels. However if computation of f takes a long time there is some speedup (tested on Core2Duo)

LaunchKernels[];

f[x_] := Nest[Sin, x, 1000];
test = N@Range[100000];

Map[f, test]; // AbsoluteTiming
ParallelMap[f, test]; // AbsoluteTiming

{5.813099, Null}

{3.848709, Null}

Or you can distribute the data manually

f[x_] := Sin[x];
test = Transpose@Partition[N@Range[1000000], $ProcessorCount];

Map[f, Flatten@test]; // AbsoluteTiming

{1.207334, Null}

DistributeDefinitions[f, test];
ParallelEvaluate[data[$KernelID] = test[[$KernelID]]];
ParallelEvaluate[Map[f, data[$KernelID]]]; // AbsoluteTiming

{1.124331, Null}

Speedup is small but it exist.

May be these examples are not the best but they show that behavior of Parallel stuff in Mathematica isn't clear.