# Why is ParallelMap way slower than Map

I recently tried to use ParallelMap instead of Map and to my surprise encountered that ParallelMap seems to be slower in general than Map, which does not make sense to me.

Here is a simple test case, that shows the behavior on my system (tested it on Linux 64 Bit QuadCore i7 and on MacOS DualCore Core2Duo, both running Mathematica 9.0.1):

LaunchKernels[];
f[x_] := Sin[x] + Cos[x] + Tan[x];
test = Table[i, {i, 100000}];


Timing results are the following (and they are consistent, additional kernels have been started before):

Map[f[#] &, test]; // AbsoluteTiming


{0.198230, Null}

ParallelMap[f[#] &, test]; // AbsoluteTiming


{0.650516, Null}

What am I missing here?

-
Just to drive the point home: ParallelXYZ works best for slow functions. Often you can save time by faster functional implementations (examples galore around here), consider e.g.: Sin[#] + Cos[#] + Tan[#] &[Range[100000]] –  Yves Klett Oct 7 '13 at 16:31
ParallelMap is a bit faster when test is replaced by test = N @ Range[1000000] and gets faster as the size grows. Approximate real versus exact results makes a 10x difference in the amount of data transferred back to the main kernel. That is surely a factor in the timings. –  Michael E2 Oct 8 '13 at 2:19

I think this may be a duplicate of: How to avoid unpacking from LanguageExtendedFullDefinition

In Mathematica parallelism is only useful when processing takes longer than data transfer, otherwise the overhead of that transfer will make the parallel operation slower than the plain one. It should be somewhat faster than your original use of ParallelMap, but still not as fast as the plain usage to add Method -> "CoarsestGrained":

"CoarsestGrained": break the computation into as many pieces as there are available kernels

ParallelMap[f, test, Method -> "CoarsestGrained"]


Note that you do not need to embed f in a Function (f[#] &) to use it.

-
Now, are we talking to ourselves (not to worry, I do it all the time, don´t I)? –  Yves Klett Oct 7 '13 at 16:21
@Yves The White Council is secretly assembling... BTW, there is another candidate for master: Why is parallel slower? –  István Zachar Oct 7 '13 at 16:33
@IstvánZachar who will get to play Radagast? Now there´s a whacky fellow (although the movie took a lot of poetic license). And yes, quite similar questions... –  Yves Klett Oct 7 '13 at 16:35
@Yves Not an easy part, lot's of qualities to fulfill. He must be a simple fool who is also a bird-tamer. We should look around at Gardening & Landscaping. –  István Zachar Oct 7 '13 at 16:46
@IstvánZachar but he has the know of the mushrooms :-) –  Yves Klett Oct 7 '13 at 16:49

I think that ParallelMap has bad implementation of the data distribution between kernels. However if computation of f takes a long time there is some speedup (tested on Core2Duo)

LaunchKernels[];

f[x_] := Nest[Sin, x, 1000];
test = N@Range[100000];

Map[f, test]; // AbsoluteTiming
ParallelMap[f, test]; // AbsoluteTiming


{5.813099, Null}

{3.848709, Null}

Or you can distribute the data manually

f[x_] := Sin[x];
test = Transpose@Partition[N@Range[1000000], $ProcessorCount]; Map[f, Flatten@test]; // AbsoluteTiming  {1.207334, Null} DistributeDefinitions[f, test]; ParallelEvaluate[data[$KernelID] = test[[$KernelID]]]; ParallelEvaluate[Map[f, data[$KernelID]]]; // AbsoluteTiming


{1.124331, Null}

Speedup is small but it exist.

May be these examples are not the best but they show that behavior of Parallel` stuff in Mathematica isn't clear.

-