# ParallelMap and Sow/Reap - not behaving as expected? [duplicate]

Is ParallelMap and Parallelize@Map expected to work with Sow and Reap construct.

This is a rather brief toy code to find all Primes just to force the use of Sow/Reap inside Map

Reap[Map[
If[PrimeQ@#, Sow@#] &,
Range[10]
]][[2]]


and the output is, as expected

{{2, 3, 5, 7}}


But when I do

Reap[Parallelize@Map[
If[PrimeQ@#, Sow@#] &,
Range[1000]
]][[2]]


I got the output

{}


And the same was the output when I ran

Reap[
ParallelMap[
If[PrimeQ@#, Sow@#] &,
Range[10]
]
][[2]]


2 questions :

1) How is this explainable ?

2) If this is the case, how do I efficiently I spawn parallel jobs which accumulate something ?

-

## marked as duplicate by Szabolcs, Leonid Shifrin, Artes, Michael E2, R. M.♦Oct 4 '13 at 1:05

I guess this will work! Check documentation for SetSharedFunction but be careful of the parallel overhead of following implementation.

SetSharedFunction[PSow];
PSow[x_] := Sow[x];
Reap[Parallelize@Map[If[PrimeQ@#, PSow@#] &, Range[10]]][[2]]


{{2, 5, 3, 7}}

-

I would recommend parallelizing these types of calculations using the ParallelCombine paradigm.

First, let's make a function out of your example:

fun = Function[list, Reap[Map[If[PrimeQ@#, Sow@#] &, list]][[2, 1]]]


The non-parallel version will be fun[Range[200]]. Parallelize it using

ParallelCombine[
fun,
Range[200],
Join
]


I spelt out Join here because in other applications you may need a different function there. Here it can be omitted.

This is clearly more work than the (also valid) SetSharedFunction approach, but it is going to avoid some overhead. SetSharedFunction simply forces the function to be evaluated on the main kernel. If the equivalent of PrimeQ for your real calculation gives True for a sizeable fraction of the values, the SetSharedFunction-based parallelization won't buy much performance, or might even be slower than a serial computation.

-
Thanks for sueggesting ParallelCombine. An interesting find! – my account_ram Oct 3 '13 at 21:53
But how is this behavior of Reap@Map@Sow giving {} explainable? – my account_ram Oct 3 '13 at 21:54
@myaccount_ram You were running Sow in a subkernel and Reap in the main kernel, so it can't work. These are separate processes that do not share any memory. The main kernel merely submits evaluations to the subkernels and collects results. – Szabolcs Oct 3 '13 at 21:59