# Filling global arrays in parallel calculations

I want to parallelize heavy computations which has to be done many times, the result should be arrays of data. Each calculation does not depend on others (but uses the same variables though). The question is how to assign the results to the given arrays. Things like this doesn't work:

num = 10; f = Range[num];
Parallelize[
For[i = 1, i <= 10, i++, f[[i]] = 1 + i;], DistributedContexts -> Full]


gets

Parallelize::nopar1: "For[i=1,i<=10,i++,f[[i]]=1+i;] cannot be parallelized; proceeding with sequential evaluation."

Using ParallelTable gives desired table:

num = 10; f = Range[num];
b[n_] := Module[{}, f[[n]] = n + 1]
ParallelTable[b[k], {k, num}, DistributedContexts -> Full]


but somehow it doesn't change f. If I needed one array, that could be an option, but there are many of them. And it is not very handy to write program as a calculation of a many-dimensional array. So the question is that is the correct/standard way to assign values to arrays in such situation?

-

Please see the answer I gave here.

Parallelization works well and reliable only if you use constructs with no side effects. This is an essential point when using Mathematica's parallel computing tools. If you are not sure what a side effect is, please read about it.

This explains why For won't be auto-parallelized.

The reason why the function b does not seem to change f is that f will be changed only on the parallel kernels, but the results won't be sent back to the main kernel. It is not even clear which f should be sent back. The one on kernel 1 or the one on kernel 2? (Obviously you want them merged in a special way, but there is no way for the computer to guess how.) I discuss this issue as well here.

You can use SetSharedVariable[f] to force all accessed to f to be synchronized through the main kernel. This will give you the desired result, but it will hurt performance because of the constant communication between the main kernel and subkernels. It is also a potential source of errors because two different subkernels might change the same value in f.

I recommend you never change any "global" variables from parallel calculations. This is always a potential source of mistakes and when done properly, it will reduce performance. Instead formulate your problem without using side effects. Your specific example would look like ParallelTable[k+1, {k, num}]. Think about if you can do this with your actual problem as well.

-

You can get f changed by using SetSharedFunction[b]:

num = 10; f = Range[num];
b[n_] := Module[{}, f[[n]] = n + 1]
SetSharedFunction[b]
ParallelTable[b[k], {k, num}, DistributedContexts -> Full]
(*
==> {2, 3, 4, 5, 6, 7, 8, 9, 10, 11}
*)
f
(*
==> {2, 3, 4, 5, 6, 7, 8, 9, 10, 11}
*)


However I think that will effectively suppress parallel evaluation of b. So you should only do that dor the actual assignment, not the complete expensive calculation:

num = 10; f = Range[num];
assign[n_,v_] := (f[[n]] = v)
SetSharedFunction[assign]
b[n_] := Module[{}, assign[f[[n]],n + 1]]
ParallelTable[b[k], {k, num}, DistributedContexts -> Full]


That way only the assignment, not the slow calculation (represented by n+1 here) will be shared.

Edit:

As Szabolcs noted in a comment, a more efficient solution is to use SetSharedVariable:

num=10;f=Range[num];
SetSharedVariable[f]
b[n_]:=Module[{},(f[[n]]=n+1)]
ParallelTable[b[k],{k,num},DistributedContexts->Full]


It also has the advantage that you don't need a helper function just for the assignment.

-
Use SetSharedVariable, not SetSharedFunction. This'll still slow things down a bit. – Szabolcs Mar 8 '12 at 13:58
@Szabolcs: Thanks, I've added it to my post. – celtschk Mar 9 '12 at 18:15