How do I share my variables and my function output in parallel computing?

Question

Here as a simple example

nnn = 81; 
x = Subscript[A, i]; y = Subscript[B, i];
Do[Subscript[A, i] = 5 i + 2, {i, 1, nnn}]; 
Do[Subscript[B, i] = 3 i , {i, 1, nnn}];

function = y - x + Sqrt[(2 y - x)^2]
ParallelDo[Subscript[F, i, j] = function, {i, 1, nnn, 1}, {j, 1, nnn, 1}]

I just want the share my input variable and my output in all my CPU processors I already use DistributeDefinitions and SetSharedVariable, SetSharedFunction. It works with ParallelTable, but with ParallelDo it's not working, especially when I use symbols with index fi as shared variable input or a shared functions output. I have no problem with the variable without indices. The problem occurs when I use symbols fi with an index. When I use [F, i, j] in anther computation, it does not work.

Answer 1

First of all, drop the Subscript notation -- it is only asking for trouble. Parallelization tasks are finicky enough without trying to confuse the kernel with layers of nested variables. So let us instead define all quantities like Subscript[A,x,y] as A[x,y] etc.

nnn = 81;
x[i_] = A[i]; y[i_] = B[i];
Do[A[i] = 5 i + 2, {i, 1, nnn}];
Do[B[i] = 3 i, {i, 1, nnn}];
function1[i_] = y[i] - x[i] + Sqrt[(2 y[i] - x[i])^2];
function2[i_, j_] = y[j] - x[i] + Sqrt[(2 y[j] - x[i])^2];

Notice that I added an argument to x and y, since it actually makes a difference when you define your function variable. I also defined function1 and function2 to demonstrate how having different i,j labels affects your definitions. In one case it is independent of j, in the other both i and j show up.

Next, since you want to use this in parallel computing, let us evaluate these lines on each parallel kernel as well, to make these definitions available locally on each kernel (this makes parallel calculations much faster than trying to share them from the global context):

ParallelEvaluate[
  nnn = 81;
  x[i_] = A[i]; y[i_] = B[i];
  Do[A[i] = 5 i + 2, {i, 1, nnn}];
  Do[B[i] = 3 i, {i, 1, nnn}];
  function1[i_] = y[i] - x[i] + Sqrt[(2 y[i] - x[i])^2];
  function2[i_, j_] = y[j] - x[i] + Sqrt[(2 y[j] - x[i])^2];
  ];

To illustrate how to use the ParallelDo to write to a shared variable, let us share the function name F with all kernels

SetSharedFunction[F]

it has to be SetSharedFunction instead of SetSharedVariable because we intend to write and save values such as F[1,2] etc. which take function arguments. Finally, your parallel do command can be either one of:

ParallelDo[F[i, j] = function1[i], {i, 1, nnn, 1}, {j, 1, nnn, 1}]

or

ParallelDo[F[i, j] = function2[i,j], {i, 1, nnn, 1}, {j, 1, nnn, 1}]

depending on your actual intended definition of function. In the above, F[i,j] values will be calculated in parallel on the sub-kernels and communicated back to the main kernel since F is a shared function. However, you will notice that this simple example actually takes more time to calculate in parallel than without parallelization. The reason is that each F[i,j] has to first be distributed in chunks to the parallel kernels, then collected and stored back into the global context on individual basis. Having a bigger calculation might increase this trouble.

Perhaps it might be more efficient to store the data in global context by avoiding the use of any shared variables and requesting the output to be delivered back to main kernel in the form of a list, e.g.

FF = ParallelTable[ function1[i], {i, 1, nnn, 1}, {j, 1, nnn, 1}];

or

FF = ParallelTable[ function2[i,j], {i, 1, nnn, 1}, {j, 1, nnn, 1}];

then you can access the desired F[i,j] elements as FF[[i,j]] instead, and the calculation returns much faster.