Parallelize two independent loops

tab = Table[Random[], {10^7}];
Block[{sum = 0, sum2 = 0}, Do[sum += tab[[i]], {i, Length[tab]}];
Do[sum2 += tab[[i]]^2, {i, Length[tab]}];
{sum, sum2}]


In the above code, the two do loops are independent of each other. I want to parallel execute the two loops. I tried

tab = Table[Random[], {10^7}];
Block[{sum = 0, sum2 = 0}, Parallelize[Do[sum += tab[[i]], {i, Length[tab]}];
Do[sum2 += tab[[i]]^2, {i, Length[tab]}];];
{sum, sum2}]


but it doesn't work. How can I parallel execute these two loops in different Kernels? The two functions of summation can be different too. For ex, if I want to $$\exp(\text{tab}[[i]])$$.

P.S.: I know there is a better way to sum a list of random numbers. I summed the list using loops to test the speed.

Here is one solution. It uses $KernelID which, if you one has two kernels, will typically have values 1 on one kernel, 2 on the other: (* setup *) SeedRandom[1]; tab=Table[Random[],{10^7}]; (* parallel *) With[{tab=tab}, ParallelEvaluate[ Switch[$KernelID,
1, sum=0;Do[sum+=t,{t,tab}];sum,
2, sum=0;Do[sum+=t^2,{t,tab}];sum,
_, 0 (* do nothing on other kernels *)
],
DistributedContexts->None]]


Discussion. The data is sent to the kernels using With. The DistributedContexts->None makes sure that nothing else is sent around without you knowing. To make different definitions on different kernels, we use Switch[$KernelID,...]. I use this kind of approach when I want to control precisely what is to be done on which kernel, as in OP's question. Clearly, this means that I the programmer am responsible for load balancing. Some other commands in the Parallel-family provide automatic load balancing, which will be the right choice in some situations, please see the answer by @LukasLang. • A good solution, but if I change the function of the sum, this solution is not valid. I want to know if running two Do loops in two different kernels is possible. Commented Jul 21, 2022 at 16:12 • Yes, this looks good. The only thing is my pc has 4 Kernels. So the Switch function for only two arguments doesn't work. Only if I put 4 arguments, it gives the correct result. Commented Jul 21, 2022 at 16:55 • Dear @sslucifer, I edited my answer again, so that there are no problems with more than 2 kernels. I hope you don't mind, just trying to keep things simple for future viewers. Commented Jul 23, 2022 at 17:28 • instead of not doing anything on the remaining kernels you could start exactly as many kernels as you need. See documentation of LaunchKernels and maybe CloseKernels for how to achieve that... Commented Jul 26, 2022 at 16:18 • Don't use$KernelID. It is neither guaranteed to start at 1, nor be contiguous. Commented Jul 31, 2022 at 17:19

Here's how I would do it using ParallelMap:

First, make your two evaluations truly independent by wrapping Block around them separately instead):

SeedRandom[1];
tab = Table[Random[], {10^7}];
{
Block[{sum = 0}, Do[sum += tab[[i]], {i, Length[tab]}]; sum],
Block[{sum = 0}, Do[sum += tab[[i]]^2, {i, Length[tab]}]; sum]
}
(* {5.00075*10^6, 3.33394*10^6} *)


You can now wrap the evaluations in Hold to prevent them from evaluating on the main kernel (here done using Hold/@Unevaluated@{...}), and then map ReleaseHold over the list using ParallelMap:

SeedRandom[1];
tab = Table[Random[], {10^7}];
ParallelMap[ReleaseHold,
Hold /@ Unevaluated@{
Block[{sum = 0}, Do[sum += tab[[i]], {i, Length[tab]}]; sum],
Block[{sum = 0}, Do[sum += tab[[i]]^2, {i, Length[tab]}]; sum]
}
]
(* {5.00075*10^6, 3.33394*10^6} *)


The advantage of this approach is that it is trivially extendable: It works with any number of available kernels and any number of available expressions. To fine tune how the expressions are distributed, you can use the Method option of ParallelMap.

Side notes

I am assuming your code is a toy example, so I didn't mention it above, but here are some things I would improve about the code:

1. Use Module instead of Block to properly localize the sum variable (notice also that this takes ~12.8s on my PC):

AbsoluteTiming@{
Module[{sum = 0}, Do[sum += tab[[i]], {i, Length[tab]}]; sum],
Module[{sum = 0}, Do[sum += tab[[i]]^2, {i, Length[tab]}]; sum]
}
(* {12.8127, {5.00075*10^6, 3.33394*10^6}} *)

2. Don't use Do if you can avoid it. Here, the loop is easily written using Sum (note how this is also significantly faster, taking only ~8.2s):

AbsoluteTiming@{
Sum[tab[[i]], {i, Length[tab]}],
Sum[tab[[i]]^2, {i, Length[tab]}]
}
(* {8.22215, {5.00075*10^6, 3.33394*10^6}} *)

3. The sum is in fact so trivial in this case that you can use Total (notice the insance speedup, it now only takes ~0.035s, a 350x improvement):

AbsoluteTiming@{
Total[tab],
Total[tab^2]
}
(* {0.0353386, {5.00075*10^6, 3.33394*10^6}} *)


As noted above, I am aware that your actual code is probably more complex. I just thought I'd mention some improvements, in case some of them apply to your actual code.

The answer given by @user293787 is correct. However, one needs only two kernels for the code to work. If one has four kernels like my pc,

With[{tab = tab}, ParallelEvaluate[sum = 0;
Switch[$KernelID, 1, Do[sum += tab[[i]], {i,Length[tab]}], 2, Do[sum += tab[[i]]^2, {i,Length[tab]}], 3, Null, 4, Null]; sum, DistributedContexts -> None]]  Using this solution, I have been able to find an alternate solution. ParallelTable[ Piecewise[ {{Block[{sum = 0}, Do[sum += tab[[i]], {i, Length[tab]}]; sum], j == 1}, {Block[{sum = 0}, Do[sum += tab[[i]]^2, {i, Length[tab]}]; sum], j == 2}}], {j, 1, 2}]  • I am glad you were able to adapt it to your case. Here are two alternatives to what you did. Alternative I: You can use LaunchKernels[2] to launch only two kernels, even if you have more available. Alternative II: You could also use something like Switch[$KernelID,1,...,2,...,_,Null] where _ means all kernels except $1$ and $2$. Commented Jul 22, 2022 at 4:22

The first reason this doesn't work is that the variables sum and sum2 are not synchronized with the parallel subkernels. You can modify the corresponding variables in the subkernels all day long and the result will not be visible in the master kernel.

The second reason is that the ; operator (CompoundExpression) explicitly requires sequential evaluation. So, your first loop is always run before the second one.

Parallelize[] will go inside the CompoundExpression and actually parallelize the first loop, then the second one, but for reason 1 above, this is pointless.

Given that you also need to distribute 10^7 values to all your kernels, you will not gain any meaningful insights from this timing experiment.

To parallelize a sequence of independent evaluations, you can put them into a list, and

Parallelize[{eval1, eval2, ...}]


will send those evaluations to different parallel subkernels and wait for all of them to finish.

So, a working example would first distribute the definition of tab (once), then run the the two loops independently with local copies of the sum variables:

DistributeDefinitions[tab]

In[7]:= Parallelize[{
sum = 0; Do[sum += tab[[i]], {i, Length[tab]}]; sum,
sum = 0; Do[sum += tab[[i]]^2, {i, Length[tab]}]; sum
}]

Out[7]= {5.00124*10^6, 3.33458*10^6}