# How to parallelize the most outer loop only?

I have two Doloops. The outer loop iterate over the inner loop. The inner loop contains a set of step by step instructions which each next step depends on the previous one so the inner loop must not be parallelized. However, the parallelization of the outer loop is desirable. How can I just parallelize the outer loop only? Does the code below is parallelized in both loops? If the code below is just parallelized on the outer loop only, then the other problem is that without using SetSharedVariable[a, b] I can't get the values of a and b and if I use it it runs much more slowly than the unparallelized version.

SetSharedVariable[a, b]
AbsoluteTiming[
ParallelDo[
{
a = Table[i + j, {i, 1.0, 100.0}, {j, 1.0, 100.0}];
Do[
{
b = a.a;
c = If[Tr[b] > 2000, b = a, Break[]];
}
, {iterator1, 1, 500}];
}, {iterator2, 1, 500}]]


## Edit

Consider the code below :

a = Table[Null, {i, 1, 1000}];
b = Table[Null, {i, 1, 10}];
SetSharedVariable[a, b]
ParallelDo[Do[a[[iterator1]] = RandomReal[], {iterator1, 1, 1000}];
b[[iterator2]] = a;, {iterator2, 1, 10}]


Now, a and b are outside of the loops and they are different for each iteration of outer loop. Indeed, b gets bigger and bigger with each outer loop iteration. Without SetSharedVariable after execution the code, a and b aren't accessible and if I use it, it is incredibly more slower than usual. How can I have a and b without losing speed?

• As an aside I think you are confusing languages here. In Mathematica {stuff} is a list, the {} do not denote blocks of code. Dec 17, 2014 at 14:53
• How can I denote a block of code then? Should I use Block, Module, or With?
– MOON
Dec 17, 2014 at 16:00
• You already have. ; is not a delimiter it is the infix form of CompoundExpression so a;b;c is CompoundExpression[a,b,c]. If you want to isolate local values then use Module (or rarely and if you're brave Block). Dec 17, 2014 at 16:42

I just cleaned up your code a little, added a Print command and lowered the maximum value of the iterators:

ParallelDo[
a = Table[i + j, {i, 1.0, 100.0}, {j, 1.0, 100.0}];
Do[b = a.a;
Print["Kernel ", $KernelID, ", iterator1 ", iterator1, ", iterator2 ", iterator2]; c = If[Tr[b] > 2000, b = a, Break[]], {iterator1, 1, 2} ], {iterator2, 1, 8}]  Kernel 4, iterator1 1, iterator2 1 Kernel 3, iterator1 1, iterator2 3 Kernel 2, iterator1 1, iterator2 5 Kernel 1, iterator1 1, iterator2 7 Kernel 4, iterator1 2, iterator2 1 Kernel 3, iterator1 2, iterator2 3 Kernel 2, iterator1 2, iterator2 5 Kernel 1, iterator1 2, iterator2 7 Kernel 4, iterator1 1, iterator2 2 Kernel 3, iterator1 1, iterator2 4 Kernel 2, iterator1 1, iterator2 6 Kernel 1, iterator1 1, iterator2 8 Kernel 4, iterator1 2, iterator2 2 Kernel 3, iterator1 2, iterator2 4 Kernel 2, iterator1 2, iterator2 6 Kernel 1, iterator1 2, iterator2 8  From this output you can see that the inner loop is not parallelized. You can add other values (like a and b) to Print to explore your code further. Using SetSharedVariable[a, b] is not useful for this code example and is only needed, if the values of these variables have to be synchronized among all parallel kernels all the time. It should not be used if each kernel should, e.g., run the inner loop with a different value for a and b. If the value for a is not different for each kernel (as it is the case in this example), than it's value should be calculated only once outside of ParallelDo. Also note the fact, that no global value for, e.g. a is set after executing the code. # Edit In the example you added in your edit you encounter a situation that is related to my last note. The problem is not, that a and b are unknown to the kernels. The problem is, that the b[[iterator2]] the values are assigned to are localized to the kernels. You can just share b: SetSharedVariable[b] ParallelDo[Do[a[[iterator1]] = RandomReal[], {iterator1, 1, 1000}]; b[[iterator2]] = a;, {iterator2, 1, 10}]  This gives a ~27-fold speedup on my PC compare to using SetSharedVariable[a, b]. Or you can use b = ParallelTable[Do[a[[iterator1]] = RandomReal[], {iterator1, 1, 1000}]; a, {iterator2, 1, 10}];  without using SetSharedVariable. Or you can implement the code with some other function, that is more suitable than ParallelDo for the task at hand. But you have to be aware of the fact, that there is a difference in what the code actually does if you use SetSharedVariable[b] instead of SetSharedVariable[a, b]! Compare the output of SetSharedVariable[a, b] ParallelDo[Do[a[[iterator1]] =$KernelID, {iterator1, 1, 1000}];
b[[iterator2]] = a;, {iterator2, 1, 10}]
b


with the output of

UnsetShared[a, b]
SetSharedVariable[b]
ParallelDo[Do[a[[iterator1]] = \$KernelID, {iterator1, 1, 1000}];
b[[iterator2]] = a;, {iterator2, 1, 10}]

• Thank you. What if a is different for each iteration of outer loop? I edited the question to reflect this matter.
– MOON
Dec 17, 2014 at 23:05
• This example is intended to show my problem. Actually I have to loops one of them 30000 and the other 500 to 1000, and this is the least number of iterations.
– MOON
Dec 18, 2014 at 9:42