# ParallelSum vs. Sum: New functions also defined in other Kernels?

Set-Up: While I had tested a few toy examples that ParallelSum produces the result faster than the Sum, here I encounter one example in my code that the ParallelSum produces either wrong/incorrect result or a warning message. My code below has some new function definitions. (In reality, my complete code has more complicated functions and mappings, but I try to boil down to something simpler to save your energy for your review.)

A_ \[CirclePlus] B_ :=
Mod[{A[[1]] + B[[1]] + A[[2]] B[[3]], A[[2]] + B[[2]],
A[[3]] + B[[3]]}, 2];

Conv[m_] := IntegerDigits[Mod[m, 8], 2, 3];
SumD[n_, o_] := (fF[Conv[n]\[CirclePlus]Conv[o]])
Do[fF[IntegerDigits[j, 2, 3]] = j, {j, 0, 7}]

Mapp[0] = {0, 0};
Mapp[1] = {0, 1};
Mapp[2] = {1, 0};
Mapp[3] = {1, 1};


While the Sum

Sum[Mapp[SumD[a, b]][[2]], {a, 0, 1}, {b, 0, 1}]


works and it produces the correct result 2,

the ParallelSum

ParallelSum[Mapp[SumD[a, b]][[2]], {a, 0, 1}, {b, 0, 1}]


does not produce the correct result. It actually produces wrong result if I make a list of a large number of summations.

It says the

Question: I am guessing the following --- Some of the functions defined by me in the earlier part of my code, they are ONLY defined in the Kernel 1, but not defined in other Kernels? Am I correct? How do we modify it to make the ParallelSum work well? [especially I need the ParallelSum, for a list of a large number of summations not shown in my shortened code above.]

• I see that how ParallelSum can speed up the Sum as my examples here: – wonderich Sep 6 '17 at 23:54
• Timing[Sum[ If[MatchQ[Mod[i + j, 8], 0 | 2 | 4] && MatchQ[Mod[j + k, 8], 0 | 2 | 4] && MatchQ[Mod[i + k, 8], 0 | 2 | 4] && (i^2 + j^2 > k^2) && (j^2 + k^2 > i^2) && (i^2 + k^2 > j^2), f[i, j, k], 0], {i, 0, 2^10}, {j, 0, 2^10}, {k, 1, 2^2 + 1}]]; – wonderich Sep 6 '17 at 23:54
• Timing[ParallelSum[ If[MatchQ[Mod[i + j, 8], 0 | 2 | 4] && MatchQ[Mod[j + k, 8], 0 | 2 | 4] && MatchQ[Mod[i + k, 8], 0 | 2 | 4] && (i^2 + j^2 > k^2) && (j^2 + k^2 > i^2) && (i^2 + k^2 > j^2), f[i, j, k], 0], {i, 0, 2^10}, {j, 0, 2^10}, {k, 1, 2^2 + 1}]]; – wonderich Sep 6 '17 at 23:54

The problem is that the definition of CirclePlus is not distributed to parallel kernels. You can test this using

DistributeDefinitions[CirclePlus]
(* {} *)

ParallelEvaluate[Print@Definition[CirclePlus]];

(kernel 1) Null

(kernel 2) Null

(kernel 3) Null

(kernel 4) Null


This is because CirclePlus is in the System context:

Context[CirclePlus]
(* "System" *)


There is built-in protection against distributing definitions from System (and a number of other contexts), as this could in some cases cause severe problems. This protection cannot be disabled (at least not without risky deep-reaching changes to the system, see ParallelParallelPrivate\$ourExcludedContexts).

I can think of two possible workarounds, none perfect:

• Do not use CirclePlus. Use another symbol that is not in System

• Create the definitions on subkernels manually:

ParallelEvaluate[
A_\[CirclePlus]B_ := Mod[{A[[1]] + B[[1]] + A[[2]] B[[3]], A[[2]] + B[[2]], A[[3]] + B[[3]]}, 2]
]


The problem with this approach is that the definitions must be explicitly re-created every time a new subkernel is started. In some cases, new subkernels are started fully automatically in mid-computation (when it is detected that a subkernel has crashed)

This situation could be improved using this undocumented method of setting initialization code for subkernels.

• Thanks for the answer I appreciate it. +1. – wonderich Sep 6 '17 at 14:29

The glitch seems to be with assigning something to Mathematica's CirclePlus which doesn't seem to be auto-shared, as opposed to introducing your own operator. If instead you did:

op[A_, B_] :=
Mod[{A[[1]] + B[[1]] + A[[2]] B[[3]], A[[2]] + B[[2]],
A[[3]] + B[[3]]}, 2];

Conv[m_] := IntegerDigits[Mod[m, 8], 2, 3];
SumD[n_, o_] := (fF[op[Conv[n], Conv[o]]])
Do[fF[IntegerDigits[j, 2, 3]] = j, {j, 0, 7}]

Mapp[0] = {0, 0};
Mapp[1] = {0, 1};
Mapp[2] = {1, 0};
Mapp[3] = {1, 1};

LaunchKernels[2]


{"KernelObject"[1, "local"], "KernelObject"[2, "local"]}

ParallelSum[Mapp[SumD[a, b]][[2]], {a, 0, 1}, {b, 0, 1}]


2

Sum[(Mapp@SumD[a, b])[[2]], {a, 0, 1}, {b, 0, 1}]


2

There is no problem.

Update: Emphasizing Szabolcs' important comment, and acknowledging thatSetSharedFunction basically defeats the point of parallelizing if it's key to what you're doing.

Upon prompting by Szalbocs, now withdrawing my nod to SetSharedFunction[CirclePlus] if you insist on defining System functions. This was the only way I could get remote kernels to play with a defined CirclePlus without explicit remote evaluations, but evidently it drags us back to basically serial operations, so really can't be recommended even if you really want to be using CirclePlus instead of your own operator.

From the documentation:

"A shared function is inefficient for mere code distribution and leads to sequential evaluation."

• SetSharedFunction is not meant for this purpose and will often cause a severe slowdown. Definition sharing is done using DistributeDefinitions, but in this case that won't work because it explicitly excludes System  context symbols. I am not sure how to work around this without risking automatically distributing System `-context definitions that really should not be distributed ... – Szabolcs Sep 6 '17 at 13:06