ParallelSum vs. Sum: New functions also defined in other Kernels?

Question

Set-Up: While I had tested a few toy examples that ParallelSum produces the result faster than the Sum, here I encounter one example in my code that the ParallelSum produces either wrong/incorrect result or a warning message. My code below has some new function definitions. (In reality, my complete code has more complicated functions and mappings, but I try to boil down to something simpler to save your energy for your review.)

A_ ⊕ B_ := 
  Mod[{A[[1]] + B[[1]] + A[[2]] B[[3]], A[[2]] + B[[2]], 
    A[[3]] + B[[3]]}, 2];

Conv[m_] := IntegerDigits[Mod[m, 8], 2, 3];
SumD[n_, o_] := (fF[Conv[n]⊕Conv[o]])
Do[fF[IntegerDigits[j, 2, 3]] = j, {j, 0, 7}]

Mapp[0] = {0, 0}; 
Mapp[1] = {0, 1}; 
Mapp[2] = {1, 0}; 
Mapp[3] = {1, 1};

While the Sum

Sum[Mapp[SumD[a, b]][[2]], {a, 0, 1}, {b, 0, 1}]

works and it produces the correct result 2,

the ParallelSum

ParallelSum[Mapp[SumD[a, b]][[2]], {a, 0, 1}, {b, 0, 1}]

does not produce the correct result. It actually produces wrong result if I make a list of a large number of summations.

It says the

Question: I am guessing the following --- Some of the functions defined by me in the earlier part of my code, they are ONLY defined in the Kernel 1, but not defined in other Kernels? Am I correct? How do we modify it to make the ParallelSum work well? [especially I need the ParallelSum, for a list of a large number of summations not shown in my shortened code above.]

Thanks in advance!

Answer 1

The problem is that the definition of CirclePlus is not distributed to parallel kernels. You can test this using

DistributeDefinitions[CirclePlus]
(* {} *)

ParallelEvaluate[Print@Definition[CirclePlus]];

(kernel 1) Null

(kernel 2) Null

(kernel 3) Null

(kernel 4) Null

This is because CirclePlus is in the System` context:

Context[CirclePlus]
(* "System`" *)

There is built-in protection against distributing definitions from System` (and a number of other contexts), as this could in some cases cause severe problems. This protection cannot be disabled (at least not without risky deep-reaching changes to the system, see Parallel`Parallel`Private`$ourExcludedContexts).

I can think of two possible workarounds, none perfect:

• Do not use CirclePlus. Use another symbol that is not in System`

• Create the definitions on subkernels manually:

  ParallelEvaluate[
   A_\[CirclePlus]B_ := Mod[{A[[1]] + B[[1]] + A[[2]] B[[3]], A[[2]] + B[[2]], A[[3]] + B[[3]]}, 2]
  ]
  

  The problem with this approach is that the definitions must be explicitly re-created every time a new subkernel is started. In some cases, new subkernels are started fully automatically in mid-computation (when it is detected that a subkernel has crashed)

  This situation could be improved using this undocumented method of setting initialization code for subkernels.

Answer 2

The glitch seems to be with assigning something to Mathematica's CirclePlus which doesn't seem to be auto-shared, as opposed to introducing your own operator. If instead you did:

op[A_, B_] := 
  Mod[{A[[1]] + B[[1]] + A[[2]] B[[3]], A[[2]] + B[[2]], 
    A[[3]] + B[[3]]}, 2];

Conv[m_] := IntegerDigits[Mod[m, 8], 2, 3];
SumD[n_, o_] := (fF[op[Conv[n], Conv[o]]])
Do[fF[IntegerDigits[j, 2, 3]] = j, {j, 0, 7}]

Mapp[0] = {0, 0};
Mapp[1] = {0, 1};
Mapp[2] = {1, 0};
Mapp[3] = {1, 1};

LaunchKernels[2]

{"KernelObject"[1, "local"], "KernelObject"[2, "local"]}

ParallelSum[Mapp[SumD[a, b]][[2]], {a, 0, 1}, {b, 0, 1}]

2

Sum[(Mapp@SumD[a, b])[[2]], {a, 0, 1}, {b, 0, 1}]

2

There is no problem.

---

Update: Emphasizing Szabolcs' important comment, and acknowledging thatSetSharedFunction basically defeats the point of parallelizing if it's key to what you're doing.

Upon prompting by Szalbocs, now withdrawing my nod to SetSharedFunction[CirclePlus] if you insist on defining System functions. This was the only way I could get remote kernels to play with a defined CirclePlus without explicit remote evaluations, but evidently it drags us back to basically serial operations, so really can't be recommended even if you really want to be using CirclePlus instead of your own operator.

From the documentation:

"A shared function is inefficient for mere code distribution and leads to sequential evaluation."

Also c.f. another of Szalbocs' answers here for additional discussion.