12
$\begingroup$

Bug introduced in 11.3.0 and persisting through 12.1.1

(CASE:4039452)


I believe I am seeing a new sort of behavior around some parallel functionality in Mathematica 11.3, because code that used to function properly in earlier versions no longer works. There's a bit of setup required, which is probably more convoluted than it strictly needs to be since it requires building some packages.

packageA =
  ToString[
   Unevaluated[
    BeginPackage["A`"];
    A`A = 17;
    EndPackage[];
   ],
   InputForm];
dir = CreateDirectory[];
WriteString[FileNameJoin@{dir, "A.wl"}, packageA];

Let's launch subkernels and make sure each one has the right $Path:

LaunchKernels[];
PrependTo[$Path, dir];
With[{path = $Path},
  ParallelEvaluate[$Path = path]];

Now, let's set a value for A`A on our local kernel:

A`A = 18;
(* 18 *)

And let's load the A` in each subkernel:

ParallelNeeds["A`"];
ParallelEvaluate[A`A]
(* {17, 17, 17, 17} *)

OK, they would never have a reason to see the new value of A`A. Let's give them one:

DistributeDefinitions["A`"]
(* {"A`"} *)

But nothing seems to have happened!

ParallelEvaluate[A`A]
(* {17, 17, 17, 17} *)

I'd expect the output to be {18, 18, 18, 18}. What am I misunderstanding? And am I mistaken that the output would have been {18, 18, 18, 18} in prior versions?

$\endgroup$
1
  • 5
    $\begingroup$ I get all 18s in V11.2, all 17s in V11.3. $\endgroup$
    – Michael E2
    Commented Mar 29, 2018 at 19:15

2 Answers 2

8
$\begingroup$

I think this is likely to be a bug.

Historically, DistributeDefinitions has always ignored all symbols in System` and various other system package contexts.

The handling of the excluded contexts has been reorganized in 11.3, in particular there now is

?Parallel`Protected`$ExcludedContexts

$ExcludedContexts is the list of contexts to always exclude from DistributeDefinitions.

which is generated on the fly and unfortunately it looks like our package context ends up on the blacklist for some reason:

MemberQ[Parallel`Protected`$ExcludedContexts, "A`"]

(* True *)

The following works as expected

Block[{Parallel`Protected`$ExcludedContexts = 
    DeleteCases[Parallel`Protected`$ExcludedContexts, "A`"]}, 
      DistributeDefinitions["A`"]
];
ParallelEvaluate[A`A]

(* {18, 18, 18, 18} *)
$\endgroup$
2
  • 1
    $\begingroup$ Interesting. Is the reason for the addition of Parallel`Protected`$ExcludedContexts to save memory? $\endgroup$
    – QuantumDot
    Commented Apr 2, 2018 at 0:44
  • $\begingroup$ I think it's just to have more control and flexibility. Previous versions also had a similar blacklist for system contexts, but it was static (set up at kernel initialization). $\endgroup$
    – ilian
    Commented Apr 2, 2018 at 20:36
4
$\begingroup$

Another straightforward workaround is to just use ParallelEvaluate @* Needs in the place of ParallelNeeds, like so:

ParallelNeeds["A`"];
ParallelEvaluate[A`A]

(* {17, 17, 17, 17} *)

A`A = 18
(* 18 *)

DistributeDefinitions["A`"];
ParallelEvaluate[A`A]
(* {18, 18, 18, 18} *)
$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.