12
$\begingroup$

Bug introduced in 11.3.0

(CASE:4039452)


I believe I am seeing a new sort of behavior around some parallel functionality in Mathematica 11.3, because code that used to function properly in earlier versions no longer works. There's a bit of setup required, which is probably more convoluted than it strictly needs to be since it requires building some packages.

packageA =
  ToString[
   Unevaluated[
    BeginPackage["A`"];
    A`A = 17;
    EndPackage[];
   ],
   InputForm];
dir = CreateDirectory[];
WriteString[FileNameJoin@{dir, "A.wl"}, packageA];

Let's launch subkernels and make sure each one has the right $Path:

LaunchKernels[];
PrependTo[$Path, dir];
With[{path = $Path},
  ParallelEvaluate[$Path = path]];

Now, let's set a value for A`A on our local kernel:

A`A = 18;
(* 18 *)

And let's load the A` in each subkernel:

ParallelNeeds["A`"];
ParallelEvaluate[A`A]
(* {17, 17, 17, 17} *)

OK, they would never have a reason to see the new value of A`A. Let's give them one:

DistributeDefinitions["A`"]
(* {"A`"} *)

But nothing seems to have happened!

ParallelEvaluate[A`A]
(* {17, 17, 17, 17} *)

I'd expect the output to be {18, 18, 18, 18}. What am I misunderstanding? And am I mistaken that the output would have been {18, 18, 18, 18} in prior versions?

$\endgroup$
  • 5
    $\begingroup$ I get all 18s in V11.2, all 17s in V11.3. $\endgroup$ – Michael E2 Mar 29 '18 at 19:15
8
$\begingroup$

I think this is likely to be a bug.

Historically, DistributeDefinitions has always ignored all symbols in System` and various other system package contexts.

The handling of the excluded contexts has been reorganized in 11.3, in particular there now is

?Parallel`Protected`$ExcludedContexts

$ExcludedContexts is the list of contexts to always exclude from DistributeDefinitions.

which is generated on the fly and unfortunately it looks like our package context ends up on the blacklist for some reason:

MemberQ[Parallel`Protected`$ExcludedContexts, "A`"]

(* True *)

The following works as expected

Block[{Parallel`Protected`$ExcludedContexts = 
    DeleteCases[Parallel`Protected`$ExcludedContexts, "A`"]}, 
      DistributeDefinitions["A`"]
];
ParallelEvaluate[A`A]

(* {18, 18, 18, 18} *)
$\endgroup$
  • 1
    $\begingroup$ Interesting. Is the reason for the addition of Parallel`Protected`$ExcludedContexts to save memory? $\endgroup$ – QuantumDot Apr 2 '18 at 0:44
  • $\begingroup$ I think it's just to have more control and flexibility. Previous versions also had a similar blacklist for system contexts, but it was static (set up at kernel initialization). $\endgroup$ – ilian Apr 2 '18 at 20:36
3
$\begingroup$

Another straightforward workaround is to just use ParallelEvaluate @* Needs in the place of ParallelNeeds, like so:

ParallelNeeds["A`"];
ParallelEvaluate[A`A]

(* {17, 17, 17, 17} *)

A`A = 18
(* 18 *)

DistributeDefinitions["A`"];
ParallelEvaluate[A`A]
(* {18, 18, 18, 18} *)
$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.