I am writing a package (for reference, this one) which I use extensively in parallel computations, and to lighten the coding load on the 'run-time' notebook, I want my package to sit on the $DistributedContexts
list of sessions in which it is called.
The way I'm doing it now is to simply call
$DistributedContexts := {$Context, "Test`"}
at the end of the package, and this mostly works. However, it has a definite disadvantage in that if the user has already put something in that variable, calling Needs["Test`"]
will overwrite that previous value, without even telling the user, and that's not completely a good thing. (It's not terrible, either, but I'd rather it didn't happen.)
So far, what I've done is include the following message,
$DistributedContexts::overwrite = "Warning: overwriting previous value of
$DistributedContexts. Reinstate your old definition, and include the
Test context to ensure proper parallelization of Test calculations.";
If[
ValueQ[$DistributedContexts],
Message[$DistributedContexts::overwrite]
]
which produces a warning when Needs["Test`"]
is called and the variable already had something in there. This draws, already, from pretty tricky territory, as $DistributedContexts
is not an easy variable to handle. For instance, on a clean kernel
Quit
ValueQ[$DistributedContexts]
will return False
, and indeed calling
$DistributedContexts
will return unevaluated, even though it clearly has a "value" in the sense that it is the default value of the DistributedContexts
option of Parallelize
and (at least some) related functions, and it is by default equal to $Context
.
What I want, to be more specific, is a way to append the context name "Test`"
on to the value of $DistributedContexts
if it already has one. So, for example, I want
$DistributedContexts:={$Context, "OtherContext`"}
Needs["Test`"]
to be equivalent to
$DistributedContexts:={$Context, "OtherContext`", "Test`"}
This turns out to be very tricky!
Calling
AppendTo[$DistributedContexts, "Test`"]
doesn't work, and returns the error messageAppendTo::rvalue: $DistributedContexts is not a variable with a value, so its value cannot be changed.
Calling
$DistributedContexts := {$DistributedContexts, "Test`"}
seems to work - it just swallows it up without complaint - but then calling, say,ParallelTable[Pause[0.1], {j, 100}]
, returns the error message$RecursionLimit::reclim2: Recursion depth of 1024 exceeded during evaluation of {$DistributedContexts,Test`}.
because, of course, the initial call was a
SetDelayed
which did not evaluate, but the parallelized evaluation tried to ask for the value of$DistributedContexts
which is of course recursively ill-defined.If
$DistributedContexts
already has a value, and particularly if it's been defined sensibly as something like$DistributedContexts:={$Context, "OtherContext`"}
, it's very hard to actually get that value, because if you do that and then try to ask for the value, you will get different answers depending on where you are on the context tree,Begin["Context1`"]; $DistributedContexts End[]; Begin["Context2`"]; $DistributedContexts End[]; (* {"Context1`", "OtherContext`"} {"Context2`", "OtherContext`"} *)
and it certainly won't return a plain
$Context
willingly on its own.
So: is there a way to produce the behaviour I want?
$DistributedContexts
, but why you are trying to do this? i.e. Is your intention to allow a package functionMyPackFunc1
(that may be operating in parallel) to call other package functions sayMyPackFunc2
, so that the other kernels are aware of the package functions? Or do you just want to update variablesSetSharedVariable
between the different kernels? $\endgroup$Needs["Test`"]
and have functions such asTest`f
available for use insideParallelTable
without any further calling of$DistributedContexts
orDistributeDefinitions
. $\endgroup$DistributeDefinitions["Test`"]
, though I'm not sure I like the relation of that with launching extra kernels in an environment that may or may not end up using parallelized environments. More generally, though, this is surely something that's doable via the$DistributedContexts
route, and I'm mostly interested in how you'd circumvent the difficulties I outlined. $\endgroup$