# Shadowing of symbols with ParallelEvaluate and packages

I'm new to Mathematica's parallel evaluation and can't really figure out how to properly use packages on parallel kernels: I get ::shdw messages complaining about shadowing of symbols in Global and package contexts.

I have a test package in FileNameJoin[{$UserBaseDirectory, "Applications", "Test.m"}] that looks like this: BeginPackage["Test"]; f::usage = "Test function."; Begin["Private"]; f[x_] := x*2; End[]; EndPackage[]; Then I start a local kernel and load the package there: k = First@LaunchKernels[1]; ParallelEvaluate[Get["Test"], k] In the kernel, the package is now on the context path: ParallelEvaluate[$ContextPath, k]

(* {"Test", "CloudPublishMenu", "SystemTools", "ExternalEvaluateLoader", "PacletManager", "System", "Global"} *)

When I call the function f like so, it works fine:

ParallelEvaluate[Symbol["f"][3], k]

(* 6 *)

But when I call the function like this it doesn't evaluate and I get an error message:

ParallelEvaluate[f[3], k, DistributedContexts -> None]

Globalf::shdw :  Symbol f appears in multiple contexts {Global,Test}; definitions in context Global may shadow or be shadowed by other definitions.
(* f[3] *)

Having the DistributedContexts option there doesn't make a difference (I've set it to None to prevent potential shadowing by Global definitions of f in the master kernel).

Am I doing anything fundamentally wrong? Not entirely sure about how symbol lookup works.

## How to fix this

In short, either load the package on the master kernel as well, or use

ParallelEvaluate[Testf[3], k]

## Why does this happen

Let's look at what's going on:

k = First@LaunchKernels[1];
ParallelEvaluate[Get["Test"], k]

This launched parallel kernels and loads the Test package on the first parallel kernel. At this point, $ContextPath contains "Test" on that kernel, but not on the master kernel. ParallelEvaluate[f[3], k] At this point it's important to realize that when Mathematica encounters a symbol that's not in any of the contexts on $$ContextPath$$, a new symbol with that name is created in$Context (unless an explicit context is specified). So let's take a look at the example at hand:

• ParallelEvaluate: Exists in System
• k: Exists in Global
• f: Does not exist (remember, on the master kernel, Test is not on $ContextPath). Therefore, it is created in$Context, i.e. Globalf

This means that the expression really is

SystemParallelEvaluate[Globalf[3], Globalk]

Now, the expression is sent to the parallel kernel k, which then complains that Globalf is shadowing Testf.

On the other hand, with

ParallelEvaluate[Symbol["f"][3], k]

the symbol is only created once on the parallel kernel. And there f already exists, because Test is in $ContextPath, so everything works as expected. • Thanks, that explains everything! Too bad, I wanted to use this for evaluating code on a remote kernel with packages that aren't necessarily installed on my machine. Using fully qualified symbols looks tedious. Will try to keep packages in sync between local and remote. – wigg0t Nov 14 '18 at 19:35 • Just out of curiosity: If I evaluate ParallelEvaluate[Unevaluated@ToString[f, InputForm], k] on a fresh parallel kernel, I get "f". I would have expected "Globalf". Is the Global context stripped by ToString? – wigg0t Nov 20 '18 at 8:43 • @wigg0t ToString only prepends the context to a symbol's name if that context is not in$ContextPath or $Context (because as long as it is, f will refer to the correct symbol). You can see this by comparing ToString@f with Block[{$ContextPath = {}, $Context = "Test"}, ToString@f] (the second example temporarily clears both$Context and \$ContextPath, so ToString has to prepend Global in order for the result to refer to the correct symbol) – Lukas Lang Nov 21 '18 at 11:30

As explained by Lukas, it seems one has to either use fully qualified names or load the package on both master and parallel kernels. I find the first option really inconvenient, and in my use case the parallel kernel is running on a remote machine and I don't want to sync the remote packages with my local computer.

What I have tried now (and it seems to work so far) is converting expressions to strings, sending them over to the parallel kernel and then evaluating them with ToExpression.

ClearAll[packExpression];
SetAttributes[packExpression, HoldFirst]
packExpression[expr_] :=
With[
{str = ToString[Unevaluated@expr, InputForm]},
Hold[ToExpression@str]
];

ClearAll[evaluate];
SetAttributes[evaluate, HoldFirst];
evaluate[expr_, k_] :=
With[
{packed = packExpression@expr},
ParallelEvaluate[Unevaluated@ReleaseHold@packed, k]
];

k = First@LaunchKernels[1];
evaluate[Get["Test`"], k];
evaluate[f[3], k]

(* 6 *)