Pass symbols through MathLink with full context information

Question

How can I ensure that symbols passed through MathLink retain their full context information? When using MathLink naively, they do not.

Demonstration, controlling another kernel:

link = LinkLaunch["\"" <> First[$CommandLine] <> "\" -mathlink"];
LinkRead[link]; (* read initial prompt *)

foo`x; (* create symbol *)

AppendTo[$ContextPath, "foo`"]; (* add foo` to context path so now x refers to foo`x *)

Context[x] (* verify the context of x *)
(* "foo`" *)

LinkWrite[link, Unevaluated@EvaluatePacket[Context[foo`x]]] (* send foo`x to the other kernel and check its context there *)
LinkReadHeld[link]
(* Hold[ReturnPacket["Global`"]] *)

The other kernel thinks that x has context Global`. It must have been sent without any context information. I want it to retain its context, foo`, while being sent over.

It's the same situation when sending symbols back through the link: they do not retain context information.

---

I do have a sort of a workaround: temporarily unset $ContextPath on both sides of the link.

Block[{$ContextPath},
  LinkWrite[link, Unevaluated@EvaluatePacket[Block[{$ContextPath}, (* code goes here*)]]
]

But this has obvious undesirable effects. For example, what if the code that is being sent has ToExpression? Parsing is influenced by $ContextPath. What if the code sets $ContextPath? Many things will break in obscure ways. This can be made more complicated to fix some of the problems, but I am hoping for a better way.

If I do the same experiment with the parallel tools (ParallelEvaluate[..., DistributedContexts -> None]), I see that context information is in fact preserved. So there must be a way to accomplish this.

Answer 1

The best I could come up with so far is the following code. It sends some code through link to another kernel for evaluation, then it retrieves the result.

linkEval[link_][HoldComplete[code_]] :=
    Module[{result},
      Block[{$ContextPath = {}, $Context = "myPrivateContext`"},
        LinkWrite[link,
          Unevaluated@EvaluatePacket[
            $result = code;
            Block[{$ContextPath = {}, $Context = "myPrivateContext`"}, 
              OwnValues[$result]
            ]
          ]
        ]
      ];
      While[True,
        result = LinkRead[link];
        Switch[result,
          _ReturnPacket, Break[],
          _TextPacket, Print@First[result],
          _, Null (*Print[result]*)
        ]
      ];
      Replace[result, ReturnPacket[{_ :> expr_}] :> HoldComplete[expr]]
    ]

The Block[{$ContextPath={}, $Context = "myPrivateContext`"}, ...] is a hack to force sending fully qualified symbol names. I want this even for System` symbols because the link may connect kernels of different versions, so we set it to {} instead of allowing the default {"System`", "Global`"}.

We don't want to evaluate code with a special $ContextPath, so we evaluate it normally and assign the result to $result.

To prevent a second evaluation of the contents of $result, we send back its OwnValues. This is again done in such a way so that symbol names are sent with the context prepended.

This seems to work in most situations, e.g. evaluating Sequence[] returns as Sequence[]. Evaluating Unevaluated[] also return as Sequence[] because several symbols we use (e.g. RuleDelayed in OwnValues) are only HoldAll and HoldSequence but not HoldAllComplete. But the parallel tools don't do any better either.