# Control Front End in “mathlink” mode

Note: This question was already posted in the Wolfram community.

In a terminal math session, we can start a front end (FE) easily by launching it though LinkLaunch or by using functions provided in the Developer context (thanks to Oleksandr for telling me about this). Here is a very basic example that starts a FE and opens a new notebook for you:

<<Developer;
lnk = DeveloperInstallFrontEnd["Server"->False];


Unfortunately, the started FE starts the default kernel which is usually a good idea, but here I want to be in control of everything. This means, I would like to provide the Kernel for the FE so that it can connect to it and use it for calculation. So what I really want to have is a Mathematica MathLink program (or JLink) that starts a kernel and an FE, connects both so that the front end uses the kernel as evaluator but all communication goes through my program.

If you are familiar with LinkSnooper, then you see the difference because there, the FE starts the LinkSnooper as special kernel. What I want (not only for snooping, but for a more general problem) is similar but more problematic, because I want to control the kernel starting behavior of the FE from a link and not as user.

Question: Is there any direct way to do it? For instance by giving options to DeveloperInstallFrontEnd and using specific FrontEndToken.

Appendix: One way that I haven't tested and which might work is modify the init.m of the FE (usually located it $UserBaseDirectory/FrontEnd/init.m), because there are the settings for the kernel: EvaluatorNames->{ "Local" -> {"AutoStartOnLaunch" -> True}, "LinkSnooper" -> {...} }  So I could theoretically alter the settings there (hopefully on the fly during startup of the FE) and force the FE to connect to my specific kernel. • By "provide the kernel for the FE" do you mean attach it to an unnamed kernel that you had started? I'm a bit hazy on which kernel you need to attach it to – b3m2a1 Aug 23 '18 at 17:51 • I'll give you further information after a bit of sleep. Too tired at the moment. – halirutan Aug 23 '18 at 18:35 • This suggests that having a single kernel that is both connected to the FE and accessible in its own right isn't possible (see the thing on the MathLink mode protocol). Of course peer-to-peer is always possible, but I think the point of this is to have two external programs access the same kernel. – b3m2a1 Aug 23 '18 at 22:20 • I take that back. It seems like it might be possible, using a programmatically constructed LinkSnooper-like thing. I'm currently hitting deadlocks, but I think I can work them out. – b3m2a1 Aug 24 '18 at 0:44 ## 1 Answer ## With Custom REPL & Access to Kernel From Outside Process ### Non-Blocking Version Update There were issues with the front-end somehow (I really have no idea how) accessing the kernel outside of the preemptive link/service link or the standard packet channel (I assume) which caused things like usage-template lookup to hang the entire system. To that effect I introduced a non-blocking version of this which frees us from having to manually control the kernel responses to the front-end, but also doesn't allow us to control/snoop on the front-end / kernel communication. On the other hand we can still get STDIN control, so we can do things like define a notebook in the notebook interface and get it out of the process. Here's a picture of that in action: This is now the default as it's much more robust than the blocking version. Until I figure out how to make the FE not try to access the kernel via a channel I don't control this will have to do. ### Code I added support for accessing the kernel state from a process outside Mathematica as well, using the OpenRead["!cat"] hack to get at stdin so I could also process that in my REPL. It's still not super robust, but, again, it's a start. In particular I think if I can get all the packet handling to work as it should stop hanging. That'll take work, though, unless someone who has used LinkSnooper in the past has a record of what the response should be for every type of packet passed through. I also changed the package name so that it's clear that it works with things other than IntelliJ (although via the $SubprocessKernelName variable you can give the kernel whatever name you want).

BeginPackage["SubprocessKernel"]

$SubprocessFrontEnd::usage="The front end link for the kernel";$SubprocessKernel::usage="The kernel link for the kernel";
$SubprocessKernelName::usage="The basic kernel name for the subprocess";$SubprocessKernelExtension::usage="The extension for the kernel name";
OpenSubprocessNotebook::usage="Opens an a subprocess notebook";
StartSubprocessREPL::usage="Starts the subprocess REPL";
$SubprocessREPLSettings::usage="Settings for the subprocess REPL"; Begin["Private"] startSubprocessFrontEnd[]:= If[Quiet[$FrontEnd[[1]]=!=$SubprocessFrontEnd||!BooleanQ@LinkReadyQ[$SubprocessFrontEnd]],
Quiet@LinkClose[$SubprocessFrontEnd]; UsingFrontEnd; launchFECommand= SystemUseFrontEndDumpCommand["Server"->False]<> Map[{" ",#1}&, Join[SystemUseFrontEndDumpFlags["Server"->False],{"-wstp"}] ];$SubprocessFrontEnd=LinkLaunch[launchFECommand];

MathLinkSetFrontEnd[$SubprocessFrontEnd] ]; If[!StringQ@$SubprocessKernelName,
$SubprocessKernelName="Subprocess"]; If[!StringQ@$SubprocessKernelExtension,
$SubprocessKernelExtension="+"<>IntegerString[RandomInteger[50000], 16, 4]]; startSubprocessKernel[]:= If[!MatchQ[$SubprocessKernel, _LinkObject?(Quiet[BooleanQ@LinkReadyQ[#]]&)],
$subprocessKernelName=$SubprocessKernelName<>$SubprocessKernelExtension;$SubprocessKernel=Quiet@LinkCreate[$subprocessKernelName, "LinkMode"->Listen],$SubprocessKernel
];

configureSubprocessFrontEnd[]:=
(
$cachedKernelConfiguration= CurrentValue[$FrontEndSession, {EvaluatorNames, $SubprocessKernelName}]; CurrentValue[$FrontEndSession, {EvaluatorNames, $SubprocessKernelName}]= { "AutoStartOnLaunch"->False, "MLOpenArguments"-> TemplateApply[ "-LinkMode Connect -LinkProtocol SharedMemory -LinkName \"\" -LinkOptions 256",$subprocessKernelName
]
};
)

$SubprocessREPLSettings= <| "InitializationMessage"->"Starting kernel...", "ProcessPacket"->processPacket, "ProcessInput"->Function[{text}, processPacket@HoldComplete[EnterTextPacket[text]]], "LinkWrite"-> Function[{link, response, packet}, (*Print@{packet, response};*) LinkWrite[link, #]&/@response ], "STDOUTWrite"-> Function[ {output, input}, Print[">> ------ Output:  ------ >>"~TemplateApply~$Line];
Print[ToString[#, InputForm]<>"\n"]&/@output;
Print["<< ------ Output:  ------ <<"~TemplateApply~$Line] ], "RouteOutput"-> <| "STDIN"->"STDOUT", "FrontEnd"->"Link" |>, "LogPackets"->False, "PollTime"->.1, "Blocking"->False |>; evaluateThings[e_]:= Reap[ GeneralUtilitiesWithMessageHandler[ CheckAbort[ReleaseHold@e,$Aborted],
With[
{
sym=Extract[#, {2, "MessageTemplate", 1}, Hold],
name=Extract[#, {2, "MessageTemplate", 2}],
params=Sequence@@Extract[#, {2, "MessageParameters"}]
},
Sow[MessagePacket[sym, name]/.Hold[s_]:>s];
Sow[TextPacket@
Replace[
InternalMessageButtonHandler[
MessageName[Evaluate@ReleaseHold[sym], name],
Hold[MessageName[sym, name]]/.Hold[s_Symbol]:>s,
Hold[Message[MessageName[sym, name], params]]/.Hold[s_Symbol]:>s
],
$Failed:> ToString[HoldForm[MessageName[sym, name]]/.Hold[s_Symbol]:>s] <>" "<> ToString@ StringForm[ Replace[ Extract[#, {2, "MessageTemplate"}], Except[_String]:>MessageName[General, name] ], params ] ] ] ]& ] ] standardEvaluationProcedure[expr_, makePacket_]:= Module[{evRes=evaluateThings[expr], e, errs},$incrementLine=True;
e=evRes[[1]];
errs=evRes[[2]];
{
errs,
Switch[e,
Null,
{},
_ErrorBox,
{
OutputNamePacket["Out[]="~TemplateApply~$Line], makePacket@e }, _, { OutputNamePacket["Out[]="~TemplateApply~$Line],
makePacket@e
}
]
}
]

logEvent[packet_]:=
With[
{
log=FileNameJoin@{$TemporaryDirectory, "subproc_repl_log.txt"} }, If[!FileExistsQ@log, CreateFile@log]; OpenAppend[log]; WriteString[log, StringRepeat["\n", 2]<> StringRepeat[" ", 5]<> StringRepeat["-", 25]<> StringRepeat["\n", 2] ]; PutAppend[packet, log]; Quiet@Close@log; ]; logEcho[event_]:= (logEvent[event];event) processPacket//Clear processPacket[p:HoldComplete@EvaluatePacket[res_]]:= If[TrueQ@$SubprocessREPLSettings["LogPackets"], logEcho, Identity][
If[TrueQ@$SubprocessREPLSettings["LogPackets"], logEvent[p]]; ReturnPacket[res] ]; processPacket[p:HoldComplete@EnterExpressionPacket[expr_]]:= If[TrueQ@$SubprocessREPLSettings["LogPackets"], logEcho, Identity][
If[TrueQ@$SubprocessREPLSettings["LogPackets"], logEvent[p]]; If[TrueQ@$SubprocessREPLSettings["LogPackets"], logEcho, Identity]@
standardEvaluationProcedure[
expr,
]
];
processPacket[p:HoldComplete@EnterTextPacket[expr_]]:=
If[TrueQ@$SubprocessREPLSettings["LogPackets"], logEcho, Identity][ If[TrueQ@$SubprocessREPLSettings["LogPackets"], logEvent[p]];
standardEvaluationProcedure[
MakeExpression[expr, StandardForm],
{},
ReturnTextPacket[ToString@#]
]&
]
];
processPacket[p:HoldComplete@InputPacket[expr_]]:=
If[TrueQ@$SubprocessREPLSettings["LogPackets"], logEcho, Identity][ If[TrueQ@$SubprocessREPLSettings["LogPackets"], logEvent[p]];
$incrementLine=True; ReturnTextPacket[expr] ]; processPacket[p:HoldComplete[packet_]]:= If[TrueQ@$SubprocessREPLSettings["LogPackets"], logEcho, Identity][
logEvent[p];
ReturnExpressionPacket[BoxData@ToBoxes@packet]
]

exitSubprocessREPL[]:=
(
$ParentLink=Null; Quiet[LinkClose@$SubprocessKernel];
Quiet[LinkClose@$SubprocessFrontEnd]; ) packetWrite[response_, packet_, src_, link_, linkWrite_, stdoutWrite_]:= If[response=!=None, If[src==="STDOUT", Block[{$ParentLink=Null},
stdoutWrite[
Flatten[{response}],
packet
]
];,
TimeConstrained[
Flatten[{response}],
packet
],
.1
]
]
]

subprocessBlockingREPL[]:=
Module[
{

kernelLink=$SubprocessKernel, packet, response, stdin=OpenRead["!cat", BinaryFormat->True], input, output, preemptiveLink=MathLink$PreemptiveLink,
ppacket,
pevalStack=<||>,
pevalNum=1,

linkWrite:=$SubprocessREPLSettings["LinkWrite"], processPacket:=$SubprocessREPLSettings["ProcessPacket"],
processInput:=$SubprocessREPLSettings["ProcessInput"], stdoutWrite:=$SubprocessREPLSettings["STDOUTWrite"],
stdinOutput:=$SubprocessREPLSettings["RouteOutput", "STDIN"], feOutput:=$SubprocessREPLSettings["RouteOutput", "FrontEnd"],
pollTime:=$SubprocessREPLSettings["PollTime"] }, (*logEvent@"Entering loop";*) While[True (*Quiet@TrueQ[LinkConnectedQ@$SubprocessFrontEnd]&&
Quiet@TrueQ[LinkConnectedQ@$SubprocessKernel]*), output=None; response=None; input=ReadString[stdin, EndOfBuffer]; (*logEvent["Link Ready: "<>ToString@LinkReadyQ@kernelLink];*) If[Quiet@LinkReadyQ@preemptiveLink, ppacket=TimeConstrained[LinkRead[preemptiveLink, HoldComplete], .5]; With[{n=pevalNum}, pevalStack[n]=ppacket; If[$VersionNumber<11.2,
Function[Null, RunScheduledTask[#, {.0001}], HoldFirst],
SessionSubmit
][
pevalStack[n]={processPacket@pevalStack[n], pevalStack[n]};
While[KeyExistsQ[pevalStack, n-1],
Pause[.0001];
];
packetWrite[
];
pevalStack[n]=.
]
]
];

If[StringQ@input&&StringLength@input>0,
output=processInput@input;
];
response=processPacket[packet];
];

If[
response===ReturnPacket[Quit]||
output===ReturnPacket[Quit],
exitSubprocessREPL[];
Break[]
];

packetWrite[
output, input, stdinOutput, kernelLink,
];

packetWrite[
response, packet, feOutput, kernelLink,
];

If[TrueQ@$incrementLine,$Line++];
$incrementLine=False; If[Head@response=!=ReturnPacket&&wroteToLink, TimeConstrained[ LinkWrite[$SubprocessKernel,
InputNamePacket["In[]:="~TemplateApply~$Line] ], .1 ] ]; If[pollTime>0, Pause[pollTime] ] ] ]; subprocessNonBlockingREPL[]:= Module[ { stdin=OpenRead["!cat", BinaryFormat->True], input, output, wroteToLink, linkWrite:=$SubprocessREPLSettings["LinkWrite"],
processPacket:=$SubprocessREPLSettings["ProcessPacket"], processInput:=$SubprocessREPLSettings["ProcessInput"],
stdoutWrite:=$SubprocessREPLSettings["STDOUTWrite"], stdinOutput:=$SubprocessREPLSettings["RouteOutput", "STDIN"],
feOutput:=$SubprocessREPLSettings["RouteOutput", "FrontEnd"], pollTime:=$SubprocessREPLSettings["PollTime"]
},
If[$VersionNumber<11.2, RunScheduledTask, Function[Null, SessionSubmit[ScheduledTask[##]], HoldFirst] ][ input=ReadString[stdin, EndOfBuffer]; output=None; If[StringQ@input&&StringLength@input>0, output=$SubprocessREPLSettings["ProcessInput"]@input;
];

If[output===ReturnPacket[Quit],
exitSubprocessREPL[];
If[$VersionNumber<11.2, RemoveScheduledTask@$ScheduledTask,
TaskRemove@$CurrentTask ] ]; packetWrite[ output, input, stdinOutput,$SubprocessKernel,
];

If[TrueQ@$incrementLine,$Line++];
TimeConstrained[
$SubprocessKernel, InputNamePacket["In[]:="~TemplateApply~$Line]
],
.1
]
];
$incrementLine=False;, pollTime ] ] StartSubprocessREPL[]:= With[ { blocking=$SubprocessREPLSettings["Blocking"]=!=False
},
$Line=1; startSubprocessFrontEnd[]; startSubprocessKernel[]; FrontEndExecute@FrontEndEvaluatorStart[$SubprocessKernelName];
If[blocking,
TimeConstrained[
LinkWrite[$SubprocessKernel, InputNamePacket["In[1]:="]], .1 ] ]; If[StringQ@$SubprocessREPLSettings["InitializationMessage"],
Print@$SubprocessREPLSettings["InitializationMessage"] ];$ParentLink=$SubprocessKernel; If[blocking, subprocessBlockingREPL[];, logEvent["REPL Started"]; subprocessNonBlockingREPL[] ]; ] OpenSubprocessNotebook[retry:True|False:True]:= Module[{res}, startSubprocessFrontEnd[]; startSubprocessKernel[]; res=CreateDocument[{}, Evaluator->$SubprocessKernelName];
Which[
res===$Failed&&retry, Quiet@LinkClose@$SubprocessKernel;
$SubprocessKernel=.; Quiet@LinkClose@$SubprocessFrontEnd;
$SubprocessFrontEnd=.; OpenSubprocessNotebook[], res===$Failed,
$Failed, True, configureSubprocessFrontEnd[]; StartSubprocessREPL[]; res ] ] End[] EndPackage[]  This opens the notebook and we can almost use it like normal: but there are a few key differences. For one, most packets that I don't explicitly handle cause it to hang indefinitely and the number of those is large. For another, we have access to exactly how the boxes are parsed in the REPL. Finally, hopefully someday we'll also be able to inject code from an external process into the kernel that this notebook is bound to. Once I figure out how to make Input work only for the stdin in the command prompt we should be golden. ## No Custom Layer (Robust) Here's a partial solution that's worth getting out here with notes for how it needs to be improved. I just made a little package that attaches a front-end to a custom kernel. I assumed this was for the IntelliJ plugin so I named it that: BeginPackage["IntelliJ"]$IntelliJFrontEnd::usage="The front end link for IntelliJ";
$IntelliJKernel::usage="The kernel link for IntelliJ"; OpenIntelliJNotebook::usage="Opens an IntelliJ Notebook"; StartIntelliJREPL::usage="Starts the IntelliJ REPL";$IntelliJREPLSettings::usage="Settings for the IntelliJ REPL";

Begin["Private"]

startIntelliJFrontEnd[]:=
If[Quiet[$FrontEnd[[1]]=!=$IntelliJFrontEnd||!BooleanQ@LinkReadyQ[$IntelliJFrontEnd]], Quiet@LinkClose[$IntelliJFrontEnd];
UsingFrontEnd;
launchFECommand=
SystemUseFrontEndDumpCommand["Server"->False]<>
Map[{" ",#1}&,
Join[SystemUseFrontEndDumpFlags["Server"->False],{"-wstp"}]
];

$IntelliJFrontEnd=LinkLaunch[launchFECommand]; MathLinkSetFrontEnd[$IntelliJFrontEnd]
];

startIntelliJKernel[]:=
If[!MatchQ[$IntelliJKernel, _LinkObject?(Quiet[BooleanQ@LinkReadyQ[#]]&)],$IntelliJKernel=Quiet@LinkCreate["IntelliJKernel", "LinkMode"->Listen],
$IntelliJKernel ]; configureIntelliJFrontEnd[]:= ( CurrentValue[$FrontEnd, {EvaluatorNames, "IntelliJ"}]=
{
"AutoStartOnLaunch"->False,
"MLOpenArguments"->
};
)

$IntelliJREPLSettings= <| "PacketHandler"-> Function[{packet, kernel, fe}, Print@packet; LinkWrite[fe, packet] ], "PollTime"->.1 |>; StartIntelliJREPL[]:= Module[ { replin, replout, startedEvaluator }, startIntelliJFrontEnd[]; startIntelliJKernel[];(* If[!TrueQ@startedEvaluator, FrontEndExecute@FrontEndEvaluatorStart["IntelliJ"]; LinkWrite[$IntelliJKernel, InputNamePacket["IntelliJ init"]];
];*)
$ParentLink=$IntelliJKernel;
(*While[Quiet@TrueQ[LinkConnectedQ@$IntelliJFrontEnd], If[LinkReadyQ[$IntelliJKernel, $IntelliJREPLSettings["PollTime"]], replin= Reap[ While[LinkReadyQ[$IntelliJKernel], Sow@Quiet@LinkRead[$IntelliJKernel]] ][[2]]; replout= With[{evf=$IntelliJREPLSettings["PacketHandler"]},
If[#===$Failed, Break[], evf[ #,$IntelliJKernel,
$IntelliJFrontEnd ] ] ]&/@replin ]; Pause[$IntelliJREPLSettings["PollTime"]]
]*)
]

OpenIntelliJNotebook[retry:True|False:True]:=
Module[{res},
startIntelliJFrontEnd[];
startIntelliJKernel[];
res=CreateDocument[{}, Evaluator->"IntelliJ"];
Which[
res===$Failed&&retry, Quiet@LinkClose@$IntelliJKernel;
$IntelliJKernel=.; Quiet@LinkClose@$IntelliJFrontEnd;
$IntelliJFrontEnd=.; OpenIntelliJNotebook[], res===$Failed,
$Failed, True, configureIntelliJFrontEnd[]; StartIntelliJREPL[]; res ] ] End[] EndPackage[]  The way I've been using it is loading it on in a terminal window: Then this pops open a notebook with this kernel as its parent link and all the variables we defined in it accessible: The trick was really to play with the EvaluatorNames via CurrentValue after creating the front-end. That way we can make a peer-to-peer connection and use that as the link the front-end starts itself with. Without that there was no way I could find in the MathLink API or the FrontEnd packet system to attach a LinkObject to a NotebookObject. After that all I had to do was set the $ParentLink and it worked immediately.

Unfortunately this is only a partial solution as we've completely lost access to our original kernel from outside. We could possibly still access is by making another connection to the "IntelliJKernel" LinkObject but I'm not certain of this.

I also wanted to have a custom REPL working underneath where we could change the packet handler and also watch for stuff via Input (you can see I've got a bunch of commented out code there). For reasons I don't totally understand, though, this wasn't working. My inspiration for that was page 55 of this paper by David Wagner. He basically previewed a LinkSnooper like this there that I wanted to reimplement using a single kernel.

So this is a start, but there's work to be done.

An interesting observation is that we can actually switch back and forth between terminal mode and notebook mode with this by setting $ParentLink back to Null or $IntelliJKernel:
We can of course see that the value of aaa I set in the notebook is kinda weird in the terminal, but it's an interesting, potentially very useful, thing to be able to do.
It also suggests that we could have multiple notebooks communicating with a single kernel in turn on different machines if the LinkObject` were made to have a port and host parameter (as it can). Still, though, not quite what we want.