Connect Frontend to Kernel Started by JVM via JLink

Question

I have a MathKernel started and controlled via JLink from a JVM. The $CommandLine for this kernel is:

$CommandLine
(* {"C:/Program Files/Wolfram Research/Mathematica/10.0/mathkernel.exe", "-mathlink", "-linkprotocol", "", "-linkmode", "connect", "-linkname", "h7r_shm"} *)

I can open a Frontend from this kernel by sending it the following string for evaluation:

Needs["JLink`"];
FrontEndLaunchCommand = mmaFrontEndPath;
ConnectToFrontEnd[]

and this string to actually open a window with a notebook

UseFrontEnd[
  nbObj = CreateWindow[
             DocumentNotebook[
                {TextCell["Launched by JVM Kernel", "Title"]}], 
             Visible -> True ]
             ]

I'd like to be able to set this notebook to use the JVM's kernel for evaluation (rather than its own) so I can use the notebook for easier debugging of the JVM launched kernel.

I've tried creating a link on the JVM's kernel using

feMathLink = LinkCreate["feMathLink"]    

and then attempting to connect to this in the Notebook using

 jvmLink = LinkConnect["feMathLink"]

I get True from LinkReadyQ[jvmLink] but attempts to read from it using LinkRead[jvmLink] just hang.

I've also tried just setting the Notebook's evaluation kernel to "feMathLink" using

SetOptions[EvaluationNotebook[], Evaluator -> "feMathLink"] 

but this also just hangs.

Apologies in advance if this really is in the manual but it is not obvious.

This all comes from a small test program here that seemed too large to post.

Update - Clarification

As suggested by halirutan, I've simplified the question by taking Java/Groovy out of the question and focusing just on the kernel to frontend connection issue.

If you run the following code from a kernel (call it kernel A) in "terminal mode" then it will launch a frontend which is connected to this kernel but has its own kernel (call it kernel B) for evaluation. That is, kernel A can send commands to the frontend via UseFrontEnd[] but this frontend doesn't use kernel A for evaluation.

 Needs["JLink`"]
 (* this is the program the JLink will use to launch a FrontEnd *)
 $FrontEndLaunchCommand = FileNameJoin[{$InstallationDirectory, "Mathematica.exe"}]
 (* this connects the kernel to a FrontEnd launched via JLink *)
 ConnectToFrontEnd[]
 (* at this point, FrontEnd is not visible, it is available only for processing some commands via UseFrontEnd[] *)
 (* this uses UseFrontEnd[] to open a new Notebook in a Window making the FrontEnd visible *)
 UseFrontEnd[
     nbObj = CreateWindow[
            DocumentNotebook[{TextCell["Launched by JVM Kernel", "Title"]}], 
            Visible -> True ]
     ]
(* you should now see an unnamed notebook with first cell as a Title "Launched by JVM Kernel" *)            

I'd like to set the evaluation kernel (currently Kernel B) to be Kernel A, the initializing kernel. The objective is to use the frontend as a debugging tool for a kernel launch via JVM (or terminal in this case).

As noted above, I've tried creating a math link in Kernel A using:

feMathLink = LinkCreate["feMathLink"] 

and setting the frontend's evaluation kernel to this link using:

jvmLink = LinkConnect["feMathLink"]
SetOptions[EvaluationNotebook[], Evaluator -> "feMathLink"] 

I've also tried reading and writing from this link but it always hangs on me.

I've put a windows batch file and a simple .m file with all of this here. This should enable you to replicate the issue on any standard windows setup without much effort, none if you have version 10 installed in the standard place - just run the runIt.bat in a cmd or Powershell terminal.

Many thanks again to halirutan for suggestions to clarify the question. It really isn't a JVM question at all, although UseFrontEnd[] is a JLink function.

Answer 1

I was able to get a satisfactory answer to my question by using a very good article by David Wagner, "MathLink Mode" available here.

I'll leave all the JVM stuff out of the answer but the approach does work just as well for a kernel launched by some external program as it does for the direct launch explained here.

I'll explain it for Windows but there isn't much that is platform specific; it is mostly Wolfram code.

Step 1: Launch a kernel direct from the command line:

%MathematicaInstallationDirectory%\MathKernel.exe

where %MathematicaInstallationDirectory%\ is the $InstallationDirectory.

This will launch a standalone kernel with a terminal interface. This emulates the kernel we want to be able to inspect and work with during debugging. (In my case this emulates the kernel I launch and use from the JVM.)

We'll refer to this kernel as the remote kernel.

As a simple test, set symbol a to be something like this. We will later read this symbol from a Mathematica Frontend.

a = "This was set in the remote kernel"

Step 3: Create a Mathlink in this kernel and set it ready to be connected to by putting it in Listen mode:

(* this is all in the remote kernel terminal interface 8)
linkName = "5000"
remoteLinkObj = LinkOpen[linkName, LinkMode -> Listen]
LinkWrite[remoteLinkObj, "Hello from remote"]

linkName can be any string on Windows but needs to be a port number on Linux or OSX. This will block the remote kernel (at line 2) until we connect to the mathlink named "5000" from the frontend. It will then immediately write back on the link the string "Hello from remote" and block again until the frontend reads this string.

Step 4: Launch Mathematica and open a new notebook and a fresh kernel.

Step 5: Create the following functions to make it easier to work with remote kernel and its link.

(* this is in the frontend notebook (and thus evaluated in the frontend's kernel) *)

drainLink[link_LinkObject] := Reap[While[LinkReadyQ[link], Sow[LinkRead[link]]]]
remoteEval[link_LinkObject, expr_] := (
    LinkWrite[link, Unevaluated[expr]];
    LinkRead[link]
    )
SetAttributes[remoteEval, HoldRest]

drainLink[] is a useful function to use if the link appears frozen. The remote kernel will block until everything it wrote to the link is read off the link by the frontend. drainLink does that for us.

remoteEval[] sends unevaluated expressions to the remote kernel for evaluation and reads the results of the remote evaluation off the link.

Step 6: Connect the frontend to remote kernel via the mathlink named "5000" and read whatever is on the link (which should be the string "Hello from remote".

Enter the following into the frontend's notebook.

(* frontend notebook *)
linkName = "5000"  (* this should match the linkName used in remote kernel *)
(* this connects to linkName *)
feLinkObject = LinkOpen[linkName, LinkMode -> Connect]
(* this will read everything off the link, if anything is there *)
drainLink[feLinkObject]
(* this should return the string "Hello from remote" *)

Step 7: return to the remote kernels terminal interface and set it into a read, evaluation and write loop so it will act as an evaluation server for the frontend. Define the following function on the remote kernel:

(* this is entered into the remote kernel's terminal *)

evalLoop[link_LinkObject] := Module[
  {resultOfEval, exprIn, exprOut, evalMessages},
  (* this collects any messages *)
  evalMessages = {};
  collectMessages[m_] := AppendTo[evalMessages, m];
  Internal`AddHandler["Message", collectMessages];

 (* this is the actual loop until Break[] *)
 While[True,
    (* reset messages *)
    evalMessages = {};
    (* read incoming expr off the link and wrap in Hold[] *)
    exprIn = LinkRead[link, Hold];
    (* release Hold and evaluate expr checking for messages *)
    resultOfEval = Check[exprOut = ReleaseHold[exprIn], $Failed ];
        Which[
           (* messages generated by evaluation so return everything *)
           resultOfEval === $Failed, LinkWrite[link, 
           EvalError["In" -> exprIn, "Out" -> exprOut, "Messages" -> evalMessages]],
       (* no messages, assume OK and just return result of evaluation *)
       True, LinkWrite[link, exprOut]
       ]
    ];

(* loop ended by sending Break[] so clean up *)
(* turn off message handler *)
Internal`RemoveHandler["Message", collectMessages];
(* write back message that evaluation loop is off *)
LinkWrite[link, "Evaluation loop is off"]
]

and run it with

evalLoop[remoteLinkObj]

this blocks the remote kernel in the While loop above until it gets a Break[] from the frontend.

It will read, eval and write until stopped by Break[].

step 8: Switch back to frontend notebook and test remote kernel evaluation.

remoteEval[feLinkObject, a]
(* should return the string "This was set in the remote kernel" *)

This sends the symbol a unevaluated back to the remote kernal where the evalLoop[] reads it, evaluates a (to the string "This was set in the remote kernel") and returns this result on the link where it is read off by the frontend in remoteEval.

In this way remoteEval and evalLoop can be used to inspect and change the remote kernel's state in an interactive environment to debug code and data sent to a kernel by some other program (in my case a JVM app).

This approach is a direct lift from the Wagner article referenced above.

There is probably a better way to connect the frontend to the remote kernel using the $ParentLink of the remote kernel and creating a new kernel connection under the Evaluation menu of the frontend (see page 55 of Wagner) but I couldn't get it to work and as I had all I needed for now I let it go.

I wrapped this into a small package that I can load and use as required without too much work on the keyboard to manually connect and disconnect.

You can debug JLink apps by adding link listeners and the like but the frontend is such a powerful interactive tool I wanted to figure out a way to use it instead.

Answer 2

This question and answer from pjc42 promises to be helpful for my own question re .NET.

Going through the question it was surprisingly difficult to replicate the OP's code:

UseFrontEnd[
    nbObj = CreateWindow[
           DocumentNotebook[{TextCell["Launched by JVM Kernel", "Title"]}], 
           Visible -> True
    ]

(halirutan has also commented that he couldn't get it to work.)

In .NET the notebook would not appear until a SelectionEvaluate was performed.

MathKernel kernel = new MathKernel();
kernel.Compute("Needs[\"JLink`\"]");
kernel.Compute("$FrontEndLaunchCommand = FileNameJoin[{$InstallationDirectory, \"Mathematica.exe\"}]");
kernel.Compute("UseFrontEnd[nb = CreateDocument[Null, Visible -> False]; NotebookWrite[nb, Cell[BoxData[RowBox[{\"CreateDocument\", \"[\", RowBox[{RowBox[{\"{\", RowBox[{\"TextCell\", \"[\", RowBox[{\"\\\"Launched by JVM Kernel\\\"\", \",\", \"\\\"Title\\\"\"}], \"]\"}], \"}\"}], \",\", RowBox[{\"Visible\", \"\\[Rule]\", \"True\"}]}], \"]\"}]], \"Input\"]]; SelectionMove[nb, Previous, Cell]; SelectionEvaluate[nb]; NotebookClose[nb]]");

Running a SelectionEvaluate is easier when simply reading from an existing notebook, but to create the notebook on-the-fly in .NET required the box format which was obtained by evaluating the second of the cells posted below:

CreateDocument[{TextCell["Launched by JVM Kernel", "Title"]}, Visible -> True]

(separate cell)

SelectionMove[InputNotebook[], Previous, Cell, 2];
NotebookRead[InputNotebook[]]

with \ replaced by \\ and " replaced by \".