11
$\begingroup$

ExternalEvaluate makes it possible to call Python code from Mathematica. The Wolfram Client for Python makes it possible to call Mathematica from Python.

When using ExternalEvaluate, is it possible to call back to the same Mathematica kernel?

For the sake of example, suppose I have a Python function nint that does numerical integration. It takes another function as argument—the function to be integrated. Is there a way to use the Python function nint through ExternalEvaluate/ExternalFunction to integrate some function f that was defined in Mathematica?

To be clear, this is just an example to help explain the problem. I am not actually looking to do numerical integration, and at this moment performance (which would be a big deal for practical numerical integration) is not of concern to me.

$\endgroup$
  • $\begingroup$ Do you mean something more complex than func = Echo; ExternalEvaluate["Python", "from wolframclient.language import wl; wl.Global.func(3)"]? $\endgroup$ – Jason B. Aug 28 at 15:38
  • $\begingroup$ @JasonB. This is not calling back to Mathematica. It is merely returning func[3], which evaluates after return. I basically want to evaluate stuff before the Python function returns, and I want to evaluate it in the calling kernel (not in a separate, independent kernel). $\endgroup$ – Szabolcs Aug 28 at 15:58
  • $\begingroup$ @JasonB. For the sake of example, evaluate func[1] and func[2] while in the Python function, then compute the sum of results in Python, then return the result to Mathematica. $\endgroup$ – Szabolcs Aug 28 at 15:58
  • $\begingroup$ My sense is that the main issue is connecting back to the evaluating kernel? It looks like this communication happens through ZMQ, so you'll likely need to open a ZMQ connection on this side, connect through this WolframKernelController object, and then talk through a session wrapped around that. It seems like an interesting little project, but probably gonna be a pain to get the bi-directionality working well. $\endgroup$ – b3m2a1 Aug 28 at 20:48
  • $\begingroup$ I want to evaluate it in the calling kernel ExternalEvaluate doesn't complete until python returns, so at the time of the callback wouldn't this just put the Mathematica function to call at the end of the evaluation queue? i.e it will not run until ExternalEvaluate has finished. I can't see a way around it without launching a new kernel, assuming ExternalEvaluate is atomic like this and Mathematica is not interruptable. $\endgroup$ – flinty Aug 29 at 17:02
7
$\begingroup$

I'm Riccardo Di Virgilio, one of the developers of ExternalEvaluate and wolframclientforpython.

There is a way to do what you want, even if the function is not documented or exposed, so the code I'm giving to you might break in future versions.

try to run this in ExternalEvaluate:

from wolframclient.language.side_effects import wl_side_effect
from wolframclient.language import wl

wl_side_effect(wl.Print(2))
wl_side_effect(wl.Print(3))
wl_side_effect(wl.Print(4))

"actual result"

So basically we have a standard python logger that can send messages back to the original Kernel. The kernel receives the message and keeps waiting for the result of the computation.

If you are writing something complex I suggest you to take a look at the externalevaluate code, which is public.

https://github.com/WolframResearch/WolframClientForPython/blob/master/wolframclient/utils/externalevaluate.py

https://github.com/WolframResearch/WolframClientForPython/blob/master/wolframclient/language/side_effects.py

| improve this answer | |
$\endgroup$
  • $\begingroup$ Thanks for the answer! One issue is that wl_side_effect does not appear to return anything. Thus, we can use this to print message in WL, but not to use the WL kernel for computations. $\endgroup$ – Szabolcs Aug 30 at 10:59
  • $\begingroup$ Right, as the name suggests the function is sending a pure side effect to the kernel, it doesn't wait for a reply, just dropping a message back. That is used internally to forward stdout to the kernel. If you need a response from the Kernel what is wrong with running ExternalEvaluate, return something, do a computation with it and call ExternalEvaluate again? $\endgroup$ – Riccardo Di Virgilio Aug 31 at 11:40
  • $\begingroup$ There are things that cannot be accomplished this way. Assume we have a Python function which takes a callback as the argument. Integration is such a case. $\endgroup$ – Szabolcs Aug 31 at 16:02
  • $\begingroup$ thanks for the feedback we will discuss a way to handle this usecase. $\endgroup$ – Riccardo Di Virgilio Sep 1 at 17:24
  • $\begingroup$ Thanks for the answer! $\endgroup$ – Szabolcs Sep 1 at 17:57
1
$\begingroup$

Yes. It's mentioned in the document of ExternalEvaluate

The command "cmd" may contain an inline template expression <*expr*>. This evaluates expr before "cmd" is sent to the external evaluator.

that we could use TemplateExpression in the string passed to ExternalEvaluate. A basic example :

ExternalEvaluate["Python", "<* Range@5 *>"]

However, this method fails when you try to define a function in python session, because the <* expr *> is evaluated before the "cmd" is sent to the external evaluator.

A solution to this is defining the function in Mathematica with SetDelayed,and then call the function in python with <* *>. An example is demonstrated in this answer of mine.

| improve this answer | |
$\endgroup$
0
$\begingroup$

Maybe with this? https://reference.wolfram.com/language/WolframClientForPython/

Usage:

For functions defined in Mathematica:

Two ways:

1.

from wolframclient.evaluation import WolframLanguageSession
from wolframclient.language import wl, wlexpr
session = WolframLanguageSession()
Eval = session.evaluate

def sin(x):
    return Eval(Eval(wl.Sin)(x))
>>> sin(wl.Plus(wl.Pi,2))
Times[-1, Sin[2]]

Notice FullForm is needed.

2.

def sinPlusCos(x):
    return Eval(Eval(wlexpr("Sin@#+Cos@#&"))(x))

Use wlexpr and pure function.

In Mathematica, here is a small example:

ExternalEvaluate["Python", {"from wolframclient.evaluation import \
WolframLanguageSession",
  "from wolframclient.language import wl, wlexpr",
  "session = WolframLanguageSession()",
  "Eval = session.evaluate",
  "def sin(x): return Eval(Eval(wl.Sin)(x))", "sin(wl.Pi)"}]

{Null, Null, Null, Null, Null, 0}

"sin(wl.Pi)" can be replace by other things, for example: "sin(2.5)"

{Null, Null, Null, Null, Null, 0.598472}

| improve this answer | |
$\endgroup$
  • 1
    $\begingroup$ This does not call back to Mathematica. It starts a separate Mathematica kernel and evaluates something in that. $\endgroup$ – Szabolcs Aug 29 at 7:00

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.