Dynamically access variables in external session?

Question

I tried to get a value from a python session to show up in a dynamic cell:

First run:

Dynamic[ExternalEvaluate[CurrentValue[
   EvaluationNotebook[], {TaggingRules, "ExternalSessions", "Python"}], "i"], 
 UpdateInterval -> 0]

Then run this in a python code cell:

global i
import time 
for x in range(10):
    time.sleep(1)
    i = x
    print(x)

But it doesn't seem to work, is something like this even possible?

Answer 1

I believe this general approach is destined to fail. You are asking Mathematica to interrupt python, get a value from it, then allow python to continue running. I am very skeptical that such a thing is possible.

A more realistic approach is to have python send back any values you wish to be kept current on. Here is a simple implementation

command = {"import time", "for x in range (10):", "    time.sleep(1)","    i = x", "    print(x)", ""};
process = StartProcess[{"python", "-i"}]
Dynamic[Refresh[Block[{temp},
    temp = ReadString[process, EndOfBuffer];
    If[temp != "", x = ToExpression[temp]];
    ], UpdateInterval -> 0.1]];
Dynamic[x]
WriteLine[process, #] & /@ command;

The basic idea here is to use a Process instead of an ExternalSession. From the documentation on StartProcess:

StartProcess can be used to interact with processes while they are running or to run "background tasks" without blocking the Wolfram Engine.

StartProcess returns immediately and does not wait for the started process to complete. Use RunProcess to wait for a process to complete.

The Wolfram Engine can communicate with a subprocess represented by a ProcessObject with functions like WriteLine, WriteString, BinaryWrite, ReadLine, ReadString, and BinaryRead.

This allows us to easily read in the outputs of a python program while it's running. Print statements are written to a Stream which can be accessed in real time. Theoretically one may be able to accomplish similar functionality with an ExternalSession by redirecting $Output to its own Stream and then reading from that, but it wasn't designed to be used this way and would be much uglier if it worked at all.

I am using the ReadString function with EndOfBuffer to make it read what's available immediately and only storing the value if it's not "" which is what's returned when the Buffer is empty. This in effect gives x the value of the last character printed by python. If you want to keep track of more complicated stuff, a more complicated print/read scheme is likely in order.

The command list looks a bit clunky, but it would be simple and elegant to write a python script, read it into a list then execute as shown above.