Wolfram Client Library: Cannot establish connection with Kernel in second Python function

Question

I have recently started using the Wolfram Client Library (WCL) for Python in my Linux machine in order to automate running a Mathematica notebook many times.

I have an Anaconda environment wolfram_env where I installed the WCL and automated the Mathematica notebook by creating a function that I call many times, let's say from script_1.py. For illustration purposes, my function has this structure:

from wolframclient.evaluation import WolframLanguageSession
from wolframclient.language import wlexpr

def run_notebook():
    session = WolframLanguageSession('/usr/local/Wolfram/Mathematica/12.1/Executables/WolframKernel')
    session.evaluate(wlexpr('n = 3'))
    session.terminate()

When calling this from script_1.py I have no problems and everything works as expected. However, I want to run this same run_notebook() function from script_2.py because my actual run_notebook() function has an import so I am feeding it other data.

When running run_notebook() from script_2.py I get the following error:

Socket exception: Socket operation aborted.
Failed to start.
Traceback (most recent call last):
  File "/home/me/anaconda3/envs/wolfram_env/lib/python3.8/site-packages/wolframclient/evaluation/kernel/kernelcontroller.py", line 435, in _kernel_start
    response = self.kernel_socket_in.recv_abortable(
  File "/home/me/anaconda3/envs/wolfram_env/lib/python3.8/site-packages/wolframclient/evaluation/kernel/zmqsocket.py", line 50, in recv_abortable
    raise SocketAborted("Socket operation aborted.")
wolframclient.evaluation.kernel.zmqsocket.SocketAborted: Socket operation aborted.
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
  File "/home/me/anaconda3/envs/wolfram_env/lib/python3.8/site-packages/wolframclient/evaluation/kernel/localsession.py", line 157, in start
    raise e
  File "/home/me/anaconda3/envs/wolfram_env/lib/python3.8/site-packages/wolframclient/evaluation/kernel/localsession.py", line 152, in start
    future.result(timeout=timeout)
  File "/home/me/anaconda3/envs/wolfram_env/lib/python3.8/concurrent/futures/_base.py", line 439, in result
    return self.__get_result()
  File "/home/me/anaconda3/envs/wolfram_env/lib/python3.8/concurrent/futures/_base.py", line 388, in __get_result
    raise self._exception
  File "/home/me/anaconda3/envs/wolfram_env/lib/python3.8/site-packages/wolframclient/evaluation/kernel/kernelcontroller.py", line 527, in run
    self._safe_kernel_start()
  File "/home/me/anaconda3/envs/wolfram_env/lib/python3.8/site-packages/wolframclient/evaluation/kernel/kernelcontroller.py", line 366, in _safe_kernel_start
    raise e
  File "/home/me/anaconda3/envs/wolfram_env/lib/python3.8/site-packages/wolframclient/evaluation/kernel/kernelcontroller.py", line 360, in _safe_kernel_start
    self._kernel_start()
  File "/home/me/anaconda3/envs/wolfram_env/lib/python3.8/site-packages/wolframclient/evaluation/kernel/kernelcontroller.py", line 455, in _kernel_start
    raise WolframKernelException(
wolframclient.exception.WolframKernelException: Failed to communicate with kernel: /usr/local/Wolfram/Mathematica/12.1/Executables/WolframKernel.
python-BaseException

If I debug script_2.py and type session inside the run_notebook() function after the first line, I get the following description of the session variable:

<WolframLanguageSession: <WolframKernelController[wolfram-kernel-1 ❌], "/usr/local/Wolfram/Mathematica/12.1/Executables/WolframKernel">>

When doing the same from script_1.py it shows:

<WolframLanguageSession: <WolframKernelController[wolfram-kernel-1 ✅], "/usr/local/Wolfram/Mathematica/12.1/Executables/WolframKernel", pid:10321, kernel sockets: (in:tcp://127.0.0.1:34411, out:tcp://127.0.0.1:37803)>>

It seems to me that this second connection cannot be started but I have no clue why. I have tried restarting the computer and running from script_2.py directly to ensure no other kernel was active before.

In summary, I am able to establish a connection with the Kernel from the console in Pycharm and from script_1.py but not from script_2.py.

Any help would be very much appreciated.

Answer 1

I tried to reproduce you issue but could not. You're not providing the second script, it I wrote the simplest possible one I could think of and ran it:

script1.py:

from wolframclient.evaluation import WolframLanguageSession
from wolframclient.language import wlexpr

def run_notebook():
    session = WolframLanguageSession('/Applications/Mathematica12.1.app/Contents/MacOS/WolframKernel')
    print(session.evaluate(wlexpr('n = 3')))
    session.terminate()

and script2.py:

import script1

script1.run_notebook()

I ran the script using: $ python3 script2.py, granted that:

python3 --version
Python 3.8.5

You code is not using with statement, so if for some reason it is interrupted before the terminate() call, it'll spawn an orphan kernel.

Here is a simple command to look up kernels spawned by the WCL:

 ps -eaf | grep "WolframKernel.*SlaveKernelPrivateStart" | grep -v grep

Answer 2

After digging around in the debugger I found the connection was not being established because a specific line of code was being run previously in script_2.py. The line causing the error was:

replicas = np.array([np.array([np.corrcoef(ts_replica.T) for ts_replica in ts_replicas_gb_batch]) for ts_replicas_gb_batch in ts_replicas_batches])

where ts_replicas_batches is an np.array of shape (1000, 99, 22, 100). This line was producing a RuntimeWarning in the np.corrcoef function.

At this stage, I am unsure why this causes kernel connection error. Could be due to the runtime error or some other reason. I will investigate further and try to produce a minimal working example to post here and on the GitHub issues section.