ExternalEvaluate for Python fails with Import::nopythonevals

Question

I tried using ExternalEvaluate on OS X with Anaconda Python, and every ExternalEvaluate call failed with the Import::nopythonevals message. This happened even though an external evaluator was successfully registered, and pyzmq was installed in Anaconda Python.

What is the problem?

Answer 1

To get this out of the way:

I believe the Import::nopythonevals message will be issued by Import[..., "PythonExpression"] if there are no usable external evaluators set up. First make sure that you have set up Python correctly (including installing pyzmq), and registered an external evaluator.

---

As you can see in my question, I have already done this, yet ExternalEvaluate was not working.

To cut a long story short, I believe that this is due to a bug in M11.2.0 where Import[..., "PythonExpression"] attempts to find a Python installation on its own, and does not even look at what installations the user has already registered using RegisterExternalEvaluator.

In my case, it was trying to use the system Python on OS X. I suspect it failed because I did not have pyzmq installed for the system Python. I prefer not to modify it, as I use a private installation of Anaconda Python instead.

Here's a workaround to force Import[..., "PythonExpression"] to use the Python installation that I have already registered:

ExternalEvaluate (* triggers loading the ExternalEvaluate framework *)
(* ExternalEvaluate *)

ExternalEvaluate`ImportExport`Private`$ImporterPythonSession = StartExternalSession["Python" -> "String"] (* this evaluator will be used to handle "PythonExpression" *)
(* ExternalSessionObject["9a23363e-5403-442d-8fbd-39f9aff7dc48"] *)

py = StartExternalSession["Python"]
(* ExternalSessionObject["5022a77c-eedb-4d39-bbb3-c8b6a644ecdf"] *)

ExternalEvaluate[py, "1+1"] (* now everything works fine! *)
(* 2 *)

I hope this information will be useful to others who want to play with ExternalEvaluate, but do not wish to modify (or even use) their system Python on OS X.

---

As mentioned by @MichaelE2, an alternative solution is to symlink your preferred Python interpreter in a location that is already being searched by the ExternalEvaluate framework. Currently, on OS X and Linux /usr/local/bin/ is such a location. To see the full list of locations, evaluate

ExternalEvaluate (* load framework *)
ExternalEvaluate`Private`$LanguageHeuristics["Python", "ExecutablePathFunction"] 

Answer 2

Not sure if you already resolved your issue - I literally beat my head against the wall for 6 weeks straight fighting the same about 5 months ago. I finally resolved the issue - it's a syntax error in the externalevaluate.py script. Follow my simple solution and hope this helps everyone:

1. Locate externalevaluate.py should be in:

C:\Program Files\Wolfram Research\<Wolfram Finance Platform>\3.0\SystemFiles\Links\WolframClientForPython\wolframclient\utils\

[note: Swap with whichever version of Mathematica you are using, or whichever is your sub-folder under \Program Files\Wolfram Research<your folder name>...]

2. Open in whichever editor you use (e.g., Notepad++), and replace line 66 as follows:

   exec(compile(ast.Module(expressions, []), '', 'exec'), current) 
   

   this is the part you copy and replace what is in line 66 with this. Very minor change, and voile!

3. You should restart 12.x and you will find ExternalEvaluate is no longer throwing that error, and opening up a Python shell "shift >" also works just as well. Of course, you will still need to use ExternalEvaluate, however, this is amazing integration!

Now, if Wolfram could post this online in their actual docs hopefully others do not lose as much sleep as I did. :)

Side note — Do not attempt to use Chocolatey to trick Mathematica into updating from 3.7.0 to 3.8.x - it will hang and freeze. What needs to be done is the ability to SystemInstall whichever version of Python we want to install as opposed to being bound to 3.7.0. Sure you can register various versions then restart, but that is not as tightly integrated as the 3.7.0 version SystemInstall does for us.

Let me know if this works for you, it did for me especially when invoking a shell "shift >" inside of a notebook, then trying to install modules...Little trick for you, just type the following for installing modules from within the shell:

import os
os.popen('pip install <module name>') # of course without the <>.

or,

import os
os.system('pip install <module name>')

Hope this helps Mathematica v12.x users get up and running with Python and use an external evaluate shell within a notebook (.nb file) quickly.