Using local variables in ExternalEvaluate Python

Question

I am sure that I am missing something here, I have some larger matrices and Mathematica takes too long to compute the eigenvalues and vectors (using Eigensystem[m]) so I would like to pass them to a python environment using ExternalEvaluate so as to take advantage of Python's much faster computation for this.

I am sure there is an easy way to pass my matrix to Python, but I cannot see how.

MWE:

In[1]:= linearMap = {{1, 0, 0, 0, 0, 0, 0}, {0, 1, 0, 0, 0, 0, 0}, {0, 0, 1, 0, 0, 0, 0},
             {0, 0, 0, 1/4, 0, 0, 0}, {0, 0, 0, 0, 1/2, 0, 0},
             {0, 0, 0, 0, 0, 1/2, 0}, {0, 0, 0, 0, 0, 0, 1/4}};

In[2]:= ExternalEvaluate["Python", "import numpy; numpy.linalg.eigvalsh(linearMap)"]
Out[2]=

I am very aware that it is not necessary for such small matrices (or indeed this matrix at all really!), but as they get larger, I would greatly appreciate the ability to utilise the speed increases.

Clearly this isn't expected to work instantly as the linearMap from Mathematica would need to be reformatted to a Python array. I am just hoping that there is an easy way to do this that I have overlooked...

Answer 1

You may use the Association syntax for ExternalEvaluate.

If numpy is installed in your Python instance then you should have a "Python-NumPy" external evaluator. Check by evaluating FindExternalEvaluators[].

Initialise the connection with

ExternalEvaluate["Python-NumPy", "1+1"]

2

Then

ExternalEvaluate["Python-NumPy",
 <|
  "Command" -> "numpy.linalg.eigvalsh",
  "Arguments" -> {linearMap}
 |>
]

{0.25, 0.25, 0.5, 0.5, 1., 1., 1.}

If you need to use this often then create a function

numpyEigvalsh[m_?MatrixQ] :=
 ExternalEvaluate["Python-NumPy",
  <|
   "Command" -> "numpy.linalg.eigvalsh",
   "Arguments" -> {m}
  |>
 ]

Then

numpyEigvalsh@linearMap

{0.25, 0.25, 0.5, 0.5, 1., 1., 1.}

Why it may be slower

Note that when using Rationals that Mathematica will take longer as it works to preserve the infinite precision of rationals.

Eigenvalues@linearMap

{1, 1, 1, 1/2, 1/2, 1/4, 1/4}

You can speed things up by using Reals. All you need to do is multiply by 1.

Eigenvalues[1. linearMap]

{1., 1., 1., 0.5, 0.5, 0.25, 0.25}

Note that the output is now with reals instead of rationals

Hope this helps.

Answer 2

If you are willing to setup and use WolframClientForPython you could do:

With Mathematica

linearMap = 1. {{1, 0, 0, 0, 0, 0, 0}, {0, 1, 0, 0, 0, 0, 0}, 
                {0, 0, 1, 0, 0, 0, 0}, {0, 0, 0, 1/4, 0, 0, 0}, 
                {0, 0, 0, 0, 1/2, 0, 0}, {0, 0, 0, 0, 0, 1/2, 0}, 
                {0, 0, 0, 0, 0, 0, 1/4}};
Export[FileNameJoin[{"C:", "temp", "linearMap.wxf"}], "WXF"]

then in Python

import numpy as np
import os
from wolframclient.evaluation import WolframLanguageSession
from wolframclient.serializers import export

math_kernel = r'C:\Program Files\Wolfram Research\Mathematica\11.3\MathKernel.exe'
output_path = r'C:\temp'

session = WolframLanguageSession(math_kernel)
session.start()

linear_map = session.evaluate('Import[FileNameJoin[{"C:", "temp", "linearMap.wxf"}]]')

linear_map = np.array(linear_map)

out = np.linalg.eigvalsh(linear_map)

export(out, os.path.join(output_path, 'out.wxf'), target_format='wxf')

session.terminate()

finally back in Mathematica

Import[FileNameJoin[{"C:", "temp", "out.wxf"}]] // Normal
(* {0.25, 0.25, 0.5, 0.5, 1., 1., 1.} *)

Answer 3

If you don't want to use the association as suggested by Edmund (or if you want to do something more general with the matrix than just sending it as an argument of a function), use ExportString with the option "PythonExpression":

Py[ex_] := ExportString[ex, "PythonExpression"] 
ExternalEvaluate["Python", "import numpy; numpy.linalg.eigvalsh(" <> Py[linearMap] <> ")"]

Notice that you will obtain the result will in the form of NumericArray, which can be transformed to a regular list by Normal[%].