3
$\begingroup$

Related question: Formatting Imported Complex Arrays from Python (csv. files)

I am trying to import a large numpy array to Mathematica but the strategy in the question above is far too slow. For a 100-by-100 array it already takes quite a long time. Is there a fast, elegant way to import a complex numpy array?

Here is some code to generate such an array in python:

import numpy as np
import numpy.random

myArray = np.zeros((1000,1000),dtype='complex')
for (i,k) in enumerate(myArray):
    myArray[i] = numpy.random.normal(0,1,1000)+1J*numpy.random.normal(0,1,1000)

np.savetxt("exampleArray",myArray)
$\endgroup$
1
  • $\begingroup$ Can you, please, specify your version number? Thanks! $\endgroup$ Sep 12, 2021 at 23:24

3 Answers 3

1
$\begingroup$

You could export binary reals from Python and Import "Real64".

Convert Python's numpy.ndarray myArray of complex numbers into reals of their real and imaginary parts and export binary with.

myArray.view(float).tofile("exampleArrayBinary")

Then Import with

r = Import[
   FileNameJoin[{$UserDocumentsDirectory, "exampleArrayBinary"}]
   , "Real64"
   ];

The import should hold $1000 \times 1000 \times 2 = 2000000$ items

Dimensions@r
{2000000}

Partition into pairs and Apply Complex

r = Complex @@@ Partition[r, 2];
r[[1]]
2.41713 +2.45023 I

The 100000 complex numbers are recovered.

Import "Real64" is instantaneous for this example. However, you do lose the dimensions of the array.

An alternative would be to

  1. find a Python library for a file type that that supports matrix exports and has support in Wolfram Language, like "MAT".
  2. Convert Python's array to reals and export the matrix
  3. Import into Mathematica and partition & convert each row as above.

Hope this helps.

$\endgroup$
1
  • $\begingroup$ Thank you, this was very helpful. I will accept your answer but also add my own because your strategy helped me come up with another that I think is also relevant. $\endgroup$
    – Diffycue
    Oct 19, 2021 at 23:04
2
$\begingroup$

You don't need to export and then import that. Just use the ExternalEvaluate functionality or an ExternalLanguageCell (on mac: Shift + .):

enter image description here

exampleArray = ExternalEvaluate["Python", "import numpy as np
import numpy.random
myArray = np.zeros((1000,1000),dtype='complex')
for (i,k) in enumerate(myArray):
    myArray[i] = numpy.random.normal(0,1,1000)+1J*numpy.random.normal(0,1,1000)
myArray
"]

enter image description here

$\endgroup$
1
  • $\begingroup$ Unfortunately I cannot set up ExternalEvaluate functionality on my machine. $\endgroup$
    – Diffycue
    Sep 10, 2021 at 22:16
0
$\begingroup$

The solution I ended up using imports the data quickly but is fragile. What I did was export the data in Python as

myArray.astype(np.complex128).tofile("myData.dat")

then in Mathematica I used the symbol BinaryReadList passing the appropriate data type in order to read in the array.

$\endgroup$
2
  • $\begingroup$ Can you, please, note your version number for future users? This is very helpful as you state that you cannot use ExternalEvaluate and may serve as a workaround for future users in the same version as you. $\endgroup$ Oct 19, 2021 at 23:33
  • $\begingroup$ When I posted the question I was on version 12.2, now I am on version 12.3. Using ExternalEvaluate is a bad call anyway. $\endgroup$
    – Diffycue
    Oct 20, 2021 at 0:06

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.