HDF5 and complex numbers

Question

I'm exporting an array with complex numbers with HDF5 format. When I'm import it, I only get the real part. Simple example:

qww = {1.0+I, 2.0+I};
Export["file.h5", qww, "DataEncoding" -> "GZIP"];
Import["file.h5", {"Datasets", "/Dataset1"}]

{1., 2.}

How can I solve this problem?

Answer 1

HDF5 does not directly support complex numbers. Programs that do seem to be able to export complex numbers (like armadillo) to HDF5 will in reality split them into real and imaginary part and use their own non-standard convention for storing these. This means that while they can sometimes read back their own data, there is no compatibility between different software regarding storing complex values in HDF5.

The solution I recommend is that you split into real and imaginary parts manually, and export them as two real datasets.

Answer 2

Szabolcs summed it up pretty well but there is also another popular convention to store complex numbers in HDF5 as compound type with two members being real numbers.

So starting from M11.1 you can import it like this:

In[1]:= Import["ExampleData/sample1.h5", "Complex/Complex64"]

Out[1]= <|"re" -> 0.58215809, "im" -> -0.92684859|>

[UPDATE for M12.0]

Both Import and Export take an option called "ComplexKeys" that can trigger automatic conversion of such compound types to complex numbers. With "ComplexKeys" -> {X, Y}, where X and Y are different Strings, Import will convert all values of the form <|X -> a, Y -> b|> to complex numbers (assuming a and b are real numbers). So

In[1]:= Import["ExampleData/sample1.h5", "Complex/Complex64", 
 "ComplexKeys" -> {"re", "im"}]

Out[1]= 0.58215809 - 0.92684859 I

And similarly, when given "ComplexKeys" -> {X, Y}, Export will save complex numbers a+bI into <|X -> a, Y -> b|>:

In[3]:= Export["test.h5", {1 + 3 I, Pi - 2.6*I}, "ComplexKeys" -> {"Re", "Im"}]

Out[3]= "test.h5"


In[4]:= Import["test.h5", "Dataset1"]

Out[4]= {<|"Re" -> 1., "Im" -> 3.|>, <|"Re" -> 3.1415927, "Im" -> -2.6|>}


In[5]:= Import["test.h5", "Dataset1", "ComplexKeys" -> {"Re", "Im"}]

Out[5]= {1. + 3. I, 3.1415927 - 2.6 I}