6
$\begingroup$

How to export/import basic Mathematica types using HDF5 file format?

I find that the official Mathematica doc is missing some examples.

$\endgroup$
2
  • 1
    $\begingroup$ CDF means something a bit different in Mathematica, so you might wish to clarify somewhere that you're using the "Common Data Format" (cf. this question). $\endgroup$ Aug 19, 2017 at 17:29
  • $\begingroup$ @J.M. I just edited my post, thanks $\endgroup$ Aug 19, 2017 at 17:39

1 Answer 1

7
$\begingroup$

Exporting data

To export a string, a scalar and a matrix

aString="aString";
aScalar=2017;
aMatrix=Table[Random[],{4},{2}];

The syntax is:

Export["foo.h5",
       {"Datasets" -> {"aString" -> aString, 
                       "aScalar" -> {aScalar}, 
                       "aMatrix" -> aMatrix},
        "DataFormat" -> {Automatic,Automatic,Automatic}}, "Rules"
]

AFAIK you must encapsulate scalars in a list (i.d. {aScalar}), otherwise you get this

Export::errelem: The Export element Datasets contains a malformed data    
structure and could not be exported to HDF5 format.

error message.

Also note that we have used Automatic for DataFormat which is very convenient especially for strings. You can also simply use {Automatic} instead of {Automatic,Automatic,Automatic}.

You can also explicitly define the format types:

Export["foo.h5",
       {"Datasets" -> {"aString" -> aString, 
                       "aScalar" -> {aScalar}, 
                       "aMatrix" -> aMatrix},
        "DataFormat" -> {Automatic,"Integer32","Real64"}}, "Rules"
 ]

Importing data

The command

Import["foo.h5"]

returns the datasets contained in the file

 {"/aMatrix", "/aScalar", "/aString"}

You can get some extra information, like the data types and their dimensions:

 Import["foo.h5", "DataFormat"]
 Import["foo.h5", "Dimensions"]

prints

 {"Real64", "Integer32", "String"}
 {{4, 2}, {1}, {}}

Finally you can import the data set you want:

 Import["foo.h5", {"Datasets", "aScalar"}] (* prints {2017} *)

or several data sets:

 Import["foo.h5", {"Datasets", {"aMatrix", "aString"}}]

which prints in my case:

 {{{0.840383, 0.0926723}, {0.657987, 0.619812}, {0.975782, 
    0.985007}, {0.604865, 0.515671}}, "aString\.00"}

CAVEAT: At least under Linux, you must take care that a terminal zero (à la C) has been added to the string: "aString.00":

For instance

 Import["foo.h5", {"Datasets", "aString"}] == "aString"

prints

 False

AFAIK you have to manually remove this final zero:

 StringDrop[
     Import["foo.h5", {"Datasets", {"aString"}}], -1] == "aString"

returns

 True

Caveat#2: It seems that this trailing zero is not present under Windows (see comments).

Note: this should also work with NetCDF or NASACDF format, just replace the .h5 file extension by .nc or .cdf.

$\endgroup$
5
  • 2
    $\begingroup$ Interesting, which version and OS have you tried that? I have never seen strings with terminating zeros. I just have tried your example and also don't see it there (mma 11.1 on Windows 7). I would consider that a bug... $\endgroup$ Aug 19, 2017 at 17:52
  • $\begingroup$ I am running under Linux (Debian distro), mma 11. I would not have suspected such difference either. That is an interesting point to clarify, thank you for the comment. Unfortunately, on my side, I have not access to mma under Windows. $\endgroup$ Aug 19, 2017 at 18:49
  • $\begingroup$ @AlbertRetey I forgot to mention your name in my answer, not sure you read it. Sorry, I just realized that now. $\endgroup$ Aug 25, 2017 at 17:22
  • $\begingroup$ No problem for not mentioning me, but I think it would make sense to mention that the trailing zero is an OS (and probably version) dependent problem... $\endgroup$ Aug 26, 2017 at 19:56
  • $\begingroup$ @AlbertRetey you are right, I have edited my post. Thanks. $\endgroup$ Aug 26, 2017 at 20:11

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.