Exporting structured data to HDF5

Question

Sadly the documentation is lacking any useful examples on how to Export more complex data in HDF5. There is only

Export["file.h5",{expr1,...},{"Datasets", {"dataset1",...}}] creates an HDF5 file, storing the data arrays {expr1,...} as separate datasets.

The information on how to import HDF5 is more helpful. Information what can be imported is given in a table reproduced below:

Is there any way to export data in a more structured manner than simply as a flat list of individual datasets? Notice the 4th and 5th entry in the table above for importing. Is there an equivalent feature for exporting as well?

Take for instance data in the form of

data = <|
         "group1" -> <|"subgroup1" -> ..., "subgroup2" -> ...|>,
         "group2" -> <|"subgroup1" -> ..., "subgroup2" -> ...|>
         ...
       |> 

Answer 1

Thanks to @dan7geo's helpful hint to ExportStructuredHDF5 exporting to HDF5 is easier than I thought.

Take for instance

data = <|
         "group1" -> <|"subgroup1" -> "group1/subgroup1", 
                       "subgroup2" -> "group1/subgroup2", 
                       "subgroup3" -> 
                              <|
                              "subsubgroup1" -> "group1/subgroup3/subsubgroup1", 
                              "subsubgroup2" -> "group1/subgroup3/subsubgroup2"
                              |>
                     |>, 
         "group2" -> <|"subgroup1" -> "group2/subgroup1"|>|>

Create a temporary file and export data

file = CreateFile[];
ExportStructuredHDF5[file, data];

Importing the file with Import[file] shows a list of datasets names within the file.

{"/group1/subgroup1", "/group1/subgroup2", 
 "/group1/subgroup3/subsubgroup1", "/group1/subgroup3/subsubgroup2",
 "/group2/subgroup1"}

Note that these are the internal names of the datasets and not the content (I choose the content of data to match these names for testing).

A neat feature is to import the file's structure with Import["/Users/Sascha/Desktop/temp.h5", "StructureGraph"]

Note that layout isn't too smart and the root node is the lightblue dot on the left.

---

Importing the actual data back as an association is done with

imported = Import[file, "Data"] 

<|"/group1/subgroup1" -> "group1/subgroup1", 
   "/group1/subgroup2" -> "group1/subgroup2", 
   "/group1/subgroup3/subsubgroup1" -> "group1/subgroup3/subsubgroup1", 
   "/group1/subgroup3/subsubgroup2" -> "group1/subgroup3/subsubgroup2", 
   "/group2/subgroup1" -> "group2/subgroup1"|>

As you see the importer isn't too smart either and just gives a flat associaton even though Import is aware of the file's structure as can be seen from the graph above.

To get back the original structure one can use ImportStructuredHDF5 instead.

ImportStructuredHDF5["file"] reads an HDF5 file previously created by ExportStructuredHDF5. The option "ImportAsRawArray" controls whether arrays of numbers are returned as RawArrays.

Answer 2

This might be helpful, although there aren't any details in the documentation.