10
$\begingroup$

I'm trying to export attributes and annotations to an HDF5 file. But the documentation here is horrible and there are no examples of this.

I spent some time playing, but I can't get it to work. Here is the simplest example that should have helped me to get started:

Export["test.h5", {{{1, 2}, {2, 3}}, {{10, 11}, {12, 13}}}, {"Datasets", {"m1", "m2"}}]
r = Import["test.h5", "Rules"]
Export["test1.h5", r, "Rules"]

but it fails with:

Export::errelem: The Export element Dataset1 contains a malformed data structure and could not be exported to HDF5 format. >>
Export::noopen: Cannot open C:\Users\ajasja\Documents\test1.h5. >>

So my question is: How can one export attributes with datasets? (preferably using built in Mathematica functionality)

EDIT For reference here are some external MMA HDF5 packages that might provide the required functionality, but I have not yet managed to write any attributes:

$\endgroup$
1
  • 1
    $\begingroup$ Just wondering, is this resolved in version 10? $\endgroup$ Apr 26, 2015 at 2:40

4 Answers 4

5
$\begingroup$

AFAIK, this isn't possible yet with Mathematica straight from the oven, but there is a package created by Scot Martin here.

As far as I can see, it implements a lot of the HDF5 functionality, including Compound Types.

For attribute creation take a look at the following functions : H5Acreate, H5Awrite, H5Aclose (the A comes from "Attribute")

BTW, I think that Annotations are deprecated in V5.

$\endgroup$
9
  • $\begingroup$ Yup, I've seen the package. There is also this which goes through intermidiate dump files and another one (I have the link at home) with C extensions. $\endgroup$
    – Ajasja
    Apr 8, 2013 at 19:18
  • $\begingroup$ Yes, you're right about the annotations, they are now attributes of the root group. $\endgroup$
    – Ajasja
    Apr 8, 2013 at 19:22
  • $\begingroup$ Also, Python makes this kind of stuff real sweet h5py.org/docs/intro/quick.html :) $\endgroup$
    – Ajasja
    Apr 8, 2013 at 19:23
  • $\begingroup$ @Ajasja as with other formats, Mma does a better job importing than exporting. I haven't an official confirmation, but my experiments show that writing attributes is not available $\endgroup$ Apr 8, 2013 at 19:23
  • $\begingroup$ Regarding the extra package I get: System.IO.FileLoadException: Mixed mode assembly is built against \ version 'v2.0.50727' of the runtime and cannot be loaded in the 4.0 \ runtime without additional configuration information. out of the box (using MMA9). So if you provide a short example of writing attributes the (admittedly nominal) bounty is yours :) Aaa, for mma9 you probably have to download the 4.0 net assemblies from hdf5.net $\endgroup$
    – Ajasja
    Apr 8, 2013 at 20:08
3
$\begingroup$

This can be done by adding a set of "Attributes" after the datasets. You can add all sorts of attributes and annotations:

Export["test.h5",
  {{{1,2},{2,3}},{{10,11},{12,13}}},
  {"Datasets",{"m1","m2"}},
  {"Attributes"->{
    "m1"->{
      "ScalarValue"->{10.0},
      "ArrayValues"->{1.0,2.0,3.0},
      "Info"->{"\"This is a text string\""}},
    "m2"->{
      "ScalarValue"->{20.0},
      "ArrayValues"->{-1.0,-2.0,-3.0},
      "Info"->{"\"This is a text string\""}}}}]

You can confirm the Attributes were saved:

In[] := Import["example.h5", "Attributes"]
Out[] := {{ArrayValues->{1.,2.,3.},Info->{"This is a text string"},ScalarValue->{10.}},{ArrayValues->{-1.,-2.,-3.},Info->{"This is a text string"},ScalarValue->{20.}}}
$\endgroup$
2
$\begingroup$

In Mathematica 12 there are two ways to export Attributes:

Either as a subelement to a dataset:

In[1]:= Export["test.h5", "path/to/dataset" -> {
    "Data" -> {1, 2, 3, 4},
    "Attributes" -> {
      "DataDescription" -> "SomeIntegers",
      "DataLength" -> 4
      }
    }
  ];
Import["test.h5", {"Attributes", "path/to/dataset"}]

Out[2]= <|"DataDescription" -> "SomeIntegers", "DataLength" -> 4|>

or as a separate Export element:

In[3]:= Export["test.h5", {
   "Datasets" ->
    "path/to/dataset" -> {1, 2, 3, 4},
   "Attributes" ->
    "path/to/dataset" -> {
      "DataDescription" -> "SomeIntegers",
      "DataLength" -> 4
     }
   }, 
   "Rules"
];
Import["test.h5", {"Attributes", "path/to/dataset"}]

Out[4]= <|"DataDescription" -> "SomeIntegers", "DataLength" -> 4|>

In M12 it is possible to add new attributes to existing objects in the file if you use OverwriteTarget -> "Append":

In[5]:= Export["test.h5",
  "path/to/dataset" -> {
    "NewAttribute" -> <|"x" -> 1, "y" -> "z"|>
   }, 
  "Attributes", 
  OverwriteTarget -> "Append"
];
Import["test.h5", {"Attributes", "path/to/dataset"}]

Out[6]= <|"DataDescription" -> "SomeIntegers", "DataLength" -> 4, 
 "NewAttribute" -> <|"x" -> 1, "y" -> "z"|>|>

It is even possible to change values of existing attributes, with "AppendMode" -> "Overwrite" option:

In[7]:= Export["test.h5",
  "path/to/dataset" -> {
    "DataDescription" -> "New description of the data."
   }, 
  "Attributes", 
  OverwriteTarget -> "Append", 
  "AppendMode" -> "Overwrite"
];
Import["test.h5", {"Attributes", "path/to/dataset"}]

Out[8]= <|"DataDescription" -> "New description of the data.", "DataLength" -> 4,
 "NewAttribute" -> <|"x" -> 1, "y" -> "z"|>|>
$\endgroup$
1
$\begingroup$

You can see:

https://github.com/scotmartin1234/HDF5Mathematica

The package has the functionality you need for writing at low level. The high level functions are designed more for reading. The package could be expanded as open source project to develop the writing part. It's straight forward to assemble the low level functions into a more Mathematica-friendly format (i.e., the higher level functions), but it would take a volunteer to do so. My efforts and work have been on developing the reading side of the equation into a Mathematica friendly format.

This is version 2.00 (August 2016) of the package that was originally provided as version 1.00 in July 2011.

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.