10
$\begingroup$

I'm trying to export attributes and annotations to an HDF5 file. But the documentation here is horrible and there are no examples of this.

I spent some time playing, but I can't get it to work. Here is the simplest example that should have helped me to get started:

Export["test.h5", {{{1, 2}, {2, 3}}, {{10, 11}, {12, 13}}}, {"Datasets", {"m1", "m2"}}]
r = Import["test.h5", "Rules"]
Export["test1.h5", r, "Rules"]

but it fails with:

Export::errelem: The Export element Dataset1 contains a malformed data structure and could not be exported to HDF5 format. >>
Export::noopen: Cannot open C:\Users\ajasja\Documents\test1.h5. >>

So my question is: How can one export attributes with datasets? (preferably using built in Mathematica functionality)

EDIT For reference here are some external MMA HDF5 packages that might provide the required functionality, but I have not yet managed to write any attributes:

$\endgroup$
  • 1
    $\begingroup$ Just wondering, is this resolved in version 10? $\endgroup$ – Oleksandr R. Apr 26 '15 at 2:40
5
$\begingroup$

AFAIK, this isn't possible yet with Mathematica straight from the oven, but there is a package created by Scot Martin here.

As far as I can see, it implements a lot of the HDF5 functionality, including Compound Types.

For attribute creation take a look at the following functions : H5Acreate, H5Awrite, H5Aclose (the A comes from "Attribute")

BTW, I think that Annotations are deprecated in V5.

$\endgroup$
  • $\begingroup$ Yup, I've seen the package. There is also this which goes through intermidiate dump files and another one (I have the link at home) with C extensions. $\endgroup$ – Ajasja Apr 8 '13 at 19:18
  • $\begingroup$ Yes, you're right about the annotations, they are now attributes of the root group. $\endgroup$ – Ajasja Apr 8 '13 at 19:22
  • $\begingroup$ Also, Python makes this kind of stuff real sweet h5py.org/docs/intro/quick.html :) $\endgroup$ – Ajasja Apr 8 '13 at 19:23
  • $\begingroup$ @Ajasja as with other formats, Mma does a better job importing than exporting. I haven't an official confirmation, but my experiments show that writing attributes is not available $\endgroup$ – Dr. belisarius Apr 8 '13 at 19:23
  • $\begingroup$ Regarding the extra package I get: System.IO.FileLoadException: Mixed mode assembly is built against \ version 'v2.0.50727' of the runtime and cannot be loaded in the 4.0 \ runtime without additional configuration information. out of the box (using MMA9). So if you provide a short example of writing attributes the (admittedly nominal) bounty is yours :) Aaa, for mma9 you probably have to download the 4.0 net assemblies from hdf5.net $\endgroup$ – Ajasja Apr 8 '13 at 20:08
3
$\begingroup$

This can be done by adding a set of "Attributes" after the datasets. You can add all sorts of attributes and annotations:

Export["test.h5",
  {{{1,2},{2,3}},{{10,11},{12,13}}},
  {"Datasets",{"m1","m2"}},
  {"Attributes"->{
    "m1"->{
      "ScalarValue"->{10.0},
      "ArrayValues"->{1.0,2.0,3.0},
      "Info"->{"\"This is a text string\""}},
    "m2"->{
      "ScalarValue"->{20.0},
      "ArrayValues"->{-1.0,-2.0,-3.0},
      "Info"->{"\"This is a text string\""}}}}]

You can confirm the Attributes were saved:

In[] := Import["example.h5", "Attributes"]
Out[] := {{ArrayValues->{1.,2.,3.},Info->{"This is a text string"},ScalarValue->{10.}},{ArrayValues->{-1.,-2.,-3.},Info->{"This is a text string"},ScalarValue->{20.}}}
$\endgroup$
1
$\begingroup$

You can see:

https://github.com/scotmartin1234/HDF5Mathematica

The package has the functionality you need for writing at low level. The high level functions are designed more for reading. The package could be expanded as open source project to develop the writing part. It's straight forward to assemble the low level functions into a more Mathematica-friendly format (i.e., the higher level functions), but it would take a volunteer to do so. My efforts and work have been on developing the reading side of the equation into a Mathematica friendly format.

This is version 2.00 (August 2016) of the package that was originally provided as version 1.00 in July 2011.

$\endgroup$
1
$\begingroup$

In Mathematica 12 there are two ways to export Attributes:

Either as a subelement to a dataset:

In[1]:= Export["test.h5", "path/to/dataset" -> {
    "Data" -> {1, 2, 3, 4},
    "Attributes" -> {
      "DataDescription" -> "SomeIntegers",
      "DataLength" -> 4
      }
    }
  ];
Import["test.h5", {"Attributes", "path/to/dataset"}]

Out[2]= <|"DataDescription" -> "SomeIntegers", "DataLength" -> 4|>

or as a separate Export element:

In[3]:= Export["test.h5", {
   "Datasets" ->
    "path/to/dataset" -> {1, 2, 3, 4},
   "Attributes" ->
    "path/to/dataset" -> {
      "DataDescription" -> "SomeIntegers",
      "DataLength" -> 4
     }
   }, 
   "Rules"
];
Import["test.h5", {"Attributes", "path/to/dataset"}]

Out[4]= <|"DataDescription" -> "SomeIntegers", "DataLength" -> 4|>

In M12 it is possible to add new attributes to existing objects in the file if you use OverwriteTarget -> "Append":

In[5]:= Export["test.h5",
  "path/to/dataset" -> {
    "NewAttribute" -> <|"x" -> 1, "y" -> "z"|>
   }, 
  "Attributes", 
  OverwriteTarget -> "Append"
];
Import["test.h5", {"Attributes", "path/to/dataset"}]

Out[6]= <|"DataDescription" -> "SomeIntegers", "DataLength" -> 4, 
 "NewAttribute" -> <|"x" -> 1, "y" -> "z"|>|>

It is even possible to change values of existing attributes, with "AppendMode" -> "Overwrite" option:

In[7]:= Export["test.h5",
  "path/to/dataset" -> {
    "DataDescription" -> "New description of the data."
   }, 
  "Attributes", 
  OverwriteTarget -> "Append", 
  "AppendMode" -> "Overwrite"
];
Import["test.h5", {"Attributes", "path/to/dataset"}]

Out[8]= <|"DataDescription" -> "New description of the data.", "DataLength" -> 4,
 "NewAttribute" -> <|"x" -> 1, "y" -> "z"|>|>
$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.