2
$\begingroup$

Assume I export some data into a file like:

data = {{1, 2, 3}, {4, 5, 6}};
Export["test.h5",data,{"Datasets","/h1"}];

How can I append {7, 8, 9} to the "test.h5" (by directly writing in the test.h5) such that the results for

Import["test.h5", data, {"Datasets", "/h1"}]

will be

{{1, 2, 3}, {4, 5, 6}, {7, 8, 9}}

I would prefer to add it at the end of /h1.

$\endgroup$
5
  • 1
    $\begingroup$ this is not possible with mathematica, except obviously you can import, append the data and re export. $\endgroup$
    – george2079
    Feb 20, 2017 at 14:29
  • $\begingroup$ george2079 is right: what you want to do is not possible. There is the option "Append" for Exporting HDF5 files with which you can add additional datasets to an existing file, but it is not possible to extend an existing dataset. $\endgroup$ Feb 20, 2017 at 16:33
  • $\begingroup$ @AlbertRetey .. thanks I didn't even know about Append, but.. does it actually work? Everything I try it just overwrites the file. $\endgroup$
    – george2079
    Feb 20, 2017 at 18:16
  • 2
    $\begingroup$ @george2079: Which version do you use, this works for me with 9, 10 and 11: Export[filename, {{1, 2, 3}}, {"Datasets", "one"}] then Export[filename, {{1, 2, 3}}, {"Datasets", "two"}, "Append" -> True] and Import[filename] then shows that the file now has two datasets as expected. Does that not work for you? $\endgroup$ Feb 20, 2017 at 18:58
  • $\begingroup$ You need to use "Append", not Append. I don't know why, and I don't know why it doesn't give an error if you use the latter. This is in 10.4.1 on Mac OS. $\endgroup$
    – Ian Hinder
    Mar 17, 2017 at 19:46

2 Answers 2

5
$\begingroup$

Extending an existing dataset is not possible with the standard Mathematica Export, at least not with any version up to 11.0.1. What does work is to add additional datasets one by one like this:

filename = FileNameJoin[{$HomeDirectory, "Desktop", "tst.h5"}]

Export[filename, {{1, 2, 3}}, {"Datasets", "one"}]
Export[filename, {{1, 2, 3}}, {"Datasets", "two"}, "Append" -> True]

using:

Import[filename]

you can verify that there are now two datasets in the file. Of course that is not the same thing, but the best that currently is possible and probably good enough for some use cases. If you need more, there are some libraries which might be possible to do what you need. Alternatively you could write your own function to access HDF5 files via e.g. LibraryLink. If you search for HDF5 on this site you will find several questions and answer to guide you either way.

$\endgroup$
3
  • $\begingroup$ Thanks a lot guys! I found the syntax "BinaryWrite" which solves the problem partially by combining with OpenAppend. Unfortunately it works only for integers 0<= x <=255. $\endgroup$ Feb 20, 2017 at 19:57
  • $\begingroup$ @HesamSoltanpanahiSarabi: I don't understand what you are trying to say: are you appending data using BinaryWrite to an existing hdf5 file? I would not expect this to work if not for very specific cases by accident. It is certainly not the way to solve that problem... $\endgroup$ Feb 20, 2017 at 21:18
  • $\begingroup$ This is what I did. But as you mentioned it works only for some specific cases. $\endgroup$ Feb 21, 2017 at 7:42
1
$\begingroup$

It seems like it has changed in Version 12.0.0.0

"Append" -> True

is now implemented as

OverwriteTarget->"Append"

$\endgroup$
3
  • $\begingroup$ So does the new / modified functionality permit modification of existing datasets, or only the appendment of new ones ? $\endgroup$ Sep 4, 2019 at 10:45
  • $\begingroup$ As far as I know it's just for adding a new dataset. Modification of existing datasets may be done with: Overwrite->"Append" , "AppendMode"->"Overwrite". But the dataset must have same dimensions. $\endgroup$ Sep 4, 2019 at 12:26
  • $\begingroup$ And how could one, instead of overwriting, really create a new file with the new modifications? $\endgroup$
    – AJHC
    Apr 9, 2020 at 16:47

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.