1
$\begingroup$

Assume I export some data into a file like:

data = {{1, 2, 3}, {4, 5, 6}};
Export["test.h5",data,{"Datasets","/h1"}];

How can I append {7, 8, 9} to the "test.h5" (by directly writing in the test.h5) such that the results for

Import["test.h5", data, {"Datasets", "/h1"}]

will be

{{1, 2, 3}, {4, 5, 6}, {7, 8, 9}}

I would prefer to add it at the end of /h1.

$\endgroup$
  • 1
    $\begingroup$ this is not possible with mathematica, except obviously you can import, append the data and re export. $\endgroup$ – george2079 Feb 20 '17 at 14:29
  • $\begingroup$ george2079 is right: what you want to do is not possible. There is the option "Append" for Exporting HDF5 files with which you can add additional datasets to an existing file, but it is not possible to extend an existing dataset. $\endgroup$ – Albert Retey Feb 20 '17 at 16:33
  • $\begingroup$ @AlbertRetey .. thanks I didn't even know about Append, but.. does it actually work? Everything I try it just overwrites the file. $\endgroup$ – george2079 Feb 20 '17 at 18:16
  • 2
    $\begingroup$ @george2079: Which version do you use, this works for me with 9, 10 and 11: Export[filename, {{1, 2, 3}}, {"Datasets", "one"}] then Export[filename, {{1, 2, 3}}, {"Datasets", "two"}, "Append" -> True] and Import[filename] then shows that the file now has two datasets as expected. Does that not work for you? $\endgroup$ – Albert Retey Feb 20 '17 at 18:58
  • $\begingroup$ You need to use "Append", not Append. I don't know why, and I don't know why it doesn't give an error if you use the latter. This is in 10.4.1 on Mac OS. $\endgroup$ – Ian Hinder Mar 17 '17 at 19:46
4
$\begingroup$

Extending an existing dataset is not possible with the standard Mathematica Export, at least not with any version up to 11.0.1. What does work is to add additional datasets one by one like this:

filename = FileNameJoin[{$HomeDirectory, "Desktop", "tst.h5"}]

Export[filename, {{1, 2, 3}}, {"Datasets", "one"}]
Export[filename, {{1, 2, 3}}, {"Datasets", "two"}, "Append" -> True]

using:

Import[filename]

you can verify that there are now two datasets in the file. Of course that is not the same thing, but the best that currently is possible and probably good enough for some use cases. If you need more, there are some libraries which might be possible to do what you need. Alternatively you could write your own function to access HDF5 files via e.g. LibraryLink. If you search for HDF5 on this site you will find several questions and answer to guide you either way.

| improve this answer | |
$\endgroup$
  • $\begingroup$ Thanks a lot guys! I found the syntax "BinaryWrite" which solves the problem partially by combining with OpenAppend. Unfortunately it works only for integers 0<= x <=255. $\endgroup$ – Hesam SoltanpanahiSarabi Feb 20 '17 at 19:57
  • $\begingroup$ @HesamSoltanpanahiSarabi: I don't understand what you are trying to say: are you appending data using BinaryWrite to an existing hdf5 file? I would not expect this to work if not for very specific cases by accident. It is certainly not the way to solve that problem... $\endgroup$ – Albert Retey Feb 20 '17 at 21:18
  • $\begingroup$ This is what I did. But as you mentioned it works only for some specific cases. $\endgroup$ – Hesam SoltanpanahiSarabi Feb 21 '17 at 7:42
1
$\begingroup$

It seems like it has changed in Version 12.0.0.0

"Append" -> True

is now implemented as

OverwriteTarget->"Append"

| improve this answer | |
$\endgroup$
  • $\begingroup$ So does the new / modified functionality permit modification of existing datasets, or only the appendment of new ones ? $\endgroup$ – High Performance Mark Sep 4 '19 at 10:45
  • $\begingroup$ As far as I know it's just for adding a new dataset. Modification of existing datasets may be done with: Overwrite->"Append" , "AppendMode"->"Overwrite". But the dataset must have same dimensions. $\endgroup$ – Thunderbird Sep 4 '19 at 12:26
  • $\begingroup$ And how could one, instead of overwriting, really create a new file with the new modifications? $\endgroup$ – AJHC Apr 9 at 16:47

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.