2
$\begingroup$

I'm attempting to Import datasets from a *.hdf5 file into Mathematica 12.2 in Win 7.

After looking at the syntax in the Wolfram Documentation Center and displaying the StructureGraph, I had no problem importing some of the datasets, from various levels, using

Import["file.hdf5",elem]

However, the one dataset I'm really interested with, just gives me $Failed, with no additional explanations. Suggestions?

I can look at the data in this dataset using an app called HDFView. It looks fine, a spreadsheet similar to those in the datasets that were imported.

using

 Import[file, {{"Dimensions", "DataFormat"}}]

I do notice one thing that is perhaps a problem? The format of one column is "bitfield". Using HDFView, this column includes values of either 01 or 00. Is this the problem and can it be solved?

Two of the other datasets in this files I also can't import, but at least I get a warning message:

LibraryFunction::fpexc: Numeric data containing a floating point exception (NaN or Inf) encountered.

When looking at these datasets with HDFView, they indeed contain some NaN values. Is there a way to import them?

Improve this question
$\endgroup$
4
  • 3
    $\begingroup$ Without an example file available, this question would appear to be unanswerable. Can you, please, provide one for others to work with? I think that pastebin is acceptable. $\endgroup$
    – CA Trevillian
    Mar 31, 2021 at 18:45
  • $\begingroup$ I had experienced the same issue recently when importing a h5 dataset generated by pandas; which works fine with MMA12.1 but fails in MMA12.2. I understand this dataset was produced with numpy.nan involved, so my workaround for MMA12.2 was to replace those np.nans with a numeric value, say -9999. $\endgroup$
    – sunt05
    Apr 1, 2021 at 8:54
  • $\begingroup$ I posted the file here gofile.io/d/fIY4P2 . The problematic dataset is "trajectories_data" $\endgroup$
    – Neri
    Apr 1, 2021 at 12:51
  • $\begingroup$ the NaN part is related to this thread $\endgroup$
    – sunt05
    Apr 15, 2021 at 10:52

1 Answer 1

Reset to default
3
$\begingroup$

To import datasets that contain indeterminate or infinite values one can set the following system option:

SetSystemOptions["LibraryLinkOptions" -> "TestFloatingPointExceptions" -> False]

This tells LibraryLink to skip the check for infinities and nans so it is useful when you want to import floating point data regardless of whether it contains indeterminate values or not. As a side effect, setting this option will slightly speed up import of floating point data.

Bear in mind that this is a system option so it affects all data that comes to WL through LibraryLink, not just the HDF5 format.

As for the other failing dataset, the "Bitfield" type is not supported in HDF5 Import, so it might be just that, although you should see a message. Please report a bug to Wolfram.

Improve this answer
$\endgroup$
4
  • $\begingroup$ Thanks refalc, "TestFloatingPointExceptions" -> False indeed enables me to Import the datasets with NaN values. It doesn't help with the Bitfield dataset, though. I'm reporting a bug in support. $\endgroup$
    – Neri
    Apr 1, 2021 at 11:55
  • $\begingroup$ An update: I reported a bug to the wolfram support, and they replicated the issue and reported to the developers. $\endgroup$
    – Neri
    Apr 13, 2021 at 16:10
  • $\begingroup$ Also, I tried reading the file in PyTables, since this automatically replaces bit-field with Boolean or UInt8, and save the file in another name. But still when importing the new file to Mathematica, I just get $Failed as before. So maybe it is not just the bit-field data type? $\endgroup$
    – Neri
    Apr 13, 2021 at 16:15
  • $\begingroup$ Can you upload this new file as well? $\endgroup$
    – rafalc
    Apr 14, 2021 at 18:26

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge that you have read and understand our privacy policy and code of conduct.

Not the answer you're looking for? Browse other questions tagged or ask your own question.