hdf5plugin

Warnings

• breaking Version 6.0.0 requires Python >= 3.9. Prior versions supported older Python versions (e.g., v5.0.0 required >=3.8, v4.0.0 required >=3.7).
  Fix: Ensure your Python environment is 3.9 or newer. Downgrade hdf5plugin if an older Python version is strictly required.
• breaking Version 5.0.0 requires h5py >= 3.0.0. This was a significant bump from previous versions.
  Fix: Upgrade h5py to version 3.0.0 or higher. You may also need to upgrade Python if h5py's requirements for newer versions conflict with your setup.
• breaking Deprecated constants `hdf5plugin.config`, `hdf5plugin.date`, `hdf5plugin.hexversion`, and `hdf5plugin.strictversion` were removed in version 5.0.0.
  Fix: Replace usage of these constants. For `hdf5plugin.config`, use `hdf5plugin.get_config()` instead. Other removed constants do not have direct replacements and should be removed if no longer necessary.
• deprecated The SZ filter has been deprecated in version 6.0.0. While it might still function, its continued support is not guaranteed.
  Fix: Migrate away from using the SZ filter to other supported filters like SZ3, Zfp, or Blosc for future compatibility and better performance/features.
• gotcha Data compressed with newer versions of the H5Z-ZFP filter (e.g., v1.1.0 in hdf5plugin v4.0.0) might not be readable by older versions of the filter, though newer versions can read older data.
  Fix: Ensure that the hdf5plugin version used for reading ZFP-compressed data is at least as new as, or newer than, the version used for writing.
• gotcha Some advanced Blosc2 compression codecs (e.g., blosc2-grok, blosc2-openhtj2k) might require additional, separately installed plugins to be present in the HDF5_PLUGIN_PATH environment variable for decompression to work correctly. hdf5plugin itself may not bundle all possible Blosc2 sub-filters.
  Fix: If encountering errors with specific Blosc2 codecs, consult the Blosc2 documentation for required external plugins and ensure they are installed and discoverable via the HDF5_PLUGIN_PATH environment variable.
• gotcha Poorly chosen HDF5 chunking strategies (e.g., very large/small chunks, or chunk shapes misaligned with common access patterns) can significantly degrade performance, even with efficient compression filters. This is a general HDF5/h5py concern but applies directly to hdf5plugin usage.
  Fix: Design chunking to align with expected read/write access patterns. Refer to HDF5 and h5py documentation on chunking best practices for optimal performance with compressed datasets.

Install

• pip install hdf5plugin Standard installation
• pip install hdf5plugin --no-binary hdf5plugin Install from source (enables optimizations)

Imports

• hdf5plugin

  import hdf5plugin

  Importing the package automatically registers the supported HDF5 compression filters with the HDF5 library used by h5py, making them available for use. No explicit filter registration calls are typically needed.
• h5py

  import h5py

  Required to interact with HDF5 files and datasets.

Quickstart

This quickstart demonstrates how to write and read a dataset compressed using one of the filters provided by hdf5plugin (LZ4 in this case). The `import hdf5plugin` statement is crucial as it registers the filters with h5py.

import numpy
import h5py
import hdf5plugin

# Create a dummy dataset
data_to_write = numpy.arange(100, dtype='i4')

# Write compressed data to an HDF5 file using an hdf5plugin filter (e.g., LZ4)
file_name = 'test_compressed.h5'
with h5py.File(file_name, 'w') as f:
    dset = f.create_dataset('data', data=data_to_write, compression=hdf5plugin.LZ4())
    print(f"Dataset 'data' written to {file_name} with LZ4 compression.")

# Read the compressed data back
with h5py.File(file_name, 'r') as f:
    read_data = f['data'][()]
    print(f"Data read successfully: {read_data}")
    assert numpy.array_equal(data_to_write, read_data)
print("Original and read data match.")