{"id":5955,"library":"hdf5plugin","title":"hdf5plugin","description":"hdf5plugin is a Python library that provides additional HDF5 compression filters for use with h5py, enabling reading and writing of compressed datasets with various algorithms like Blosc, Bitshuffle, LZ4, Zstd, and more. It is actively maintained with frequent releases, currently at version 6.0.0, and often updates its embedded compression libraries and introduces new filters.","status":"active","version":"6.0.0","language":"en","source_language":"en","source_url":"https://github.com/silx-kit/hdf5plugin","tags":["HDF5","compression","h5py","data science","scientific computing","filter"],"install":[{"cmd":"pip install hdf5plugin","lang":"bash","label":"Standard installation"},{"cmd":"pip install hdf5plugin --no-binary hdf5plugin","lang":"bash","label":"Install from source (enables optimizations)"}],"dependencies":[{"reason":"hdf5plugin registers filters with h5py; it's a core dependency for its functionality.","package":"h5py","optional":false},{"reason":"Commonly used for data manipulation with h5py, especially in examples.","package":"numpy","optional":true}],"imports":[{"note":"Importing the package automatically registers the supported HDF5 compression filters with the HDF5 library used by h5py, making them available for use. No explicit filter registration calls are typically needed.","symbol":"hdf5plugin","correct":"import hdf5plugin"},{"note":"Required to interact with HDF5 files and datasets.","symbol":"h5py","correct":"import h5py"}],"quickstart":{"code":"import numpy\nimport h5py\nimport hdf5plugin\n\n# Create a dummy dataset\ndata_to_write = numpy.arange(100, dtype='i4')\n\n# Write compressed data to an HDF5 file using an hdf5plugin filter (e.g., LZ4)\nfile_name = 'test_compressed.h5'\nwith h5py.File(file_name, 'w') as f:\n    dset = f.create_dataset('data', data=data_to_write, compression=hdf5plugin.LZ4())\n    print(f\"Dataset 'data' written to {file_name} with LZ4 compression.\")\n\n# Read the compressed data back\nwith h5py.File(file_name, 'r') as f:\n    read_data = f['data'][()]\n    print(f\"Data read successfully: {read_data}\")\n    assert numpy.array_equal(data_to_write, read_data)\nprint(\"Original and read data match.\")","lang":"python","description":"This quickstart demonstrates how to write and read a dataset compressed using one of the filters provided by hdf5plugin (LZ4 in this case). The `import hdf5plugin` statement is crucial as it registers the filters with h5py."},"warnings":[{"fix":"Ensure your Python environment is 3.9 or newer. Downgrade hdf5plugin if an older Python version is strictly required.","message":"Version 6.0.0 requires Python >= 3.9. Prior versions supported older Python versions (e.g., v5.0.0 required >=3.8, v4.0.0 required >=3.7).","severity":"breaking","affected_versions":">=6.0.0"},{"fix":"Upgrade h5py to version 3.0.0 or higher. You may also need to upgrade Python if h5py's requirements for newer versions conflict with your setup.","message":"Version 5.0.0 requires h5py >= 3.0.0. This was a significant bump from previous versions.","severity":"breaking","affected_versions":">=5.0.0"},{"fix":"Replace usage of these constants. For `hdf5plugin.config`, use `hdf5plugin.get_config()` instead. Other removed constants do not have direct replacements and should be removed if no longer necessary.","message":"Deprecated constants `hdf5plugin.config`, `hdf5plugin.date`, `hdf5plugin.hexversion`, and `hdf5plugin.strictversion` were removed in version 5.0.0.","severity":"breaking","affected_versions":">=5.0.0"},{"fix":"Migrate away from using the SZ filter to other supported filters like SZ3, Zfp, or Blosc for future compatibility and better performance/features.","message":"The SZ filter has been deprecated in version 6.0.0. While it might still function, its continued support is not guaranteed.","severity":"deprecated","affected_versions":">=6.0.0"},{"fix":"Ensure that the hdf5plugin version used for reading ZFP-compressed data is at least as new as, or newer than, the version used for writing.","message":"Data compressed with newer versions of the H5Z-ZFP filter (e.g., v1.1.0 in hdf5plugin v4.0.0) might not be readable by older versions of the filter, though newer versions can read older data.","severity":"gotcha","affected_versions":">=4.0.0"},{"fix":"If encountering errors with specific Blosc2 codecs, consult the Blosc2 documentation for required external plugins and ensure they are installed and discoverable via the HDF5_PLUGIN_PATH environment variable.","message":"Some advanced Blosc2 compression codecs (e.g., blosc2-grok, blosc2-openhtj2k) might require additional, separately installed plugins to be present in the HDF5_PLUGIN_PATH environment variable for decompression to work correctly. hdf5plugin itself may not bundle all possible Blosc2 sub-filters.","severity":"gotcha","affected_versions":"All"},{"fix":"Design chunking to align with expected read/write access patterns. Refer to HDF5 and h5py documentation on chunking best practices for optimal performance with compressed datasets.","message":"Poorly chosen HDF5 chunking strategies (e.g., very large/small chunks, or chunk shapes misaligned with common access patterns) can significantly degrade performance, even with efficient compression filters. This is a general HDF5/h5py concern but applies directly to hdf5plugin usage.","severity":"gotcha","affected_versions":"All"}],"env_vars":null,"last_verified":"2026-04-14T00:00:00.000Z","next_check":"2026-07-13T00:00:00.000Z"}