{"id":1584,"library":"numcodecs","title":"Numcodecs","description":"Numcodecs is a Python package providing a diverse set of buffer compression and transformation codecs. It is widely used in data storage and communication applications, especially as a dependency for the Zarr N-dimensional array store. The current version is 0.16.5, with frequent patch and minor releases, typically on a monthly or bi-monthly cadence.","status":"active","version":"0.16.5","language":"en","source_language":"en","source_url":"https://github.com/zarr-developers/numcodecs","tags":["compression","serialization","data-transformation","zarr","codec","ndarray"],"install":[{"cmd":"pip install numcodecs","lang":"bash","label":"Base installation"},{"cmd":"pip install \"numcodecs[all]\"","lang":"bash","label":"Install with all optional codecs"}],"dependencies":[{"reason":"Most codecs operate on NumPy arrays; while not a strict install_requires, it's a de-facto dependency for practical use.","package":"numpy","optional":false},{"reason":"Often used in conjunction with Zarr for array compression and serialization.","package":"zarr","optional":true},{"reason":"Provides the highly optimized Blosc compression codec.","package":"blosc","optional":true},{"reason":"Provides the fast LZ4 compression codec.","package":"python-lz4","optional":true},{"reason":"Provides the Snappy compression codec.","package":"python-snappy","optional":true},{"reason":"Provides the Zstd compression codec.","package":"python-zstd","optional":true}],"imports":[{"symbol":"numcodecs","correct":"import numcodecs"},{"note":"Specific codecs like Blosc, Zstd, GZip reside in their own submodules, not directly under the top-level numcodecs namespace.","wrong":"from numcodecs import Blosc","symbol":"Blosc","correct":"from numcodecs.blosc import Blosc"},{"note":"Specific codecs like Blosc, Zstd, GZip reside in their own submodules, not directly under the top-level numcodecs namespace.","wrong":"from numcodecs import Zstd","symbol":"Zstd","correct":"from numcodecs.zstd import Zstd"},{"note":"Specific codecs like Blosc, Zstd, GZip reside in their own submodules, not directly under the top-level numcodecs namespace.","wrong":"from numcodecs import GZip","symbol":"GZip","correct":"from numcodecs.gzip import GZip"}],"quickstart":{"code":"import numpy as np\nimport numcodecs\n\n# Define some data to encode\ndata = np.arange(10000, dtype='i4').reshape(100, 100)\n\n# Choose a codec. Blosc is common, but requires 'pip install \"numcodecs[blosc]\"'.\n# If Blosc isn't installed, GZip is a good fallback.\ntry:\n    codec = numcodecs.blosc.Blosc(cname='lz4', clevel=5, shuffle=numcodecs.blosc.SHUFFLE)\n    print(\"Using Blosc codec.\")\nexcept ImportError:\n    print(\"Blosc not installed. Falling back to GZip codec.\")\n    codec = numcodecs.gzip.GZip(level=5)\n\n# Encode the data\nencoded_data = codec.encode(data.tobytes())\nprint(f\"Original data shape: {data.shape}, dtype: {data.dtype}\")\nprint(f\"Original bytes: {data.nbytes}\")\nprint(f\"Encoded bytes: {len(encoded_data)}\")\n\n# Decode the data\ndecoded_bytes = codec.decode(encoded_data)\n\n# Reconstruct the numpy array\ndecoded_data = np.frombuffer(decoded_bytes, dtype=data.dtype).reshape(data.shape)\n\n# Verify that the decoded data matches the original\nassert np.array_equal(data, decoded_data)\nprint(\"Data successfully encoded and decoded!\")\n\n# You can also register codecs globally for retrieval by ID\nnumcodecs.register_codec(codec)\nretrieved_codec = numcodecs.get_codec({'id': codec.codec_id, **codec.get_config()})\nassert retrieved_codec.codec_id == codec.codec_id\nprint(f\"Codec '{retrieved_codec.codec_id}' registered and retrieved successfully.\")","lang":"python","description":"This quickstart demonstrates how to instantiate a codec (Blosc or GZip as a fallback), encode a NumPy array into bytes, and then decode it back. It also shows how to globally register and retrieve codecs, which is often used in frameworks like Zarr."},"warnings":[{"fix":"Upgrade your Python environment to 3.11 or later, or pin numcodecs to an older compatible version (e.g., `<0.16.0`).","message":"Numcodecs versions 0.16.x and higher require Python 3.11 or newer. Installations on older Python versions will fail.","severity":"breaking","affected_versions":">=0.16.0"},{"fix":"Install numcodecs with the required extras, for example, `pip install \"numcodecs[blosc]\"` for Blosc, or `pip install \"numcodecs[all]\"` to include all optional codecs.","message":"Many high-performance codecs (e.g., Blosc, Zstd, LZ4, Snappy) are implemented as optional dependencies. If you try to use them without their respective packages installed, an `ImportError` or `ValueError` will occur.","severity":"gotcha","affected_versions":"All"},{"fix":"Ensure you are using a numcodecs version compatible with your Zarr-Python version. Refer to the Zarr-Python documentation and numcodecs changelog for specific compatibility matrices. Update numcodecs to the latest stable version for the best Zarr v3 compatibility.","message":"Compatibility with Zarr v3 has evolved across recent numcodecs versions. Changes in Zarr3 configuration serialization (v0.15.1) and handling for Zarr 3.1.0 (v0.16.2) may affect users relying on specific Zarr3 features or metadata structures.","severity":"breaking","affected_versions":">=0.15.0"},{"fix":"Upgrade numcodecs to version 0.16.3 or newer if you are using Zstd compression on a 32-bit system.","message":"Prior to v0.16.3, there was a known issue with Zstd decompression leading to negative size errors on 32-bit platforms.","severity":"gotcha","affected_versions":"<0.16.3"}],"env_vars":null,"last_verified":"2026-04-09T00:00:00.000Z","next_check":"2026-07-08T00:00:00.000Z"}