{"id":531,"library":"lz4","title":"LZ4 Bindings for Python","description":"The lz4 library provides Python bindings for the high-performance LZ4 compression algorithm by Yann Collet. It supports both the frame and block formats, with the frame format being recommended for most applications due to its interoperability. The library is actively maintained with frequent releases, currently at version 4.4.5, and offers a Pythonic API that can serve as a drop-in alternative to standard library compression modules like `zlib` or `gzip`.","status":"active","version":"4.4.5","language":"python","source_language":"en","source_url":"https://github.com/python-lz4/python-lz4","tags":["compression","lz4","performance","binary","data-processing"],"install":[{"cmd":"pip install lz4","lang":"bash","label":"Install latest version"}],"dependencies":[{"reason":"Required Python interpreter version","package":"python","version":">=3.9","optional":false}],"imports":[{"note":"Recommended for most applications due to standard frame format and interoperability.","symbol":"lz4.frame","correct":"import lz4.frame"},{"note":"For raw block compression/decompression; less interoperable than lz4.frame.","symbol":"lz4.block","correct":"import lz4.block"}],"quickstart":{"code":"import lz4.frame\nimport os\n\noriginal_data = os.urandom(1024 * 1024) # 1 MB of random bytes\n\n# Compress data using the LZ4 frame format\ncompressed_data = lz4.frame.compress(original_data)\n\n# Decompress data\ndecompressed_data = lz4.frame.decompress(compressed_data)\n\n# Verify integrity\nassert original_data == decompressed_data\nprint(f\"Original size: {len(original_data)} bytes\")\nprint(f\"Compressed size: {len(compressed_data)} bytes\")\nprint(\"Data compressed and decompressed successfully.\")","lang":"python","description":"This quickstart demonstrates basic compression and decompression using the recommended `lz4.frame` module. It generates random bytes, compresses them into an LZ4 frame, then decompresses the data and verifies its integrity. Note that input data must be in bytes format."},"warnings":[{"fix":"Ensure data is of type `bytes` before passing to compression functions.","message":"Input data for compression functions (`lz4.frame.compress`, `lz4.block.compress`) must be `bytes`. Passing a standard Python string directly will result in a `TypeError`. Always encode strings (e.g., `my_string.encode('utf-8')`) before compression.","severity":"gotcha","affected_versions":"All versions"},{"fix":"For `lz4.block.decompress`, either provide the exact `uncompressed_size` or implement a loop with `max_size` and error handling for `LZ4BlockError`.","message":"When using `lz4.block.decompress`, if the `uncompressed_size` parameter is not provided or is too small, a `lz4.block.LZ4BlockError` may be raised. It's often impossible to distinguish between insufficient buffer size and corrupt input data in this case. Consider setting an absolute upper bound (`max_size`) for memory allocation during decompression.","severity":"gotcha","affected_versions":"All versions (especially when `uncompressed_size` is unknown or underestimated)"},{"fix":"Prioritize `import lz4.frame` and its `compress`/`decompress` functions for broad compatibility unless specific block-level control is required.","message":"The `lz4.frame` module is generally recommended for most applications as it defines a standard container format that ensures interoperability with other LZ4 implementations and language bindings. Using `lz4.block` for direct block compression without a container can lead to compatibility issues when exchanging data with other systems.","severity":"gotcha","affected_versions":"All versions"},{"fix":"Use `lz4.frame` for stream-like or file-based compression, which offers robust and maintained functionality for large data handling.","message":"The `lz4.stream` sub-package is considered experimental, unmaintained, and not built into distributed wheels. It requires manual compilation from source with specific environment variables. Avoid using it in production environments.","severity":"deprecated","affected_versions":"All versions (status since at least 4.3.2)"},{"fix":"Benchmark and explicitly set `compression_level` and `block_size` based on your application's specific performance and compression ratio requirements.","message":"The `compression_level` and `block_size` parameters significantly impact the trade-off between compression ratio and speed. While `lz4` is known for speed, higher `compression_level` values (e.g., 5 or 9) will yield smaller outputs but take longer. Default values might change or differ from other LZ4 implementations, potentially causing unexpected performance shifts.","severity":"gotcha","affected_versions":"All versions"},{"fix":"Ensure a C compiler is available in your build environment. For Alpine-based Python images (like `python:3.13-alpine`), this usually means installing the `build-base` package (e.g., `apk add build-base`) before running `pip install lz4`.","message":"Installing the `lz4` library requires a C compiler (e.g., `gcc`) to build its C extensions. This error typically occurs in minimal environments where development tools are not pre-installed.","severity":"breaking","affected_versions":"All versions (where C extensions need compilation)"}],"env_vars":null,"last_verified":"2026-05-12T14:42:27.883Z","next_check":"2026-06-26T00:00:00.000Z","problems":[{"fix":"Ensure the lz4 library is correctly installed in your environment: `pip install lz4`","cause":"The 'lz4' Python package is either not installed, or there's an issue with the Python environment preventing the 'lz4.frame' submodule (or 'lz4.block' or the top-level 'lz4' module) from being found.","error":"ModuleNotFoundError: No module named 'lz4.frame'"},{"fix":"Verify the integrity of the compressed data. If using `lz4.block.decompress`, ensure the `uncompressed_size` argument is accurate, or use `lz4.frame.decompress` if the data was compressed using the frame format, which usually embeds size information. Example of handling unknown size: `while True: try: decompressed = lz4.block.decompress(compressed, uncompressed_size=usize) break except lz4.block.LZ4BlockError: usize *= 2`","cause":"This error typically occurs during decompression when the input data is corrupted, the provided 'uncompressed_size' is incorrect or too small, or there's a mismatch between the compression and decompression methods (e.g., block vs. frame, or data compressed by a different LZ4 implementation).","error":"lz4.block.LZ4BlockError: Decompression failed: corrupt input or insufficient space in destination buffer. Error code: XXX"},{"fix":"Encode the Python string to bytes before compressing and decode the resulting bytes back to a string after decompressing. Example: `original_string.encode('utf-8')` before compression, and `decompressed_bytes.decode('utf-8')` after decompression.","cause":"The LZ4 compression and decompression functions operate on byte sequences, not standard Python strings. This error occurs when a plain string is passed as input without explicit encoding.","error":"TypeError: a bytes-like object is required, not 'str'"},{"fix":"Install the missing 'deprecation' package: `pip install deprecation`. Alternatively, upgrade 'lz4' to a newer version which might no longer have this dependency or handles it more robustly: `pip install --upgrade lz4`.","cause":"Older versions of the 'lz4' library might have a dependency on the 'deprecation' package which is not installed in the current Python environment.","error":"ModuleNotFoundError: No module named 'deprecation'"}],"ecosystem":"pypi","meta_description":null,"install_score":30,"install_tag":"stale","quickstart_score":30,"quickstart_tag":"draft","pypi_latest":null,"install_checks":{"last_tested":"2026-05-12","tag":"stale","tag_description":"widespread failures or data too old to trust","results":[{"runtime":"python:3.10-alpine","python_version":"3.10","os_libc":"alpine (musl)","variant":"default","exit_code":1,"wheel_type":null,"failure_reason":null,"install_time_s":null,"import_time_s":null,"mem_mb":null,"disk_size":null},{"runtime":"python:3.10-slim","python_version":"3.10","os_libc":"slim (glibc)","variant":"default","exit_code":0,"wheel_type":null,"failure_reason":null,"install_time_s":null,"import_time_s":0,"mem_mb":0.1,"disk_size":"24M"},{"runtime":"python:3.11-alpine","python_version":"3.11","os_libc":"alpine (musl)","variant":"default","exit_code":1,"wheel_type":null,"failure_reason":null,"install_time_s":null,"import_time_s":null,"mem_mb":null,"disk_size":null},{"runtime":"python:3.11-slim","python_version":"3.11","os_libc":"slim (glibc)","variant":"default","exit_code":0,"wheel_type":null,"failure_reason":null,"install_time_s":null,"import_time_s":0,"mem_mb":0.1,"disk_size":"26M"},{"runtime":"python:3.12-alpine","python_version":"3.12","os_libc":"alpine (musl)","variant":"default","exit_code":1,"wheel_type":null,"failure_reason":null,"install_time_s":null,"import_time_s":null,"mem_mb":null,"disk_size":null},{"runtime":"python:3.12-slim","python_version":"3.12","os_libc":"slim (glibc)","variant":"default","exit_code":0,"wheel_type":null,"failure_reason":null,"install_time_s":null,"import_time_s":0,"mem_mb":0.1,"disk_size":"18M"},{"runtime":"python:3.13-alpine","python_version":"3.13","os_libc":"alpine (musl)","variant":"default","exit_code":1,"wheel_type":null,"failure_reason":null,"install_time_s":null,"import_time_s":null,"mem_mb":null,"disk_size":null},{"runtime":"python:3.13-slim","python_version":"3.13","os_libc":"slim (glibc)","variant":"default","exit_code":0,"wheel_type":null,"failure_reason":null,"install_time_s":null,"import_time_s":0,"mem_mb":0.1,"disk_size":"17M"},{"runtime":"python:3.9-alpine","python_version":"3.9","os_libc":"alpine (musl)","variant":"default","exit_code":1,"wheel_type":null,"failure_reason":null,"install_time_s":null,"import_time_s":null,"mem_mb":null,"disk_size":null},{"runtime":"python:3.9-slim","python_version":"3.9","os_libc":"slim (glibc)","variant":"default","exit_code":0,"wheel_type":null,"failure_reason":null,"install_time_s":null,"import_time_s":0.01,"mem_mb":0.1,"disk_size":"23M"}]},"quickstart_checks":{"last_tested":"2026-04-23","tag":"draft","tag_description":"notable failures across runtimes","results":[{"runtime":"python:3.10-alpine","exit_code":1},{"runtime":"python:3.10-slim","exit_code":0},{"runtime":"python:3.11-alpine","exit_code":1},{"runtime":"python:3.11-slim","exit_code":0},{"runtime":"python:3.12-alpine","exit_code":1},{"runtime":"python:3.12-slim","exit_code":0},{"runtime":"python:3.13-alpine","exit_code":1},{"runtime":"python:3.13-slim","exit_code":0},{"runtime":"python:3.9-alpine","exit_code":1},{"runtime":"python:3.9-slim","exit_code":0}]}}