LZ4 Bindings for Python

Warnings

• gotcha Input data for compression functions (`lz4.frame.compress`, `lz4.block.compress`) must be `bytes`. Passing a standard Python string directly will result in a `TypeError`. Always encode strings (e.g., `my_string.encode('utf-8')`) before compression.
  Fix: Ensure data is of type `bytes` before passing to compression functions.
• gotcha When using `lz4.block.decompress`, if the `uncompressed_size` parameter is not provided or is too small, a `lz4.block.LZ4BlockError` may be raised. It's often impossible to distinguish between insufficient buffer size and corrupt input data in this case. Consider setting an absolute upper bound (`max_size`) for memory allocation during decompression.
  Fix: For `lz4.block.decompress`, either provide the exact `uncompressed_size` or implement a loop with `max_size` and error handling for `LZ4BlockError`.
• gotcha The `lz4.frame` module is generally recommended for most applications as it defines a standard container format that ensures interoperability with other LZ4 implementations and language bindings. Using `lz4.block` for direct block compression without a container can lead to compatibility issues when exchanging data with other systems.
  Fix: Prioritize `import lz4.frame` and its `compress`/`decompress` functions for broad compatibility unless specific block-level control is required.
• deprecated The `lz4.stream` sub-package is considered experimental, unmaintained, and not built into distributed wheels. It requires manual compilation from source with specific environment variables. Avoid using it in production environments.
  Fix: Use `lz4.frame` for stream-like or file-based compression, which offers robust and maintained functionality for large data handling.
• gotcha The `compression_level` and `block_size` parameters significantly impact the trade-off between compression ratio and speed. While `lz4` is known for speed, higher `compression_level` values (e.g., 5 or 9) will yield smaller outputs but take longer. Default values might change or differ from other LZ4 implementations, potentially causing unexpected performance shifts.
  Fix: Benchmark and explicitly set `compression_level` and `block_size` based on your application's specific performance and compression ratio requirements.

Install

• pip install lz4 Install latest version

Imports

• lz4.frame

  import lz4.frame

  Recommended for most applications due to standard frame format and interoperability.
• lz4.block

  import lz4.block

  For raw block compression/decompression; less interoperable than lz4.frame.

Quickstart

This quickstart demonstrates basic compression and decompression using the recommended `lz4.frame` module. It generates random bytes, compresses them into an LZ4 frame, then decompresses the data and verifies its integrity. Note that input data must be in bytes format.

import lz4.frame
import os

original_data = os.urandom(1024 * 1024) # 1 MB of random bytes

# Compress data using the LZ4 frame format
compressed_data = lz4.frame.compress(original_data)

# Decompress data
decompressed_data = lz4.frame.decompress(compressed_data)

# Verify integrity
assert original_data == decompressed_data
print(f"Original size: {len(original_data)} bytes")
print(f"Compressed size: {len(compressed_data)} bytes")
print("Data compressed and decompressed successfully.")