cramjam

Warnings

• gotcha When integrating `cramjam`'s LZ4 implementation or migrating from other LZ4 libraries, be aware that the default compression level in `cramjam`'s LZ4 might differ. For example, a change from level 0 to 9 in a dependent library using `cramjam`'s LZ4 was observed to cause significant performance degradation (over 100% latency increase) if not explicitly set. Always verify the `compression_level` if performance is critical.
  Fix: Explicitly set the `compression_level` parameter when calling `cramjam.lz4.compress` or ensure any integrating library configures it appropriately. The default behavior may vary and impact performance.
• gotcha For significant performance improvements (1.5-3x speedup), especially when decompressing or compressing to a standard `bytes` or `bytearray` object, provide the `output_len` argument if the exact output length is known beforehand. This allows for single buffer allocation. This optimization is less relevant when using `cramjam.Buffer` or `cramjam.File` objects.
  Fix: Pass the `output_len=some_integer` argument to `compress` or `decompress` functions when the final size is predictable.
• gotcha Some compression algorithms (e.g., Blosc2, ISA-L backends like igzip, ideflate, izlib) are considered experimental. These typically require building `cramjam` from source with specific feature flags enabled. They are not available out-of-the-box with a standard `pip install`.
  Fix: Refer to the official `cramjam` GitHub repository or documentation for instructions on building from source with experimental features if you need to use them.

Install

• pip install --upgrade cramjam Install latest stable version

Imports

• cramjam

  import cramjam

  General import for accessing all compression variants (e.g., cramjam.snappy, cramjam.brotli).
• snappy

  from cramjam import snappy

  Import a specific compression algorithm directly.
• Buffer

  from cramjam import Buffer

  Used for efficient in-memory buffer operations, particularly with de/compress_into methods.

Quickstart

This quickstart demonstrates basic compression and decompression using Snappy and Brotli. It also includes an example of using `cramjam.Buffer` for more efficient in-place compression operations, which is beneficial when dealing with `bytes`, `bytearray`, or `numpy.array` objects.

import cramjam

original_data = b"This is some data to compress using cramjam!"

# Compress data using Snappy
compressed_data = cramjam.snappy.compress(original_data)
print(f"Original size: {len(original_data)} bytes")
print(f"Compressed size (Snappy): {len(compressed_data)} bytes")

# Decompress data
decompressed_data = cramjam.snappy.decompress(compressed_data)
print(f"Decompressed data matches original: {original_data == decompressed_data}")

# Example with Brotli
compressed_brotli = cramjam.brotli.compress(original_data)
print(f"Compressed size (Brotli): {len(compressed_brotli)} bytes")
decompressed_brotli = cramjam.brotli.decompress(compressed_brotli)
print(f"Decompressed Brotli matches original: {original_data == decompressed_brotli}")

# Using cramjam.Buffer for in-place operations
from cramjam import snappy, Buffer
import numpy as np

data_np = np.frombuffer(b'some bytes here for buffer', dtype=np.uint8)
output_buffer = Buffer()
snappy.compress_into(data_np, output_buffer)

output_buffer.seek(0) # Reset buffer position for reading
decompressed_buffer_data = snappy.decompress(output_buffer)
print(f"Buffer decompressed data matches original: {bytes(data_np) == bytes(decompressed_buffer_data)}")