python-snappy

Warnings

• gotcha The `python-snappy` library is a wrapper around the Snappy C++ library. You must have the underlying `snappy` library installed on your system before `pip install python-snappy` will succeed.
  Fix: Install `libsnappy-dev` (Debian/Ubuntu), `snappy-devel` (RHEL/Fedora), or `snappy` (macOS via Homebrew) using your system's package manager before installing the Python package.
• gotcha `python-snappy` functions (like `snappy.compress` and `snappy.decompress`) only operate on `bytes` objects. Passing a `str` object will result in a `TypeError`.
  Fix: Ensure all data intended for compression or decompression is of type `bytes`. Use `my_string.encode('utf-8')` to convert strings to bytes.
• gotcha When using `StreamCompressor`, it's crucial to call `compressor.flush()` at the end of the data stream to ensure all buffered data is emitted. Failing to do so can lead to incomplete or corrupted compressed output.
  Fix: Always append the result of `compressor.flush()` to your compressed data stream after all chunks have been processed by `compressor.compress()`.
• gotcha The latest versions of `python-snappy` (0.7.0+) explicitly require Python 3.7 or newer. Older Python 3 versions (e.g., 3.6) or Python 2 are no longer supported.
  Fix: Ensure your environment is running Python 3.7 or newer. If you must use an older Python version, you might need to pin `python-snappy` to an older compatible version (e.g., `<0.7.0`).

Install

• pip install python-snappy Install `python-snappy`
• sudo apt-get install libsnappy-dev # Debian/Ubuntu sudo yum install snappy-devel # RHEL/Fedora brew install snappy # macOS with Homebrew Install system-level Snappy library (prerequisite)

Imports

• snappy

  import snappy

• compress

  snappy.compress

  Called as a function from the top-level 'snappy' module.
• decompress

  snappy.decompress

  Called as a function from the top-level 'snappy' module.
• StreamCompressor

  snappy.StreamCompressor

  Used for incremental compression of data streams.
• StreamDecompressor

  snappy.StreamDecompressor

  Used for incremental decompression of data streams.

Quickstart

This quickstart demonstrates basic one-shot compression/decompression and the use of `StreamCompressor` and `StreamDecompressor` for handling larger data in chunks. Ensure the underlying Snappy C library is installed first.

import snappy

# Basic compression and decompression
original_data = b"This is some data to be compressed using snappy!" * 100
compressed_data = snappy.compress(original_data)
decompressed_data = snappy.decompress(compressed_data)

print(f"Original size: {len(original_data)} bytes")
print(f"Compressed size: {len(compressed_data)} bytes")
print(f"Decompressed data matches original: {original_data == decompressed_data}")

# Streaming compression and decompression
compressor = snappy.StreamCompressor()
compressed_stream_chunks = []

# Simulate sending data in chunks
for chunk in [b'part one', b'part two', b'part three']:
    compressed_stream_chunks.append(compressor.compress(chunk))
compressed_stream_chunks.append(compressor.flush()) # Important to flush remaining data

full_compressed_stream = b''.join(compressed_stream_chunks)

decompressor = snappy.StreamDecompressor()
decompressed_stream_chunks = []

# Simulate receiving and decompressing data in chunks
for chunk in [full_compressed_stream[:15], full_compressed_stream[15:]]:
    decompressed_stream_chunks.append(decompressor.decompress(chunk))

final_decompressed_stream = b''.join(decompressed_stream_chunks)
print(f"Stream decompressed data: {final_decompressed_stream}")
print(f"Stream decompressed data matches original: {final_decompressed_stream == b'part onepart twopart three'}")