Brotli Compression Library

raw JSON →
1.2.0 verified Tue May 12 auth: no python install: verified quickstart: stale

Python bindings for the Brotli compression library. It offers high-performance lossless compression, often achieving better ratios than gzip, particularly for text-based data. The library's releases are typically synchronized with updates to the underlying C Brotli reference implementation.

pip install brotli
error ModuleNotFoundError: No module named '_brotli' OR ImportError: DLL load failed while importing _brotli: The specified module could not be found.
cause This error typically occurs on Windows when the C extension for brotli is not properly built or found, often due to missing Visual C++ Build Tools or an incorrect Python environment.
fix
Ensure you have the Microsoft Visual C++ Build Tools installed (available through Visual Studio Installer). Then, try uninstalling and reinstalling brotli: pip uninstall brotli followed by pip install brotli. It is also recommended to use a virtual environment.
error AttributeError: module 'brotli' has no attribute 'error'
cause This error usually arises from a conflict between the `brotli` package and `brotlipy` (or `brotlicffi`), where another library's `import brotli` statement resolves to a different or older implementation that lacks the `error` exception class.
fix
First, try updating or reinstalling brotli: pip install --upgrade brotli. If the issue persists, explicitly uninstall both brotli and brotlipy/brotlicffi and then reinstall only brotli: pip uninstall brotli brotlipy brotlicffi then pip install brotli.
error ERROR: Could not build wheels for brotli
cause This installation error indicates that pip was unable to compile the C source code for the brotli Python bindings, most commonly due to missing C/C++ compiler tools or other build dependencies.
fix
On Windows, install the Microsoft Visual C++ Build Tools. On Linux, ensure you have build essentials (e.g., sudo apt-get install build-essential or sudo yum groupinstall 'Development Tools'). Then retry pip install brotli.
error brotli.brotli.Error: Decompression error: incomplete compressed stream.
cause This error occurs when attempting to decompress data that is corrupted, truncated, or not valid Brotli compressed data.
fix
Verify the integrity and completeness of the Brotli-compressed data you are trying to decompress. Ensure the data source is reliable and that the entire compressed stream is received before attempting decompression. If fetching from a server, check server-side compression settings and content delivery.
breaking Older versions (<= 1.1.0) are vulnerable to Denial of Service (DoS) attacks via 'decompression bombs' (e.g., highly compressed zero-filled data expanding to excessive sizes), leading to memory exhaustion. Version 1.2.0 introduces `Decompressor::can_accept_more_data` and an optional `output_buffer_limit` argument to `Decompressor::process` to mitigate this.
fix Upgrade to brotli v1.2.0 or newer. For streaming decompression, actively use the `output_buffer_limit` parameter in `Decompressor.process()` and handle `brotli.error` if the limit is exceeded.
gotcha Versions of brotli prior to v1.0.9 were vulnerable to an integer overflow in the decoder when processing input chunks larger than 2GiB (CVE-2020-8927).
fix Upgrade to brotli v1.0.9 or newer.
gotcha Attempting to decompress data that was not compressed with Brotli will raise a `brotli.error` exception. Ensure the input data is valid Brotli compressed data.
fix Wrap decompression calls in a `try...except brotli.error` block to gracefully handle invalid input.
gotcha When using `brotli.Compressor` for streaming compression, it's crucial to call the `.finish()` method at the end of the data stream. Failing to do so will result in an incomplete or corrupted compressed output, as buffered data may not be flushed.
fix Always call `compressor.finish()` after feeding all data chunks to `compressor.process()` to ensure all remaining data is emitted.
gotcha When interacting with files, Brotli operates on raw bytes. Files must be opened in binary mode (`'rb'` for reading, `'wb'` for writing) to avoid `TypeError` or unexpected corruption.
fix Always open files for Brotli operations with `b` in the mode string (e.g., `open('file.br', 'wb')`).
gotcha HTTP client libraries like `requests` and `urllib3` automatically detect and decompress Brotli (and other) encodings if the `brotli` library is installed. If you attempt to manually decompress content that has already been automatically decompressed, it will result in `brotli.error`.
fix Check the `Content-Encoding` header or the documentation of your HTTP client to determine if automatic decompression is occurring before attempting manual decompression.
breaking The `finish()` method is exclusive to the `brotli.Compressor` object for flushing remaining compressed data. The `brotli.Decompressor` object does not have a `finish()` method. Attempting to call it will result in an `AttributeError`.
fix Do not call `finish()` on a `brotli.Decompressor` object. For streaming decompression, feed all compressed data chunks into `decompressor.process()` until all data is decompressed.
gotcha The `brotli.Decompressor` object does not have a `finish()` method. Unlike `brotli.Compressor`, there is no explicit 'finish' step for decompressors. All decompressed output is returned by `decompressor.process()` as input chunks are fed, or when the final `process()` call is made with remaining input.
fix Remove calls to `decompressor.finish()`. Ensure all output from `decompressor.process()` is collected during streaming decompression. When all input is processed, any remaining decompressed data will have been emitted by the last `decompressor.process()` call.
python os / libc status wheel install import disk
3.10 alpine (musl) - - 0.00s 21.5M
3.10 slim (glibc) - - 0.00s 23M
3.11 alpine (musl) - - 0.00s 23.3M
3.11 slim (glibc) - - 0.00s 25M
3.12 alpine (musl) - - 0.00s 15.2M
3.12 slim (glibc) - - 0.00s 17M
3.13 alpine (musl) - - 0.00s 14.8M
3.13 slim (glibc) - - 0.00s 17M
3.9 alpine (musl) - - 0.00s 21.0M
3.9 slim (glibc) - - 0.00s 23M

This quickstart demonstrates basic one-shot compression and decompression, as well as a simple streaming compression and decompression using `brotli.Compressor` and `brotli.Decompressor`. It also shows how to handle the `output_buffer_limit` introduced in v1.2.0 for security.

import brotli

# Example data (must be bytes)
original_data = b"This is some sample text data that we want to compress. It's repetitive and will benefit from Brotli's algorithm."

# Compress data with default quality (11, highest compression)
# quality can range from 0 (fastest) to 11 (highest compression ratio)
compressed_data = brotli.compress(original_data, quality=5)

print(f"Original size: {len(original_data)} bytes")
print(f"Compressed size: {len(compressed_data)} bytes")

# Decompress data
decompressed_data = brotli.decompress(compressed_data)

print(f"Decompressed size: {len(decompressed_data)} bytes")
print(f"Data matches original: {original_data == decompressed_data}")

# Example of streaming compression
compressor = brotli.Compressor(quality=4)
streaming_compressed = b''

for i in range(3):
    chunk = b'chunk ' + str(i).encode() + b' of data\n'
    streaming_compressed += compressor.process(chunk)
streaming_compressed += compressor.finish()

print(f"\nStreaming compressed size: {len(streaming_compressed)} bytes")

# Example of streaming decompression (with output_buffer_limit for safety)
decompressor = brotli.Decompressor()
streaming_decompressed = b''
max_output_len = 1024 # Set a reasonable limit

try:
    # process might raise brotli.error if output_buffer_limit is exceeded
    for chunk in [streaming_compressed]: # In real usage, this would be chunks of compressed data
        streaming_decompressed += decompressor.process(chunk, output_buffer_limit=max_output_len)
    streaming_decompressed += decompressor.finish()
    print(f"Streaming decompressed size: {len(streaming_decompressed)} bytes")
except brotli.error as e:
    print(f"Decompression error: {e}")
    print("Output buffer limit likely exceeded or corrupted data.")