{"id":6254,"library":"stream-zip","title":"Stream ZIP","description":"Stream-zip is a Python library designed to construct ZIP archives on the fly without requiring the entire archive or its constituent files to be held in memory or on disk. This makes it particularly suitable for memory-constrained environments or generating ZIP files for streaming HTTP responses in web servers. It offers both synchronous and asynchronous interfaces and is currently at version 0.0.84, with frequent minor updates.","status":"active","version":"0.0.84","language":"en","source_language":"en","source_url":"https://github.com/uktrade/stream-zip","tags":["zip","streaming","archive","compression","filesystem","memory-efficient"],"install":[{"cmd":"pip install stream-zip","lang":"bash","label":"Install stable version"}],"dependencies":[{"reason":"Required for AES-256 encrypted ZIP files.","package":"pycryptodome","optional":true},{"reason":"Required for type checking, likely related to asynchronous contexts.","package":"types-contextvars","optional":true}],"imports":[{"note":"The primary function for creating ZIP streams.","symbol":"stream_zip","correct":"from stream_zip import stream_zip"},{"note":"Constant for specifying the ZIP compression method, limited to 4GiB. Also `ZIP_64` for larger files.","symbol":"ZIP_32","correct":"from stream_zip import ZIP_32"},{"note":"The signature of `async_stream_zip` changed in v0.0.81, affecting how arguments like `chunk_size` are passed. Refer to documentation for the updated signature if using versions prior to 0.0.81.","wrong":"from stream_zip import async_stream_zip(..., chunk_size=...)","symbol":"async_stream_zip","correct":"from stream_zip import async_stream_zip"}],"quickstart":{"code":"from datetime import datetime\nfrom stat import S_IFREG\nfrom stream_zip import ZIP_32, stream_zip, ZIP_64\n\ndef generate_file_content(data):\n    yield data.encode('utf-8')\n\ndef member_files():\n    modified_at = datetime.now()\n    mode = S_IFREG | 0o600 # Regular file, owner read/write\n\n    # Example 1: Small file using ZIP_32 (default)\n    yield (\n        'my-file-1.txt',\n        modified_at,\n        mode,\n        ZIP_32,\n        generate_file_content('This is some content for file 1.')\n    )\n\n    # Example 2: Potentially larger file or for explicit 64-bit support\n    yield (\n        'my-file-2.json',\n        modified_at,\n        mode,\n        ZIP_64,\n        generate_file_content('{\"key\": \"value\", \"data\": [1, 2, 3]}')\n    )\n\n    # Example 3: An empty directory\n    yield (\n        'my-directory/',\n        modified_at,\n        S_IFREG | 0o755, # Directory permissions\n        ZIP_32,\n        ()\n    )\n\n# Stream the ZIP file chunks\nzipped_chunks = stream_zip(member_files())\n\n# In a real application, you would send these chunks directly as an HTTP response\n# or write them to a file. For this example, we'll print them.\n# You might also use io.BytesIO to collect them into a single bytes object for testing.\n\n# Example of consuming chunks (e.g., writing to a file)\n# with open('output.zip', 'wb') as f:\n#     for chunk in zipped_chunks:\n#         f.write(chunk)\n\nprint(\"Generated ZIP chunks (first few bytes of each):\")\nfor i, chunk in enumerate(zipped_chunks):\n    print(f\"Chunk {i}: {len(chunk)} bytes (starts with: {chunk[:20]})\")\n    if i > 5: # Limit output for demonstration\n        print(\"...\")\n        break","lang":"python","description":"This quickstart demonstrates how to use `stream_zip` to create a ZIP archive containing multiple files and a directory. The `stream_zip` function takes an iterable of member file definitions, where each member file is a tuple specifying its name, modification time, mode, compression method (`ZIP_32` or `ZIP_64`), and an iterable of binary content chunks. The function returns an iterable that yields chunks of the resulting ZIP file."},"warnings":[{"fix":"Consult the `stream-zip` documentation or changelog for `v0.0.81` to update the `async_stream_zip` call with the corrected signature, specifically regarding parameters like `chunk_size`.","message":"The signature for the `async_stream_zip` function changed in version `0.0.81`. If you were using the asynchronous interface, verify the updated argument order and names.","severity":"breaking","affected_versions":">=0.0.81"},{"fix":"For files potentially exceeding 4GiB (compressed or uncompressed size, or total archive size), use `ZIP_64`. Otherwise, `ZIP_32` offers broader compatibility. Be mindful of these limits for individual files and the overall archive.","message":"Client code must explicitly choose between `ZIP_32` and `ZIP_64` compression methods for each member file. The library cannot automatically determine if `ZIP_64` (for files > 4GiB) is needed during streaming, as this information is required before the compressed data is processed. Incorrect choice can lead to `ZipOverflowError` for large files.","severity":"gotcha","affected_versions":"All versions"},{"fix":"Avoid `NO_COMPRESSION_*` for large files if memory efficiency is critical. If uncompressed data must be included, consider if another streaming approach or a different file format is more suitable, or accept the memory footprint for these specific files.","message":"Using `NO_COMPRESSION_32` or `NO_COMPRESSION_64` for uncompressed files will buffer the *entire content* of those files in memory. This negates the streaming benefit for uncompressed large files, as their full size and CRC32 must be known before their data in the ZIP stream.","severity":"gotcha","affected_versions":"All versions"},{"fix":"Be aware of potential memory spikes if creating ZIPs with an extremely high count of member files, even if each individual file's content is streamed efficiently.","message":"Due to the ZIP file format specification, small bits of metadata for each member file (like names) must be placed at the end of the ZIP archive. `stream-zip` buffers this metadata in memory until it can be output. For archives with a very large number of small files, this could still lead to increased memory usage for metadata.","severity":"gotcha","affected_versions":"All versions"}],"env_vars":null,"last_verified":"2026-04-14T00:00:00.000Z","next_check":"2026-07-13T00:00:00.000Z"}