{"id":7107,"library":"conda-package-streaming","title":"Conda Package Streaming","description":"An efficient library to read from new and old format .conda and .tar.bz2 conda packages. It enables downloading conda metadata from packages without transferring the entire file and getting metadata from local .tar.bz2 packages without reading entire files. The library, currently at version 0.12.0, uses enhanced pip lazy_wheel for `.conda` files and `tarfile.open` for `.tar.bz2` to stream data efficiently. It maintains a regular release cadence, with major updates roughly every few months.","status":"active","version":"0.12.0","language":"en","source_language":"en","source_url":"https://github.com/conda/conda-package-streaming","tags":["conda","package management","streaming","metadata","tar.bz2",".conda"],"install":[{"cmd":"pip install conda-package-streaming","lang":"bash","label":"Pip"},{"cmd":"conda install conda-package-streaming -c conda-forge","lang":"bash","label":"Conda (Conda-Forge channel)"}],"dependencies":[{"reason":"Required runtime environment.","package":"python","version":">=3.9"},{"reason":"Required for HTTP requests to fetch remote packages.","package":"requests","optional":false},{"reason":"Required for handling Zstandard compressed streams (used in .conda packages).","package":"zstandard","version":">=0.15","optional":false},{"reason":"Optional dependency for S3 streaming functionality.","package":"boto3","optional":true}],"imports":[{"note":"The primary streaming functions like `stream_conda_info` are nested within submodules like `url`, `s3`, or directly under `package_streaming` for local files, not directly under the top-level package.","wrong":"from conda_package_streaming import stream_conda_info","symbol":"stream_conda_info","correct":"from conda_package_streaming.url import stream_conda_info"},{"note":"Use the `s3` submodule for streaming directly from S3 buckets, requiring `boto3` to be installed.","symbol":"stream_conda_info (for S3)","correct":"from conda_package_streaming.s3 import stream_conda_info"},{"note":"For local file paths, `stream_conda_info` is accessed via the `package_streaming` submodule.","symbol":"stream_conda_info (for local files)","correct":"from conda_package_streaming import package_streaming\n# then use package_streaming.stream_conda_info(...)"},{"note":"Provides a file-like object from a URL, used for more granular control over streaming.","symbol":"conda_reader_for_url","correct":"from conda_package_streaming.url import conda_reader_for_url"}],"quickstart":{"code":"import json\nfrom conda_package_streaming.url import stream_conda_info\n\n# Replace with a valid .conda or .tar.bz2 URL\n# For example: url = 'https://repo.anaconda.com/pkgs/main/linux-64/python-3.9.7-h62f7035_1.conda'\nurl = 'https://repo.anaconda.com/pkgs/main/linux-64/zlib-1.2.13-h5eee18b_0.conda'\n\nprint(f\"Streaming info from: {url}\")\n\ntry:\n    found_index_json = False\n    for tar, member in stream_conda_info(url):\n        if member.name == 'info/index.json':\n            index_json = json.load(tar.extractfile(member))\n            print(\"\\n--- Found info/index.json ---\")\n            print(json.dumps(index_json, indent=2))\n            found_index_json = True\n            break # Stop once index.json is found\n    \n    if not found_index_json:\n        print(\"\\n--- info/index.json not found in package ---\")\n\nexcept Exception as e:\n    print(f\"An error occurred: {e}\")","lang":"python","description":"This quickstart demonstrates how to stream package metadata (specifically `info/index.json`) from a remote `.conda` or `.tar.bz2` package URL without downloading the entire file. It iterates through the package's members and extracts the `index.json` content."},"warnings":[{"fix":"Be mindful of the package format you are processing. For `.conda` files, you can `break` early once the desired metadata is found. For `.tar.bz2`, be prepared to iterate through more members or implement specific filtering.","message":"The behavior of `stream_conda_info` differs between `.conda` and `.tar.bz2` formats. For `.tar.bz2`, it yields all members, while for `.conda`, it yields members from the requested inner archive, allowing early termination.","severity":"gotcha","affected_versions":"All versions"},{"fix":"Ensure your file-like objects for `.conda` are seekable if using `conda_reader_for_url`. If dealing with non-seekable streams and `.conda` packages, consider alternative streaming approaches or pre-downloading if seeking is unavoidable.","message":"When using `conda_reader_for_url` for `.conda` files, the returned file-like object *must* be seekable. For `.tar.bz2` files, it only needs to be readable.","severity":"gotcha","affected_versions":"All versions"},{"fix":"If your use case requires the entire package content, download the `.conda` or `.tar.bz2` file first using a standard HTTP client, then use the file-based APIs provided by `conda-package-streaming` or `conda-package-handling`.","message":"This library is optimized for *streaming metadata* and individual components without full package download. Using its URL-based APIs to extract an entire package is inefficient and against its design; prefer pre-downloading the package for full extraction.","severity":"gotcha","affected_versions":"All versions"}],"env_vars":null,"last_verified":"2026-04-16T00:00:00.000Z","next_check":"2026-07-15T00:00:00.000Z","problems":[{"fix":"Correct the import path to `from conda_package_streaming import package_streaming` and then access functions like `package_streaming.stream_conda_info`.","cause":"Attempting to import `package_streaming` or its functions directly from the top-level `conda_package_streaming` module, instead of as a submodule.","error":"ModuleNotFoundError: No module named 'conda_package_streaming.package_streaming'"},{"fix":"Ensure `member.name` correctly identifies a JSON file (e.g., `'info/index.json'`) and that `tar.extractfile(member)` returns a valid, non-empty file-like object containing well-formed JSON before attempting to load it.","cause":"Attempting to `json.load()` a `tarfile.TarInfo` object directly, or an extracted file-like object that is empty or does not contain valid JSON, often due to an incorrect `member.name` check or reading the wrong file.","error":"json.JSONDecodeError: Expecting value: line 1 column 1 (char 0)"}]}