{"id":2621,"library":"obstore","title":"obstore: Python Object Storage Interface","description":"obstore is a Python library providing a simple, high-throughput interface for various object storage services like Amazon S3, Google Cloud Storage, Azure Blob Storage, and S3-compliant APIs. It features both synchronous and asynchronous APIs, streaming downloads/uploads, and automatic multipart uploads for large files, powered by a Rust backend for performance. The current version is 0.9.2, with a frequent release cadence, often introducing minor updates and fixes.","status":"active","version":"0.9.2","language":"en","source_language":"en","source_url":"https://github.com/developmentseed/obstore","tags":["object-storage","s3","gcs","azure-blob-storage","cloud-storage","async","rust-powered","high-performance"],"install":[{"cmd":"pip install obstore","lang":"bash","label":"Install stable version"}],"dependencies":[{"reason":"Required for `S3Store.from_session()` to integrate with AWS sessions.","package":"boto3","optional":true},{"reason":"Required for `obstore.list(return_arrow=True)` to enable zero-copy Arrow integration for listing results.","package":"arro3-core","optional":true}],"imports":[{"symbol":"obstore","correct":"import obstore as obs"},{"note":"Store classes like MemoryStore, S3Store, GCSStore are located in the `obstore.store` submodule, not directly under `obstore`.","wrong":"import obstore.MemoryStore","symbol":"MemoryStore","correct":"from obstore.store import MemoryStore"},{"note":"All operations (put, get, list, delete, copy, etc.) are top-level functions exported by the `obstore` module, not methods of the store object itself.","wrong":"store.put('path', b'data')","symbol":"put","correct":"obs.put(store, 'path', b'data')"}],"quickstart":{"code":"import obstore as obs\nfrom obstore.store import MemoryStore\n\n# Initialize an in-memory store for demonstration\nstore = MemoryStore()\n\n# Define a file path and content\nfile_path = \"my_document.txt\"\nfile_content = b\"Hello, obstore world!\"\n\n# Put the object into the store\nobs.put(store, file_path, file_content)\nprint(f\"Object '{file_path}' put into store.\")\n\n# Get the object from the store\nresponse = obs.get(store, file_path)\nretrieved_content = response.bytes()\nprint(f\"Retrieved content: {retrieved_content.decode()}\")\n\nassert retrieved_content == file_content\nprint(\"Content matches!\")\n\n# Asynchronous operations are also available (requires an async context)\n# async def main():\n#    await obs.put_async(store, 'async_file.txt', b'async data')\n#    res_async = await obs.get_async(store, 'async_file.txt')\n#    print(f\"Async retrieved: {res_async.bytes().decode()}\")\n# import asyncio\n# asyncio.run(main())","lang":"python","description":"This quickstart demonstrates how to initialize an in-memory object store and perform basic synchronous `put` and `get` operations. Note that operations like `put` and `get` are top-level functions within the `obstore` module, taking the store instance as their first argument. Asynchronous counterparts (e.g., `put_async`, `get_async`) are also available."},"warnings":[{"fix":"Upgrade Python to 3.10 or newer.","message":"Support for Python 3.9 was deprecated in `obstore` version 0.9.0. Users on Python 3.9 should upgrade their Python environment.","severity":"breaking","affected_versions":">=0.9.0"},{"fix":"Refactor S3 store construction to use explicit credential providers or environment variable configuration instead of `from_session`.","message":"In older versions (prior to 0.7.0), `S3Store.from_session()` and `S3Store._from_native()` were removed. Users should transition to using credential providers for S3 authentication.","severity":"breaking","affected_versions":"<0.7.0"},{"fix":"Review all path inputs to `obstore` functions and ensure they are properly formed (e.g., not already percent-encoded if they contain special characters).","message":"Path encoding behavior changed in version 0.8.2 to prevent unintentional double-encoding. Users must now ensure that paths provided to `obstore` are valid and correctly encoded.","severity":"breaking","affected_versions":">=0.8.2"},{"fix":"Update `AzureStore` constructor calls to use `container_name='my-container'` instead of `container='my-container'`.","message":"In version 0.5.0, the `container` parameter for `AzureStore`'s constructor was renamed to `container_name` and became a keyword-only argument. Using `container` will raise an error.","severity":"breaking","affected_versions":">=0.5.0"},{"fix":"Always call object storage operations as `obstore.<function_name>(store_instance, ...)`.","message":"Unlike many object storage libraries, `obstore`'s core operations (`put`, `get`, `list`, `delete`, `copy`, etc.) are top-level functions (e.g., `obs.put(store, ...)`) rather than methods on the store object (e.g., `store.put(...)`).","severity":"gotcha","affected_versions":"All versions"},{"fix":"If using `return_arrow=True`, ensure `pip install obstore[arrow]` or `pip install arro3-core` is run.","message":"When listing objects with `obstore.list(return_arrow=True)`, the optional `arro3-core` dependency is required. Without it, attempting to use this feature will fail.","severity":"gotcha","affected_versions":"All versions"}],"env_vars":null,"last_verified":"2026-04-10T00:00:00.000Z","next_check":"2026-07-09T00:00:00.000Z"}