OpenDAL Python Binding

Common errors

• RuntimeError: await wasn't called

  cause Attempting to call an asynchronous method (e.g., `op.read()`, `op.write()`) of `opendal.Operator` directly without `await` keyword or outside an `asyncio` event loop.
  fix Ensure all calls to `opendal.Operator` methods are preceded by `await` and executed within an `asyncio.run()` block or an already running event loop.

• opendal.Error: ServiceError { kind: InvalidConfig, ... }

  cause The dictionary passed to `opendal.Operator()` for configuration is missing required parameters or contains incorrect values for the specified `scheme`.
  fix Double-check the OpenDAL documentation for the specific storage service you are trying to connect to. Verify all required configuration keys (e.g., `bucket`, `region`, `access_key_id`) are present and their values are correct.

• AttributeError: 'BlockingOperator' object has no attribute '__aenter__'

  cause Attempting to use `async with` context manager with `opendal.BlockingOperator`. The `BlockingOperator` is designed for synchronous use and does not implement the asynchronous context manager protocol.
  fix Use `opendal.Operator` with `async with` if an asynchronous context manager is desired. For `opendal.BlockingOperator`, use it directly without `async with`.

• AttributeError: 'NoneType' object has no attribute 'decode' (after a write operation)

  cause In `opendal` Python binding version 0.46.0, `op.write()` returns `None`. Your code is attempting to access attributes or methods (like `decode()`) on the `None` return value, assuming it received data or metadata.
  fix Recognize that `op.write()` returns `None` in `opendal` v0.46.0. If you need to verify content or access metadata, perform a subsequent `op.read()` or `op.stat()` operation. Be aware that future versions will return `Metadata`.

Warnings

• gotcha OpenDAL Python provides two distinct entry points: `opendal.Operator` for asynchronous (async/await) operations and `opendal.BlockingOperator` for synchronous use. Mixing these can lead to `RuntimeError` or `TypeError` if an async method is called synchronously, or vice-versa.
  Fix: Always `await` methods of `opendal.Operator` within an `asyncio` event loop. For synchronous code, use `opendal.BlockingOperator` and its methods directly without `await`.
• gotcha Incorrect or incomplete configuration for a chosen storage scheme (e.g., S3, Azure Blob, GCS) will result in `opendal.Error: ServiceError { kind: InvalidConfig, ... }`. Each service requires specific parameters (e.g., bucket, region, credentials).
  Fix: Refer to the official OpenDAL documentation for the exact configuration parameters required for your specific storage service. Ensure all mandatory parameters are provided with correct values.
• breaking In Rust core v0.52.0 and later (which will affect future Python binding versions greater than 0.46.0), `write` operations will return `Metadata` instead of `None`. Code currently assuming `op.write()` returns `None` will need to be updated.
  Fix: If upgrading to a future OpenDAL Python binding, review code that calls `op.write()`. It will receive a `Metadata` object, allowing access to properties like `content_length` or `etag` immediately after writing.
• breaking In Rust core v0.55.0 and later (which will affect future Python binding versions greater than 0.46.0), timestamp fields in `Metadata` (e.g., `last_modified`) will use `jiff.Timestamp` objects. Current versions might return a different type (e.g., `datetime.datetime`).
  Fix: When upgrading to a future OpenDAL Python binding, ensure your code handles `jiff.Timestamp` objects for metadata timestamp fields. You may need to `pip install jiff` and adapt your datetime parsing/formatting logic.

Install

• pip install opendal Install OpenDAL

Imports

• Operator

  from opendal import Operator

• BlockingOperator

  from opendal import BlockingOperator

• Error

  from opendal import Error

Quickstart

This quickstart demonstrates how to initialize an OpenDAL S3 operator and perform basic asynchronous operations: write, read, get metadata, and delete. Ensure your S3 bucket, region, and credentials are set as environment variables or provided directly in the configuration. For a simpler start, replace the config with `op = opendal.Operator("memory")`.

import opendal
import asyncio
import os

async def main():
    # Configure OpenDAL for S3. Replace with your actual credentials or use a different scheme.
    # Using os.environ.get for security and flexibility.
    config = {
        "scheme": "s3",
        "bucket": os.environ.get("OPENDAL_S3_BUCKET", "your-s3-bucket"),
        "region": os.environ.get("OPENDAL_S3_REGION", "us-east-1"),
        "access_key_id": os.environ.get("OPENDAL_S3_ACCESS_KEY_ID", ""),
        "secret_access_key": os.environ.get("OPENDAL_S3_SECRET_ACCESS_KEY", ""),
    }
    
    try:
        # Initialize the asynchronous Operator
        op = opendal.Operator(config)
        key = "hello_opendal.txt"
        content_to_write = b"Hello from OpenDAL Python!"
        
        # Write data asynchronously
        await op.write(key, content_to_write)
        print(f"Successfully wrote '{content_to_write.decode()}' to {key}")
        
        # Read data asynchronously
        read_content = await op.read(key)
        print(f"Successfully read '{read_content.decode()}' from {key}")
        
        # Get metadata asynchronously
        metadata = await op.stat(key)
        print(f"Metadata for {key}: size={metadata.content_length}, last_modified={metadata.last_modified}")
        
        # Clean up
        await op.delete(key)
        print(f"Successfully deleted {key}")

    except opendal.Error as e:
        print(f"OpenDAL Error: {e}")
    except Exception as e:
        print(f"An unexpected error occurred: {e}")

if __name__ == "__main__":
    # Ensure the environment variables are set for S3 or use a simpler scheme like 'memory'.
    # Example for 'memory' scheme (no credentials needed, replace config):
    # op = opendal.Operator("memory")
    asyncio.run(main())