{"id":6007,"library":"multi-storage-client","title":"Multi-Storage Client (MSC)","description":"The Multi-Storage Client (MSC) is a unified, high-performance Python client designed for seamless interaction with various object and file stores, including AWS S3, Azure Blob Storage, Google Cloud Storage (GCS), NVIDIA AIStore, Oracle Cloud Infrastructure (OCI) Object Storage, and POSIX file systems. It offers a generic interface, simplifying data access and enabling storage location changes without extensive code modifications. As of its current version 0.46.0, it maintains an active release cadence with frequent updates.","status":"active","version":"0.46.0","language":"en","source_language":"en","source_url":"https://github.com/NVIDIA/multi-storage-client","tags":["cloud storage","object storage","file system","S3","GCS","Azure Blob","NVIDIA AIStore","OCI","multi-cloud","data access","high-performance"],"install":[{"cmd":"pip install multi-storage-client","lang":"bash","label":"Base client (POSIX file systems)"},{"cmd":"pip install \"multi-storage-client[boto3]\"","lang":"bash","label":"For AWS S3"},{"cmd":"pip install \"multi-storage-client[google-cloud-storage]\"","lang":"bash","label":"For Google Cloud Storage (GCS)"},{"cmd":"pip install \"multi-storage-client[azure-storage-blob]\"","lang":"bash","label":"For Azure Blob Storage"}],"dependencies":[{"reason":"Required for AWS S3 and S3-compatible object storage support.","package":"boto3","optional":true},{"reason":"Required for Google Cloud Storage (GCS) support.","package":"google-cloud-storage","optional":true},{"reason":"Required for Azure Blob Storage support.","package":"azure-storage-blob","optional":true},{"reason":"Required for Oracle Cloud Infrastructure (OCI) Object Storage support.","package":"oci","optional":true},{"reason":"Required for NVIDIA AIStore support.","package":"aistore","optional":true},{"reason":"Used for `vault` extra, incorrectly added as core dependency in 0.45.0, restored as optional.","package":"cryptography","optional":true}],"imports":[{"symbol":"StorageClient","correct":"from multistorageclient.client import StorageClient"},{"symbol":"msc_path","correct":"from multistorageclient.path import msc_path"}],"quickstart":{"code":"import os\nimport yaml\nfrom multistorageclient.client import StorageClient\n\n# Create a dummy MSC configuration file for local testing\nconfig_content = '''\nprofiles:\n  local-files:\n    storage_provider:\n      type: posix\n      options:\n        base_path: ./msc_data\n'''\n\n# Create a directory for local files\nos.makedirs('./msc_data', exist_ok=True)\n\nconfig_path = './msc_config.yaml'\nwith open(config_path, 'w') as f:\n    f.write(config_content)\n\n# Set the MSC_CONFIG environment variable to point to the config file\nos.environ['MSC_CONFIG'] = config_path\n\ntry:\n    # Instantiate the client\n    client = StorageClient()\n\n    # Example: Write a file using the 'local-files' profile\n    local_file_path = 'msc://local-files/example.txt'\n    with client.open(local_file_path, 'w') as f:\n        f.write('Hello from Multi-Storage Client!')\n    print(f\"Content written to {local_file_path}\")\n\n    # Example: Read the file\n    with client.open(local_file_path, 'r') as f:\n        content = f.read()\n    print(f\"Content read from {local_file_path}: {content}\")\n\n    # Example: List objects in the 'local-files' profile\n    objects = client.list_objects('msc://local-files/')\n    print(\"Objects in 'msc://local-files/':\")\n    for obj in objects:\n        print(f\"- {obj.uri}\")\n\nfinally:\n    # Clean up the dummy config file and data directory\n    if os.path.exists(config_path):\n        os.remove(config_path)\n    if os.path.exists('./msc_data/example.txt'):\n        os.remove('./msc_data/example.txt')\n    if os.path.exists('./msc_data'):\n        os.rmdir('./msc_data')\n    del os.environ['MSC_CONFIG']\n","lang":"python","description":"This quickstart demonstrates how to initialize the `StorageClient` using a YAML configuration file and perform basic file operations (write, read, list) with a local POSIX storage profile. Authentication for cloud providers is typically managed within the YAML profiles or via environment variables, avoiding hardcoding in Python code."},"warnings":[{"fix":"Upgrade your Python environment to version 3.10 or later. The library requires Python <4, >=3.10.","message":"Python 3.9 support was dropped in version 0.43.0. Users on Python 3.9 must upgrade to Python 3.10 or newer.","severity":"breaking","affected_versions":">=0.43.0"},{"fix":"Define your storage provider profiles (including credentials and options) in a YAML file and set the `MSC_CONFIG` environment variable to its path.","message":"The Multi-Storage Client heavily relies on external YAML configuration files for defining storage profiles and credentials. Direct in-code configuration for providers/credentials is not the primary pattern.","severity":"gotcha","affected_versions":">=0.1.0"},{"fix":"Install the required extras for your cloud provider, e.g., `pip install \"multi-storage-client[boto3]\"` for AWS S3.","message":"To support specific cloud storage providers (e.g., AWS S3, Google Cloud Storage), you must install the corresponding optional dependencies using `pip install \"multi-storage-client[extra_name]\"`. The base `multi-storage-client` package only supports POSIX file systems by default.","severity":"gotcha","affected_versions":">=0.1.0"},{"fix":"If you intend to use `vault` integration, install it via `pip install \"multi-storage-client[vault]\"`. Otherwise, avoid importing the instrumentation.auth module if the extra is not installed.","message":"Attempting to import `multistorageclient.instrumentation.auth` without installing the `[vault]` extra will result in an `ImportError`.","severity":"gotcha","affected_versions":">=0.45.0"},{"fix":"Upgrade to version 0.45.1 or newer to ensure `cryptography` is an optional dependency. If you require `cryptography` features, ensure you install the relevant extra (e.g., `[vault]`).","message":"The `cryptography` package was incorrectly made a core requirement in version 0.45.0. This was corrected in a subsequent release (0.45.1) to restore it as an optional dependency, reducing the default install footprint for users not needing its features.","severity":"deprecated","affected_versions":"0.45.0"}],"env_vars":null,"last_verified":"2026-04-14T00:00:00.000Z","next_check":"2026-07-13T00:00:00.000Z"}