cloudpathlib

Warnings

• breaking The `CloudPath.copy` method's first parameter was renamed from `destination` to `target`. Code relying on keyword arguments or positional argument names will break.
  Fix: Update calls to `CloudPath.copy` to use `target` as the parameter name instead of `destination`.
• breaking Support for Python 3.7 has been removed. The last version compatible with Python 3.7 was v0.18.1.
  Fix: Upgrade your Python environment to 3.8 or newer.
• breaking The `CloudPath` constructor changed how it handles a client object as the second argument. Previously, it could implicitly accept a client. Now, it needs to be passed explicitly as a keyword argument (e.g., `CloudPath('s3://...', client=my_client)`).
  Fix: If you were passing a client object as the second positional argument to `CloudPath` or `AnyPath`, ensure you pass it using the `client=` keyword argument.
• deprecated The environment variable `CLOUPATHLIB_FILE_CACHE_MODE` (with a typo) was deprecated and support for it has been removed. The correct environment variable is `CLOUDPATHLIB_FILE_CACHE_MODE`.
  Fix: Update your environment configurations to use `CLOUDPATHLIB_FILE_CACHE_MODE` instead of `CLOUPATHLIB_FILE_CACHE_MODE`.
• gotcha The default for `missing_ok` in `CloudPath.unlink()` is `True`, unlike `pathlib.Path.unlink()` where it defaults to `False`. This means `unlink()` will not raise an error if the file does not exist by default.
  Fix: If you require an error to be raised when attempting to unlink a non-existent file, explicitly pass `missing_ok=False` to `unlink()`.
• gotcha `CloudPath.rmdir()` will raise a `DirectoryNotEmptyError` if the directory is not empty. To recursively remove a non-empty directory, you must use `CloudPath.rmtree()`.
  Fix: Use `path.rmtree()` for recursive directory deletion, or ensure directories are empty before calling `path.rmdir()`.
• gotcha An `ImportError` due to an incompatible `google-cloud-storage` version was fixed in v0.18.1 by not using `transfer_manager` if unavailable. Users on older `google-cloud-storage` versions might encounter this.
  Fix: Upgrade to `cloudpathlib` v0.18.1 or newer, and ensure your `google-cloud-storage` dependency is compatible or also updated.

Install

• pip install cloudpathlib Base installation
• pip install "cloudpathlib[s3]" For AWS S3
• pip install "cloudpathlib[gs]" For Google Cloud Storage
• pip install "cloudpathlib[azure]" For Azure Blob Storage
• pip install "cloudpathlib[all]" For all supported cloud services

Imports

• CloudPath

  from cloudpathlib import CloudPath

• AnyPath

  from cloudpathlib import AnyPath

• S3Path

  from cloudpathlib.s3 import S3Path

• GSPath

  from cloudpathlib.gs import GSPath

• AzureBlobPath

  from cloudpathlib.azure import AzureBlobPath

  The specific class for Azure Blob Storage is AzureBlobPath, not a generic AzurePath.
• HttpsPath

  from cloudpathlib.https import HttpsPath

Quickstart

This quickstart demonstrates how to initialize a CloudPath object for S3, write text to it, read text from it, check its existence, and then delete it. Users should ensure their cloud provider authentication environment variables are correctly set for automatic authentication. Similar patterns apply to GSPath and AzureBlobPath.

import os
from cloudpathlib import CloudPath

# Ensure environment variables are set for the chosen cloud provider
# e.g., for S3: AWS_ACCESS_KEY_ID, AWS_SECRET_ACCESS_KEY
# for GS: GOOGLE_APPLICATION_CREDENTIALS (path to JSON key file)
# for Azure: AZURE_STORAGE_CONNECTION_STRING

# Example for S3
# Replace with your actual bucket and file name
s3_file_path = CloudPath("s3://your-test-bucket/hello.txt")

try:
    # Write to the cloud file
    s3_file_path.write_text("Hello from cloudpathlib!")
    print(f"Successfully wrote to {s3_file_path}")

    # Read from the cloud file
    content = s3_file_path.read_text()
    print(f"Content read: '{content}'")

    # Check if the file exists
    if s3_file_path.exists():
        print(f"File {s3_file_path} exists.")

    # Clean up (optional)
    s3_file_path.unlink()
    print(f"File {s3_file_path} deleted.")

except Exception as e:
    print(f"An error occurred: {e}")