cloudpathlib
raw JSON → 0.23.0 verified Tue May 12 auth: no python install: verified
cloudpathlib provides pathlib-style classes for interacting with files and directories in various cloud storage services such as AWS S3, Google Cloud Storage, and Azure Blob Storage. It aims to offer a familiar filesystem interface, abstracting away cloud-specific details. The library is actively maintained, with a typical release cadence that includes bug fixes and new features. The current version is 0.23.0.
pip install cloudpathlib Common errors
error cloudpathlib.exceptions.MissingDependenciesError: To use S3Path, you must install cloudpathlib with the 's3' extra: pip install "cloudpathlib[s3]" ↓
cause You are trying to use a cloud-specific Path class (e.g., `S3Path`, `GSPath`, `AzureBlobPath`) or `CloudPath` with a cloud URI scheme, but the necessary cloud provider SDK dependencies (like `boto3` for S3, `google-cloud-storage` for GCS, or `azure-storage-blob` for Azure) were not installed along with `cloudpathlib` as optional 'extras'.
fix
Install
cloudpathlib with the appropriate extras for your cloud provider. For S3, use pip install "cloudpathlib[s3]". For Google Cloud Storage, use pip install "cloudpathlib[gs]". For Azure Blob Storage, use pip install "cloudpathlib[azure]". To install all, use pip install "cloudpathlib[all]". error botocore.exceptions.NoCredentialsError: Unable to locate credentials. ↓
cause You are attempting to access a cloud storage resource (e.g., S3, GCS, Azure Blob Storage) using `cloudpathlib` without configuring the necessary authentication credentials for the respective cloud service.
fix
Configure your cloud credentials. For AWS S3, set
AWS_ACCESS_KEY_ID and AWS_SECRET_ACCESS_KEY environment variables or use an AWS_PROFILE. For public S3 buckets, instantiate S3Client with no_sign_request=True and use that client. For Google Cloud Storage, set the GOOGLE_APPLICATION_CREDENTIALS environment variable. For Azure, set AZURE_STORAGE_CONNECTION_STRING. Alternatively, explicitly instantiate and configure a client object (e.g., S3Client, GSClient, AzureBlobClient) with your credentials. error OSError: Cannot save file into a non-existent directory: '/var/folders/hf/cnzhkc851mqcmg0f8nc41z6h0000gn/T/tmp67yk4w9b/human-datalake/projects/raw/folder' ↓
cause When using `cloudpathlib` objects with external libraries (like Pandas' `to_csv` or `to_parquet`) that rely on `os.fspath()` or expect a local file path for write operations, `cloudpathlib`'s `__fspath__` method returns the path to its local cache. If the parent directories for this local cache path do not exist, the external library will raise an `OSError`. Additionally, writing to this local cache directly will not upload the file to the cloud.
fix
Instead of directly passing a
CloudPath object to external libraries for writing, explicitly open the CloudPath object for writing and pass the resulting file-like object, or write to a temporary local file first and then upload it using cloudpathlib's upload_from method. For example, for Pandas: with my_cloud_path.open('w') as f: df.to_csv(f) (or to_parquet, etc.) or df.to_csv('local_temp_file.csv'); my_cloud_path.upload_from('local_temp_file.csv'). error ModuleNotFoundError: No module named 'pathlib._local' ↓
cause This error occurs when `cloudpathlib` is used with Python 3.13 or 3.14 (or potentially newer versions) due to internal changes in Python's `pathlib` module, where private APIs that `cloudpathlib` previously relied on have been moved or removed.
fix
Upgrade
cloudpathlib to a version that officially supports Python 3.13+ (e.g., v0.20.0 or later, as mentioned in a GitHub issue for 3.13 support). If a newer version of cloudpathlib is not yet released that supports your specific Python version, you may need to either downgrade your Python version or wait for an updated cloudpathlib release. Check the cloudpathlib GitHub repository or PyPI page for the latest compatibility information. Warnings
breaking The `CloudPath.copy` method's first parameter was renamed from `destination` to `target`. Code relying on keyword arguments or positional argument names will break. ↓
fix Update calls to `CloudPath.copy` to use `target` as the parameter name instead of `destination`.
breaking Support for Python 3.7 has been removed. The last version compatible with Python 3.7 was v0.18.1. ↓
fix Upgrade your Python environment to 3.8 or newer.
breaking The `CloudPath` constructor changed how it handles a client object as the second argument. Previously, it could implicitly accept a client. Now, it needs to be passed explicitly as a keyword argument (e.g., `CloudPath('s3://...', client=my_client)`). ↓
fix If you were passing a client object as the second positional argument to `CloudPath` or `AnyPath`, ensure you pass it using the `client=` keyword argument.
deprecated The environment variable `CLOUPATHLIB_FILE_CACHE_MODE` (with a typo) was deprecated and support for it has been removed. The correct environment variable is `CLOUDPATHLIB_FILE_CACHE_MODE`. ↓
fix Update your environment configurations to use `CLOUDPATHLIB_FILE_CACHE_MODE` instead of `CLOUPATHLIB_FILE_CACHE_MODE`.
gotcha The default for `missing_ok` in `CloudPath.unlink()` is `True`, unlike `pathlib.Path.unlink()` where it defaults to `False`. This means `unlink()` will not raise an error if the file does not exist by default. ↓
fix If you require an error to be raised when attempting to unlink a non-existent file, explicitly pass `missing_ok=False` to `unlink()`.
gotcha `CloudPath.rmdir()` will raise a `DirectoryNotEmptyError` if the directory is not empty. To recursively remove a non-empty directory, you must use `CloudPath.rmtree()`. ↓
fix Use `path.rmtree()` for recursive directory deletion, or ensure directories are empty before calling `path.rmdir()`.
gotcha An `ImportError` due to an incompatible `google-cloud-storage` version was fixed in v0.18.1 by not using `transfer_manager` if unavailable. Users on older `google-cloud-storage` versions might encounter this. ↓
fix Upgrade to `cloudpathlib` v0.18.1 or newer, and ensure your `google-cloud-storage` dependency is compatible or also updated.
gotcha When performing operations on cloud paths, `cloudpathlib` requires appropriate authentication credentials for the target cloud provider (e.g., S3, GCS, Azure). Without these, operations will fail with 'Unable to locate credentials' or similar authentication errors. ↓
fix Ensure your environment or system is configured with valid credentials for the respective cloud provider. For AWS S3, set `AWS_ACCESS_KEY_ID`, `AWS_SECRET_ACCESS_KEY`, and `AWS_SESSION_TOKEN` (if applicable). For Google Cloud Storage, set `GOOGLE_APPLICATION_CREDENTIALS` to the path of your service account key file. For Azure Blob Storage, configure environment variables like `AZURE_STORAGE_ACCOUNT_NAME` and `AZURE_STORAGE_ACCOUNT_KEY` or `AZURE_STORAGE_CONNECTION_STRING`.
gotcha Operations with cloud providers (S3, GCS, Azure) will fail with 'Unable to locate credentials' if environment variables, configuration files, or other authentication methods are not correctly set up for the respective cloud client library. ↓
fix Ensure your environment is configured with appropriate credentials for your cloud provider (e.g., AWS_ACCESS_KEY_ID, AWS_SECRET_ACCESS_KEY, AWS_REGION for S3; GOOGLE_APPLICATION_CREDENTIALS for GCS; AZURE_STORAGE_ACCOUNT_NAME, AZURE_STORAGE_ACCOUNT_KEY for Azure).
Install
pip install "cloudpathlib[s3]" pip install "cloudpathlib[gs]" pip install "cloudpathlib[azure]" pip install "cloudpathlib[all]" Install compatibility verified last tested: 2026-05-12
python os / libc variant status wheel install import disk
3.10 alpine (musl) all sdist - 2.31s 94.0M
3.10 alpine (musl) all - - 2.05s 92.7M
3.10 alpine (musl) azure wheel - 1.33s 48.1M
3.10 alpine (musl) azure - - 1.25s 47.0M
3.10 alpine (musl) gs sdist - 1.41s 52.3M
3.10 alpine (musl) gs - - 1.29s 51.2M
3.10 alpine (musl) s3 wheel - 0.78s 51.4M
3.10 alpine (musl) s3 - - 0.75s 51.2M
3.10 alpine (musl) cloudpathlib wheel - 0.17s 18.6M
3.10 alpine (musl) cloudpathlib - - 0.18s 18.6M
3.10 slim (glibc) all wheel 7.9s 1.80s 95M
3.10 slim (glibc) all - - 1.60s 93M
3.10 slim (glibc) azure wheel 3.9s 0.95s 48M
3.10 slim (glibc) azure - - 0.92s 47M
3.10 slim (glibc) gs wheel 4.8s 1.04s 53M
3.10 slim (glibc) gs - - 0.96s 52M
3.10 slim (glibc) s3 wheel 4.0s 0.60s 52M
3.10 slim (glibc) s3 - - 0.57s 52M
3.10 slim (glibc) cloudpathlib wheel 1.5s 0.12s 19M
3.10 slim (glibc) cloudpathlib - - 0.12s 19M
3.11 alpine (musl) all sdist - 2.87s 101.2M
3.11 alpine (musl) all - - 3.20s 99.9M
3.11 alpine (musl) azure wheel - 1.50s 52.4M
3.11 alpine (musl) azure - - 1.64s 51.3M
3.11 alpine (musl) gs sdist - 1.96s 56.6M
3.11 alpine (musl) gs - - 2.25s 55.5M
3.11 alpine (musl) s3 wheel - 0.92s 54.2M
3.11 alpine (musl) s3 - - 1.01s 54.0M
3.11 alpine (musl) cloudpathlib wheel - 0.21s 20.3M
3.11 alpine (musl) cloudpathlib - - 0.21s 20.3M
3.11 slim (glibc) all wheel 7.5s 2.31s 102M
3.11 slim (glibc) all - - 2.24s 101M
3.11 slim (glibc) azure wheel 3.6s 1.34s 53M
3.11 slim (glibc) azure - - 1.24s 52M
3.11 slim (glibc) gs wheel 4.6s 1.52s 57M
3.11 slim (glibc) gs - - 1.43s 56M
3.11 slim (glibc) s3 wheel 3.6s 0.81s 55M
3.11 slim (glibc) s3 - - 0.78s 54M
3.11 slim (glibc) cloudpathlib wheel 1.6s 0.17s 21M
3.11 slim (glibc) cloudpathlib - - 0.16s 21M
3.12 alpine (musl) all sdist - 2.95s 92.2M
3.12 alpine (musl) all - - 3.18s 90.9M
3.12 alpine (musl) azure wheel - 1.77s 43.8M
3.12 alpine (musl) azure - - 1.78s 42.7M
3.12 alpine (musl) gs sdist - 2.05s 48.1M
3.12 alpine (musl) gs - - 2.25s 47.0M
3.12 alpine (musl) s3 wheel - 0.83s 45.8M
3.12 alpine (musl) s3 - - 0.87s 45.6M
3.12 alpine (musl) cloudpathlib wheel - 0.19s 12.1M
3.12 alpine (musl) cloudpathlib - - 0.19s 12.1M
3.12 slim (glibc) all wheel 6.5s 2.64s 93M
3.12 slim (glibc) all - - 2.81s 92M
3.12 slim (glibc) azure wheel 3.2s 1.81s 44M
3.12 slim (glibc) azure - - 1.81s 43M
3.12 slim (glibc) gs wheel 3.9s 1.82s 49M
3.12 slim (glibc) gs - - 2.09s 48M
3.12 slim (glibc) s3 wheel 3.3s 0.88s 46M
3.12 slim (glibc) s3 - - 0.84s 46M
3.12 slim (glibc) cloudpathlib wheel 1.4s 0.19s 13M
3.12 slim (glibc) cloudpathlib - - 0.20s 13M
3.13 alpine (musl) all sdist - 2.83s 91.7M
3.13 alpine (musl) all - - 3.11s 90.3M
3.13 alpine (musl) azure wheel - 1.86s 43.5M
3.13 alpine (musl) azure - - 1.76s 42.3M
3.13 alpine (musl) gs sdist - 1.97s 47.8M
3.13 alpine (musl) gs - - 2.22s 46.5M
3.13 alpine (musl) s3 wheel - 0.84s 45.5M
3.13 alpine (musl) s3 - - 0.87s 45.2M
3.13 alpine (musl) cloudpathlib wheel - 0.18s 11.9M
3.13 alpine (musl) cloudpathlib - - 0.18s 11.8M
3.13 slim (glibc) all wheel 6.2s 2.62s 93M
3.13 slim (glibc) all - - 2.90s 91M
3.13 slim (glibc) azure wheel 3.3s 1.66s 44M
3.13 slim (glibc) azure - - 1.71s 43M
3.13 slim (glibc) gs wheel 3.9s 1.75s 49M
3.13 slim (glibc) gs - - 2.07s 47M
3.13 slim (glibc) s3 wheel 2.9s 0.80s 46M
3.13 slim (glibc) s3 - - 0.84s 46M
3.13 slim (glibc) cloudpathlib wheel 1.5s 0.20s 12M
3.13 slim (glibc) cloudpathlib - - 0.18s 12M
3.9 alpine (musl) all sdist - 2.12s 93.9M
3.9 alpine (musl) all - - 1.88s 92.7M
3.9 alpine (musl) azure wheel - 1.29s 48.1M
3.9 alpine (musl) azure - - 1.19s 47.1M
3.9 alpine (musl) gs sdist - 1.30s 52.3M
3.9 alpine (musl) gs - - 1.21s 51.2M
3.9 alpine (musl) s3 wheel - 0.66s 50.8M
3.9 alpine (musl) s3 - - 0.65s 50.7M
3.9 alpine (musl) cloudpathlib wheel - 0.15s 18.1M
3.9 alpine (musl) cloudpathlib - - 0.15s 18.1M
3.9 slim (glibc) all wheel 9.3s 1.94s 95M
3.9 slim (glibc) all - - 1.67s 94M
3.9 slim (glibc) azure wheel 4.5s 1.19s 48M
3.9 slim (glibc) azure - - 1.05s 47M
3.9 slim (glibc) gs wheel 5.7s 1.27s 53M
3.9 slim (glibc) gs - - 1.08s 52M
3.9 slim (glibc) s3 wheel 4.9s 0.70s 51M
3.9 slim (glibc) s3 - - 0.55s 51M
3.9 slim (glibc) cloudpathlib wheel 1.8s 0.13s 19M
3.9 slim (glibc) cloudpathlib - - 0.13s 19M
Imports
- CloudPath
from cloudpathlib import CloudPath - AnyPath
from cloudpathlib import AnyPath - S3Path
from cloudpathlib.s3 import S3Path - GSPath
from cloudpathlib.gs import GSPath - AzureBlobPath wrong
from cloudpathlib.azure import AzurePathcorrectfrom cloudpathlib.azure import AzureBlobPath - HttpsPath
from cloudpathlib.https import HttpsPath
Quickstart last tested: 2026-04-24
import os
from cloudpathlib import CloudPath
# Ensure environment variables are set for the chosen cloud provider
# e.g., for S3: AWS_ACCESS_KEY_ID, AWS_SECRET_ACCESS_KEY
# for GS: GOOGLE_APPLICATION_CREDENTIALS (path to JSON key file)
# for Azure: AZURE_STORAGE_CONNECTION_STRING
# Example for S3
# Replace with your actual bucket and file name
s3_file_path = CloudPath("s3://your-test-bucket/hello.txt")
try:
# Write to the cloud file
s3_file_path.write_text("Hello from cloudpathlib!")
print(f"Successfully wrote to {s3_file_path}")
# Read from the cloud file
content = s3_file_path.read_text()
print(f"Content read: '{content}'")
# Check if the file exists
if s3_file_path.exists():
print(f"File {s3_file_path} exists.")
# Clean up (optional)
s3_file_path.unlink()
print(f"File {s3_file_path} deleted.")
except Exception as e:
print(f"An error occurred: {e}")