s3path

Warnings

• breaking Version 0.6.0 removed support for Python 3.8 and introduced support for Python 3.14. Projects still on Python 3.8 will need to upgrade their Python version before upgrading s3path.
  Fix: Upgrade Python environment to 3.9 or higher.
• breaking From version 0.6.0, the `glob` and `rglob` methods exclusively use a new, optimized algorithm. The `glob_new_algorithm` configuration parameter, previously used to switch algorithms, is now deprecated and its functionality removed.
  Fix: Remove any `glob_new_algorithm` configurations. The new algorithm is now the default and only option.
• gotcha The `.exists()` method in versions prior to 0.6.5 could return `True` for partial key matches (e.g., querying for 'foo' would return true if 'foobar' existed). This was fixed in 0.6.5 to correctly use `ListObjectsV2`.
  Fix: Upgrade to version 0.6.5 or newer to ensure accurate existence checks.
• gotcha A caching system for the `is_dir` method was introduced in 0.6.2 but subsequently reverted and removed in 0.6.3 due to issues. Relying on this caching for performance in intermediate 0.6.2 releases is not advised.
  Fix: Be aware that performance for `is_dir` in 0.6.3+ does not include this specific caching mechanism.
• gotcha The `walk` method can be very heavy on AWS S3 API calls, potentially leading to increased costs and slower performance, especially for large buckets. It is generally recommended to use recursive `glob` instead for most traversal needs.
  Fix: Prefer `S3Path.glob('**/*', ...)` for recursive directory traversal when possible.
• deprecated The `glob_new_algorithm` configuration parameter, used to switch glob algorithms, entered a deprecation cycle in version 0.6.0 as the new algorithm became the sole implementation.
  Fix: Remove usage of this configuration parameter; it no longer has an effect.
• breaking Version 0.6.3 fixed a null encryption key vulnerability. Older versions might be susceptible to this security flaw.
  Fix: Upgrade to version 0.6.3 or newer to patch this security vulnerability.

Install

• pip install s3path PyPI

Imports

• S3Path

  from s3path import S3Path

  The primary class for interacting with S3 paths.
• PureS3Path

  from s3path import PureS3Path

  For path manipulation without requiring AWS API calls.
• VersionedS3Path

  from s3path import VersionedS3Path

  For working with S3 objects in versioned buckets.
• configuration_map

  from s3path import configuration_map

  Provides access to boto3 resource configuration (added in 0.5.7).

Quickstart

This example demonstrates how to initialize `S3Path`, configure the underlying `boto3` session using environment variables, write text to an S3 object, read its content, and list objects within an S3 prefix using `iterdir()`.

import os
import boto3
from s3path import S3Path

# Configure boto3 session (recommended for programmatic access)
# Replace with your actual region and credentials or profile
# For local testing, ensure AWS_ACCESS_KEY_ID and AWS_SECRET_ACCESS_KEY are set
boto3.setup_default_session(
    region_name=os.environ.get('AWS_REGION', 'us-east-1'),
    aws_access_key_id=os.environ.get('AWS_ACCESS_KEY_ID', 'test'),
    aws_secret_access_key=os.environ.get('AWS_SECRET_ACCESS_KEY', 'test')
)

bucket_name = 'your-s3-bucket'
file_key = 'my-folder/my-file.txt'

# Create an S3Path object
s3_path = S3Path(f'/{bucket_name}/{file_key}')
print(f"S3 Path: {s3_path}")

# Check if the file exists (requires S3 interaction)
if not s3_path.exists():
    print(f"Creating file: {s3_path}")
    s3_path.write_text('Hello from s3path!')
    print("File created.")
else:
    print(f"File '{s3_path}' already exists.")

# Read content
content = s3_path.read_text()
print(f"Content of '{s3_path}': {content}")

# List contents of a directory-like prefix
directory_path = S3Path(f'/{bucket_name}/my-folder/')
print(f"\nListing contents of '{directory_path}':")
for p in directory_path.iterdir():
    print(f" - {p}")

# Clean up (optional)
# s3_path.unlink() # Uncomment to delete the file
# print(f"Deleted {s3_path}")