s3pathlib

Warnings

• gotcha S3Path objects are immutable. Operations that modify the S3 object (like `write_text()` or `write_bytes()`) will return a *new* S3Path object, especially in S3 versioning-enabled buckets. Always reassign the result if you need to work with the updated path object.
  Fix: Always capture the return value of write operations: `s3_path = s3_path.write_text('new content')`.
• gotcha Write operations (`write_text()`, `write_bytes()`) will silently overwrite existing files by default. There is no automatic error or warning if the target S3 object already exists.
  Fix: To prevent accidental overwrites, explicitly check for file existence using `if not s3_path.exists():` before performing write operations.
• gotcha S3 does not have a true directory concept; `s3pathlib` provides a 'logical' or 'soft' directory abstraction. A path ending with `/` is treated as a directory. Understanding this distinction is crucial for directory-related operations.
  Fix: Be mindful of trailing slashes (`/`) when defining S3 paths to denote directories, e.g., `S3Path('bucket', 'my-folder/')`. Use methods like `.to_dir()` if unsure.
• gotcha While `s3pathlib` mimics the `pathlib.Path` API, `S3Path` is not a direct subclass of `pathlib.Path`. This means `isinstance(s3_path_object, pathlib.Path)` checks will return `False` and code expecting a `pathlib.Path` might break.
  Fix: Avoid `isinstance(obj, pathlib.Path)` checks if you intend to support `S3Path`. Instead, check for `isinstance(obj, S3Path)` or rely on duck typing if only standard pathlib methods are used.
• gotcha Implicit `boto3` session usage can lead to unexpected AWS credential behavior. If you don't explicitly attach a `boto3` session using `context.attach_boto_session()`, `s3pathlib` will rely on `boto3`'s default credential chain (environment variables, shared credential file, IAM roles), which might not be what you intend.
  Fix: Explicitly configure and attach your `boto3` session using `from s3pathlib import context; context.attach_boto_session(your_boto3_session)` to ensure predictable AWS authentication and region behavior.

Install

• pip install s3pathlib Install s3pathlib

Imports

• S3Path

  from s3pathlib import S3Path

• context

  from s3pathlib import context

  Used for attaching a boto3 session or configuring AWS credentials.

Quickstart

This quickstart demonstrates how to initialize `s3pathlib` with AWS credentials, create an `S3Path` object, write content to S3, and read it back. It shows how to use `context.attach_boto_session` for explicit credential management.

import boto3
import os
from s3pathlib import S3Path, context

# Configure AWS credentials (e.g., from environment variables or a profile)
# In a real application, consider using AWS IAM roles or proper credential management.
aws_region = os.environ.get("AWS_REGION", "us-east-1")
aws_profile = os.environ.get("AWS_PROFILE", None)

# Attach a boto3 session for s3pathlib to use
if aws_profile:
    session = boto3.session.Session(region_name=aws_region, profile_name=aws_profile)
else:
    session = boto3.session.Session(region_name=aws_region)
context.attach_boto_session(session)

# Define an S3 path object
bucket_name = os.environ.get("S3_BUCKET_NAME", "your-test-bucket-12345")
s3_path = S3Path(bucket_name, "my-folder", "hello.txt")

# Example: Write text to S3
print(f"Writing to {s3_path.uri}...")
s3_path.write_text("Hello, s3pathlib!")
print("Content written.")

# Example: Read text from S3
if s3_path.exists():
    content = s3_path.read_text()
    print(f"Content read from S3: '{content}'")

# Example: Check if a folder exists and list its contents
s3_dir = S3Path(bucket_name, "my-folder/")
print(f"Checking if {s3_dir.uri} exists: {s3_dir.exists()}")

# Clean up (optional)
# s3_path.delete() # Deletes the object
# s3_dir.delete_dir() # Deletes all objects under the prefix (careful!)

# Detach the boto3 session when done (optional, for explicit resource management)
context.detach_boto_session()