{"id":5048,"library":"s3pathlib","title":"s3pathlib","description":"s3pathlib is a Python package that provides an intuitive, object-oriented programming (OOP) interface for manipulating AWS S3 objects and directories. Its API closely resembles the Python standard library's `pathlib` module, making S3 interactions feel familiar and Pythonic. The library is actively maintained, with the current version being 2.3.6, and receives regular minor updates.","status":"active","version":"2.3.6","language":"en","source_language":"en","source_url":"https://github.com/MacHu-GWU/s3pathlib-project","tags":["aws","s3","pathlib","cloud storage","boto3"],"install":[{"cmd":"pip install s3pathlib","lang":"bash","label":"Install s3pathlib"}],"dependencies":[{"reason":"Required for all interactions with AWS S3. It's an implicit dependency that s3pathlib leverages for its underlying AWS calls.","package":"boto3","optional":false}],"imports":[{"symbol":"S3Path","correct":"from s3pathlib import S3Path"},{"note":"Used for attaching a boto3 session or configuring AWS credentials.","symbol":"context","correct":"from s3pathlib import context"}],"quickstart":{"code":"import boto3\nimport os\nfrom s3pathlib import S3Path, context\n\n# Configure AWS credentials (e.g., from environment variables or a profile)\n# In a real application, consider using AWS IAM roles or proper credential management.\naws_region = os.environ.get(\"AWS_REGION\", \"us-east-1\")\naws_profile = os.environ.get(\"AWS_PROFILE\", None)\n\n# Attach a boto3 session for s3pathlib to use\nif aws_profile:\n    session = boto3.session.Session(region_name=aws_region, profile_name=aws_profile)\nelse:\n    session = boto3.session.Session(region_name=aws_region)\ncontext.attach_boto_session(session)\n\n# Define an S3 path object\nbucket_name = os.environ.get(\"S3_BUCKET_NAME\", \"your-test-bucket-12345\")\ns3_path = S3Path(bucket_name, \"my-folder\", \"hello.txt\")\n\n# Example: Write text to S3\nprint(f\"Writing to {s3_path.uri}...\")\ns3_path.write_text(\"Hello, s3pathlib!\")\nprint(\"Content written.\")\n\n# Example: Read text from S3\nif s3_path.exists():\n    content = s3_path.read_text()\n    print(f\"Content read from S3: '{content}'\")\n\n# Example: Check if a folder exists and list its contents\ns3_dir = S3Path(bucket_name, \"my-folder/\")\nprint(f\"Checking if {s3_dir.uri} exists: {s3_dir.exists()}\")\n\n# Clean up (optional)\n# s3_path.delete() # Deletes the object\n# s3_dir.delete_dir() # Deletes all objects under the prefix (careful!)\n\n# Detach the boto3 session when done (optional, for explicit resource management)\ncontext.detach_boto_session()","lang":"python","description":"This quickstart demonstrates how to initialize `s3pathlib` with AWS credentials, create an `S3Path` object, write content to S3, and read it back. It shows how to use `context.attach_boto_session` for explicit credential management."},"warnings":[{"fix":"Always capture the return value of write operations: `s3_path = s3_path.write_text('new content')`.","message":"S3Path objects are immutable. Operations that modify the S3 object (like `write_text()` or `write_bytes()`) will return a *new* S3Path object, especially in S3 versioning-enabled buckets. Always reassign the result if you need to work with the updated path object.","severity":"gotcha","affected_versions":"All versions"},{"fix":"To prevent accidental overwrites, explicitly check for file existence using `if not s3_path.exists():` before performing write operations.","message":"Write operations (`write_text()`, `write_bytes()`) will silently overwrite existing files by default. There is no automatic error or warning if the target S3 object already exists.","severity":"gotcha","affected_versions":"All versions"},{"fix":"Be mindful of trailing slashes (`/`) when defining S3 paths to denote directories, e.g., `S3Path('bucket', 'my-folder/')`. Use methods like `.to_dir()` if unsure.","message":"S3 does not have a true directory concept; `s3pathlib` provides a 'logical' or 'soft' directory abstraction. A path ending with `/` is treated as a directory. Understanding this distinction is crucial for directory-related operations.","severity":"gotcha","affected_versions":"All versions"},{"fix":"Avoid `isinstance(obj, pathlib.Path)` checks if you intend to support `S3Path`. Instead, check for `isinstance(obj, S3Path)` or rely on duck typing if only standard pathlib methods are used.","message":"While `s3pathlib` mimics the `pathlib.Path` API, `S3Path` is not a direct subclass of `pathlib.Path`. This means `isinstance(s3_path_object, pathlib.Path)` checks will return `False` and code expecting a `pathlib.Path` might break.","severity":"gotcha","affected_versions":"All versions"},{"fix":"Explicitly configure and attach your `boto3` session using `from s3pathlib import context; context.attach_boto_session(your_boto3_session)` to ensure predictable AWS authentication and region behavior.","message":"Implicit `boto3` session usage can lead to unexpected AWS credential behavior. If you don't explicitly attach a `boto3` session using `context.attach_boto_session()`, `s3pathlib` will rely on `boto3`'s default credential chain (environment variables, shared credential file, IAM roles), which might not be what you intend.","severity":"gotcha","affected_versions":"All versions"}],"env_vars":null,"last_verified":"2026-04-12T00:00:00.000Z","next_check":"2026-07-11T00:00:00.000Z"}