{"id":724,"library":"universal-pathlib","title":"Universal Pathlib","description":"Universal Pathlib is a Python library that extends the `pathlib.Path` interface to provide a unified, Pythonic way of interacting with a wide variety of backend filesystems, including local, cloud storage (S3, GCS, Azure), HTTP(S), and more, by leveraging `fsspec` backends. It is currently at version 0.3.10 and maintains a regular release cadence with frequent updates and bug fixes.","status":"active","version":"0.3.10","language":"python","source_language":"en","source_url":"https://github.com/fsspec/universal_pathlib","tags":["pathlib","filesystem","fsspec","cloud storage","object storage","remote filesystems"],"install":[{"cmd":"pip install universal-pathlib","lang":"bash","label":"Basic Installation"},{"cmd":"pip install universal-pathlib fsspec[s3,gcs]","lang":"bash","label":"With S3 and GCS support"}],"dependencies":[{"reason":"Core dependency for connecting to various filesystems.","package":"fsspec","optional":false},{"reason":"Required for AWS S3 filesystem support.","package":"s3fs","optional":true},{"reason":"Required for Google Cloud Storage (GCS) support.","package":"gcsfs","optional":true},{"reason":"Required for Azure Storage (Azure Data Lake Storage Gen1 and Gen2) support.","package":"adlfs","optional":true},{"reason":"Required for Hugging Face filesystem support.","package":"huggingface_hub","optional":true},{"reason":"Required for SFTP and SSH filesystems.","package":"paramiko","optional":true},{"reason":"Required for SMB filesystems.","package":"smbprotocol","optional":true},{"reason":"Required for WebDAV-based filesystems.","package":"webdav4[fsspec]","optional":true}],"imports":[{"symbol":"UPath","correct":"from upath import UPath"}],"quickstart":{"code":"import os\nfrom upath import UPath\n\n# Example for S3 (requires 's3fs' to be installed and AWS credentials configured)\n# You can use environment variables for credentials or pass storage_options\nS3_BUCKET = os.environ.get('S3_TEST_BUCKET', 'your-test-s3-bucket')\nS3_KEY = 'example.txt'\n\n# Create a UPath object for an S3 location\ns3path = UPath(f\"s3://{S3_BUCKET}\") / S3_KEY\n\n# Write content to the S3 file\ntry:\n    s3path.write_text('Hello from Universal Pathlib!')\n    print(f\"Successfully wrote to {s3path}.\")\n\n    # Read content from the S3 file\n    content = s3path.read_text()\n    print(f\"Content of {s3path}: '{content}'.\")\n\n    # Check if the path exists\n    print(f\"Does {s3path} exist? {s3path.exists()}.\")\n\n    # List contents of the parent directory (if bucket exists and is not empty)\n    print(f\"Listing contents of {s3path.parent}:\")\n    for p in s3path.parent.iterdir():\n        print(f\"  - {p}\")\n\nfinally:\n    # Clean up: delete the created file\n    if s3path.exists():\n        s3path.unlink()\n        print(f\"Cleaned up: deleted {s3path}.\")\n\n# Example for a local file (no additional fsspec backend needed)\nlocal_path = UPath(\"local_example.txt\")\nlocal_path.write_text(\"This is a local file.\")\nprint(f\"Local file content: {local_path.read_text()}\")\nlocal_path.unlink()","lang":"python","description":"This quickstart demonstrates how to use `UPath` for both a remote S3 bucket and a local file. It shows common operations like writing text, reading text, checking existence, and iterating directory contents. For S3, ensure `s3fs` is installed and AWS credentials are configured (e.g., via environment variables or `~/.aws/credentials`). For actual usage, replace `your-test-s3-bucket` with an accessible S3 bucket."},"warnings":[{"fix":"Migrate to subclassing `upath.extensions.ProxyUPath` for extending the API. Consult the official documentation for `ProxyUPath` usage examples.","message":"The `_protocol_dispatch=False` argument for extending the UPath API has been deprecated. Using it may lead to incorrect behavior or be removed in future versions.","severity":"breaking","affected_versions":">=0.3.4"},{"fix":"Ensure the protocol in the path string matches the `UPath` subclass being instantiated, or use the base `UPath` class which dispatches to the correct subclass based on the protocol.","message":"Attempting to instantiate a specific `UPath` subclass (e.g., `S3Path`) with a path string for a different protocol (e.g., `http://` or `file://`) will now raise a `TypeError` due to protocol incompatibility, rather than silently failing or using an incorrect backend.","severity":"gotcha","affected_versions":">=0.3.9"},{"fix":"Prioritize using protocols with explicitly supported `UPath` implementations listed in the documentation. If using an untested protocol, thoroughly test its behavior and consider contributing an implementation if issues arise.","message":"While `UPath` can attempt to handle any `fsspec`-compatible protocol, explicit implementations are tested and guaranteed. If no specialized `UPath` implementation exists for a given protocol, `UPath` will return a generic instance, but its behavior might not be fully reliable or tested.","severity":"gotcha","affected_versions":"All versions"},{"fix":"Ensure AWS credentials are correctly configured for the environment where the Python script is being run. Refer to the AWS documentation on configuring credentials for details (e.g., using environment variables, IAM roles, or the AWS CLI config file).","message":"Operations on S3Path (and potentially other cloud storage paths) require proper AWS credentials to be configured in the environment (e.g., via environment variables AWS_ACCESS_KEY_ID, AWS_SECRET_ACCESS_KEY, AWS_SESSION_TOKEN, or AWS config files).","severity":"breaking","affected_versions":"All versions"},{"fix":"Ensure that the execution environment has the necessary AWS credentials configured according to AWS best practices (e.g., AWS_ACCESS_KEY_ID, AWS_SECRET_ACCESS_KEY environment variables, or an ~/.aws/credentials file).","message":"When interacting with cloud storage services like S3, proper authentication credentials must be configured in the environment (e.g., AWS environment variables, credential files, or IAM roles). Without credentials, operations will fail with a `NoCredentialsError`.","severity":"breaking","affected_versions":"All versions"}],"env_vars":null,"last_verified":"2026-05-12T18:19:20.799Z","next_check":"2026-06-26T00:00:00.000Z","problems":[{"fix":"pip install universal-pathlib","cause":"The 'universal-pathlib' library has not been installed, or there's a typo in the import statement.","error":"ModuleNotFoundError: No module named 'upath'"},{"fix":"from upath import UPath","cause":"The user is attempting to import using the package installation name 'universal_pathlib' instead of the aliased module name 'upath'.","error":"ModuleNotFoundError: No module named 'universal_pathlib'"},{"fix":"from upath import UPath","cause":"The user is trying to import a class named 'Path' from the 'upath' module, but the correct class name is 'UPath'.","error":"ImportError: cannot import name 'Path' from 'upath'"},{"fix":"pip install s3fs","cause":"The required fsspec backend package for the specified protocol (e.g., 's3fs' for S3, 'gcsfs' for GCS) is not installed.","error":"fsspec.exceptions.NoBackendError: No backend found for 's3'"}],"ecosystem":"pypi","meta_description":null,"install_score":100,"install_tag":"verified","quickstart_score":0,"quickstart_tag":"stale","pypi_latest":"0.3.10","install_checks":{"last_tested":"2026-05-12","tag":"verified","tag_description":"installs cleanly on critical runtimes, fast import, recently tested","results":[{"runtime":"python:3.10-alpine","python_version":"3.10","os_libc":"alpine (musl)","variant":" $EXIT -eq 0 ","exit_code":0,"wheel_type":"wheel","failure_reason":null,"install_time_s":null,"import_time_s":0.23,"mem_mb":8.1,"disk_size":"20.2M"},{"runtime":"python:3.10-alpine","python_version":"3.10","os_libc":"alpine (musl)","variant":"s3,gcs","exit_code":0,"wheel_type":"sdist","failure_reason":null,"install_time_s":null,"import_time_s":0.2,"mem_mb":8.3,"disk_size":"118.1M"},{"runtime":"python:3.10-alpine","python_version":"3.10","os_libc":"alpine (musl)","variant":"default","exit_code":0,"wheel_type":null,"failure_reason":null,"install_time_s":null,"import_time_s":0.24,"mem_mb":8.1,"disk_size":"20.2M"},{"runtime":"python:3.10-alpine","python_version":"3.10","os_libc":"alpine (musl)","variant":"s3,gcs","exit_code":0,"wheel_type":null,"failure_reason":null,"install_time_s":null,"import_time_s":0.22,"mem_mb":8.3,"disk_size":"117.0M"},{"runtime":"python:3.10-slim","python_version":"3.10","os_libc":"slim (glibc)","variant":" $EXIT -eq 0 ","exit_code":0,"wheel_type":"wheel","failure_reason":null,"install_time_s":1.8,"import_time_s":0.17,"mem_mb":8.1,"disk_size":"21M"},{"runtime":"python:3.10-slim","python_version":"3.10","os_libc":"slim (glibc)","variant":"s3,gcs","exit_code":0,"wheel_type":"wheel","failure_reason":null,"install_time_s":13.2,"import_time_s":0.18,"mem_mb":8.2,"disk_size":"118M"},{"runtime":"python:3.10-slim","python_version":"3.10","os_libc":"slim (glibc)","variant":"default","exit_code":0,"wheel_type":null,"failure_reason":null,"install_time_s":null,"import_time_s":0.16,"mem_mb":8.1,"disk_size":"21M"},{"runtime":"python:3.10-slim","python_version":"3.10","os_libc":"slim (glibc)","variant":"s3,gcs","exit_code":0,"wheel_type":null,"failure_reason":null,"install_time_s":null,"import_time_s":0.17,"mem_mb":8.2,"disk_size":"117M"},{"runtime":"python:3.11-alpine","python_version":"3.11","os_libc":"alpine (musl)","variant":" $EXIT -eq 0 ","exit_code":0,"wheel_type":"wheel","failure_reason":null,"install_time_s":null,"import_time_s":0.3,"mem_mb":9.4,"disk_size":"22.6M"},{"runtime":"python:3.11-alpine","python_version":"3.11","os_libc":"alpine (musl)","variant":"s3,gcs","exit_code":0,"wheel_type":"sdist","failure_reason":null,"install_time_s":null,"import_time_s":0.36,"mem_mb":9.4,"disk_size":"125.8M"},{"runtime":"python:3.11-alpine","python_version":"3.11","os_libc":"alpine (musl)","variant":"default","exit_code":0,"wheel_type":null,"failure_reason":null,"install_time_s":null,"import_time_s":0.34,"mem_mb":9.4,"disk_size":"22.6M"},{"runtime":"python:3.11-alpine","python_version":"3.11","os_libc":"alpine (musl)","variant":"s3,gcs","exit_code":0,"wheel_type":null,"failure_reason":null,"install_time_s":null,"import_time_s":0.34,"mem_mb":9.4,"disk_size":"124.8M"},{"runtime":"python:3.11-slim","python_version":"3.11","os_libc":"slim (glibc)","variant":" $EXIT -eq 0 ","exit_code":0,"wheel_type":"wheel","failure_reason":null,"install_time_s":1.9,"import_time_s":0.27,"mem_mb":9.4,"disk_size":"23M"},{"runtime":"python:3.11-slim","python_version":"3.11","os_libc":"slim (glibc)","variant":"s3,gcs","exit_code":0,"wheel_type":"wheel","failure_reason":null,"install_time_s":11.8,"import_time_s":0.29,"mem_mb":9.4,"disk_size":"126M"},{"runtime":"python:3.11-slim","python_version":"3.11","os_libc":"slim (glibc)","variant":"default","exit_code":0,"wheel_type":null,"failure_reason":null,"install_time_s":null,"import_time_s":0.25,"mem_mb":9.4,"disk_size":"23M"},{"runtime":"python:3.11-slim","python_version":"3.11","os_libc":"slim (glibc)","variant":"s3,gcs","exit_code":0,"wheel_type":null,"failure_reason":null,"install_time_s":null,"import_time_s":0.26,"mem_mb":9.4,"disk_size":"125M"},{"runtime":"python:3.12-alpine","python_version":"3.12","os_libc":"alpine (musl)","variant":" $EXIT -eq 0 ","exit_code":0,"wheel_type":"wheel","failure_reason":null,"install_time_s":null,"import_time_s":0.49,"mem_mb":11.9,"disk_size":"14.3M"},{"runtime":"python:3.12-alpine","python_version":"3.12","os_libc":"alpine (musl)","variant":"s3,gcs","exit_code":0,"wheel_type":"sdist","failure_reason":null,"install_time_s":null,"import_time_s":0.57,"mem_mb":11.9,"disk_size":"116.8M"},{"runtime":"python:3.12-alpine","python_version":"3.12","os_libc":"alpine (musl)","variant":"default","exit_code":0,"wheel_type":null,"failure_reason":null,"install_time_s":null,"import_time_s":0.8,"mem_mb":11.9,"disk_size":"14.3M"},{"runtime":"python:3.12-alpine","python_version":"3.12","os_libc":"alpine (musl)","variant":"s3,gcs","exit_code":0,"wheel_type":null,"failure_reason":null,"install_time_s":null,"import_time_s":0.54,"mem_mb":11.9,"disk_size":"115.7M"},{"runtime":"python:3.12-slim","python_version":"3.12","os_libc":"slim (glibc)","variant":" $EXIT -eq 0 ","exit_code":0,"wheel_type":"wheel","failure_reason":null,"install_time_s":1.7,"import_time_s":0.46,"mem_mb":11.9,"disk_size":"15M"},{"runtime":"python:3.12-slim","python_version":"3.12","os_libc":"slim (glibc)","variant":"s3,gcs","exit_code":0,"wheel_type":"wheel","failure_reason":null,"install_time_s":9,"import_time_s":0.53,"mem_mb":11.9,"disk_size":"116M"},{"runtime":"python:3.12-slim","python_version":"3.12","os_libc":"slim (glibc)","variant":"default","exit_code":0,"wheel_type":null,"failure_reason":null,"install_time_s":null,"import_time_s":0.49,"mem_mb":11.9,"disk_size":"15M"},{"runtime":"python:3.12-slim","python_version":"3.12","os_libc":"slim (glibc)","variant":"s3,gcs","exit_code":0,"wheel_type":null,"failure_reason":null,"install_time_s":null,"import_time_s":0.5,"mem_mb":11.9,"disk_size":"116M"},{"runtime":"python:3.13-alpine","python_version":"3.13","os_libc":"alpine (musl)","variant":" $EXIT -eq 0 ","exit_code":0,"wheel_type":"wheel","failure_reason":null,"install_time_s":null,"import_time_s":0.5,"mem_mb":11.6,"disk_size":"14.1M"},{"runtime":"python:3.13-alpine","python_version":"3.13","os_libc":"alpine (musl)","variant":"s3,gcs","exit_code":0,"wheel_type":"sdist","failure_reason":null,"install_time_s":null,"import_time_s":0.52,"mem_mb":11.6,"disk_size":"116.4M"},{"runtime":"python:3.13-alpine","python_version":"3.13","os_libc":"alpine (musl)","variant":"default","exit_code":0,"wheel_type":null,"failure_reason":null,"install_time_s":null,"import_time_s":0.51,"mem_mb":11.6,"disk_size":"14.0M"},{"runtime":"python:3.13-alpine","python_version":"3.13","os_libc":"alpine (musl)","variant":"s3,gcs","exit_code":0,"wheel_type":null,"failure_reason":null,"install_time_s":null,"import_time_s":0.55,"mem_mb":11.6,"disk_size":"115.1M"},{"runtime":"python:3.13-slim","python_version":"3.13","os_libc":"slim (glibc)","variant":" $EXIT -eq 0 ","exit_code":0,"wheel_type":"wheel","failure_reason":null,"install_time_s":1.8,"import_time_s":0.45,"mem_mb":11.6,"disk_size":"15M"},{"runtime":"python:3.13-slim","python_version":"3.13","os_libc":"slim (glibc)","variant":"s3,gcs","exit_code":0,"wheel_type":"wheel","failure_reason":null,"install_time_s":8.9,"import_time_s":0.45,"mem_mb":11.6,"disk_size":"116M"},{"runtime":"python:3.13-slim","python_version":"3.13","os_libc":"slim (glibc)","variant":"default","exit_code":0,"wheel_type":null,"failure_reason":null,"install_time_s":null,"import_time_s":0.47,"mem_mb":11.6,"disk_size":"14M"},{"runtime":"python:3.13-slim","python_version":"3.13","os_libc":"slim (glibc)","variant":"s3,gcs","exit_code":0,"wheel_type":null,"failure_reason":null,"install_time_s":null,"import_time_s":0.49,"mem_mb":11.6,"disk_size":"115M"},{"runtime":"python:3.9-alpine","python_version":"3.9","os_libc":"alpine (musl)","variant":" $EXIT -eq 0 ","exit_code":0,"wheel_type":"wheel","failure_reason":null,"install_time_s":null,"import_time_s":0.17,"mem_mb":7.3,"disk_size":"19.7M"},{"runtime":"python:3.9-alpine","python_version":"3.9","os_libc":"alpine (musl)","variant":"s3,gcs","exit_code":0,"wheel_type":"sdist","failure_reason":null,"install_time_s":null,"import_time_s":0.19,"mem_mb":7.3,"disk_size":"95.7M"},{"runtime":"python:3.9-alpine","python_version":"3.9","os_libc":"alpine (musl)","variant":"default","exit_code":0,"wheel_type":null,"failure_reason":null,"install_time_s":null,"import_time_s":0.19,"mem_mb":7.3,"disk_size":"19.7M"},{"runtime":"python:3.9-alpine","python_version":"3.9","os_libc":"alpine (musl)","variant":"s3,gcs","exit_code":0,"wheel_type":null,"failure_reason":null,"install_time_s":null,"import_time_s":0.21,"mem_mb":7.3,"disk_size":"94.6M"},{"runtime":"python:3.9-slim","python_version":"3.9","os_libc":"slim (glibc)","variant":" $EXIT -eq 0 ","exit_code":0,"wheel_type":"wheel","failure_reason":null,"install_time_s":2.2,"import_time_s":0.15,"mem_mb":7.3,"disk_size":"20M"},{"runtime":"python:3.9-slim","python_version":"3.9","os_libc":"slim (glibc)","variant":"s3,gcs","exit_code":0,"wheel_type":"wheel","failure_reason":null,"install_time_s":13.7,"import_time_s":0.18,"mem_mb":7.3,"disk_size":"99M"},{"runtime":"python:3.9-slim","python_version":"3.9","os_libc":"slim (glibc)","variant":"default","exit_code":0,"wheel_type":null,"failure_reason":null,"install_time_s":null,"import_time_s":0.18,"mem_mb":7.3,"disk_size":"20M"},{"runtime":"python:3.9-slim","python_version":"3.9","os_libc":"slim (glibc)","variant":"s3,gcs","exit_code":0,"wheel_type":null,"failure_reason":null,"install_time_s":null,"import_time_s":0.18,"mem_mb":7.3,"disk_size":"98M"}]},"quickstart_checks":{"last_tested":"2026-04-24","tag":"stale","tag_description":"widespread failures or data too old to trust","results":[{"runtime":"python:3.10-alpine","exit_code":1},{"runtime":"python:3.10-slim","exit_code":1},{"runtime":"python:3.11-alpine","exit_code":1},{"runtime":"python:3.11-slim","exit_code":1},{"runtime":"python:3.12-alpine","exit_code":1},{"runtime":"python:3.12-slim","exit_code":1},{"runtime":"python:3.13-alpine","exit_code":1},{"runtime":"python:3.13-slim","exit_code":1},{"runtime":"python:3.9-alpine","exit_code":1},{"runtime":"python:3.9-slim","exit_code":1}]}}