s3cmd Command Line Tool

2.4.0 · active · verified Fri Apr 10

s3cmd is a free, feature-rich, command-line tool for managing Amazon S3 and other S3-compatible cloud storage services. Written in Python, it allows users to upload, retrieve, and manage data with simple shell commands. The current version is 2.4.0, and it follows an active maintenance release cadence.

Warnings

breaking Since version 2.0.0, AWS Signature Version 4 (SigV4) is the default and preferred signing method. If you have an existing `.s3cfg` configuration that explicitly forces AWS Signature Version 2 (`signature_v2 = True`), operations might fail or behave unexpectedly with newer AWS regions.
Fix: Review your `~/.s3cfg` file and remove or set `signature_v2 = False`. Ensure your AWS region supports SigV4 (most modern regions do). Re-run `s3cmd --configure` to generate a new, compliant configuration if needed.
gotcha s3cmd relies heavily on its configuration file, typically located at `~/.s3cfg`. Incorrect or missing configurations (e.g., access keys, secret keys, default region) will lead to authentication failures or unexpected behavior.
Fix: Run `s3cmd --configure` to create or update your `~/.s3cfg` file, providing your AWS Access Key, Secret Key, and optionally a default region. Ensure the file has appropriate permissions (e.g., `chmod 600 ~/.s3cfg`).
gotcha Starting with version 2.0.0, Server-Side Encryption (SSE-S3) is enabled by default for new object uploads. If you require unencrypted objects or a different encryption method, you must explicitly specify it.
Fix: To upload objects without SSE-S3 encryption, use the `--no-encryption` option. For other encryption types (e.g., SSE-C, SSE-KMS), refer to the s3cmd documentation for specific options like `--sse` or `--encrypt`.
gotcha When deleting objects or buckets, s3cmd often requires the `--force` option. Using `--force` can lead to irreversible data loss if used incorrectly, especially with recursive deletes or bucket removals.
Fix: Always double-check your `s3cmd del` or `s3cmd rm` commands before adding `--force`. Consider using the `--dry-run` option first to preview the operations that would be performed without actually executing them.
gotcha Connecting to non-AWS S3-compatible storage services (e.g., MinIO, DigitalOcean Spaces) requires specifying custom host and bucket host parameters, as defaults are for AWS S3.
Fix: Use the `--host` and `--host-bucket` options to point to the specific endpoint of your S3-compatible service. For example, `--host=sgp1.digitaloceanspaces.com --host-bucket=%(bucket)s.sgp1.digitaloceanspaces.com` for DigitalOcean Spaces.

Install

pip install s3cmd Install s3cmd

Imports

s3cmd CLI
```
import subprocess
subprocess.run(['s3cmd', 'version'], check=True)
```
s3cmd is primarily a command-line utility, not a Python library intended for direct programmatic import. To interact with it from Python, execute the `s3cmd` binary via the `subprocess` module.

Quickstart

Demonstrates how to interact with the s3cmd command-line tool from a Python script using the `subprocess` module to list S3 buckets. Users must ensure s3cmd is installed and configured with appropriate AWS credentials (e.g., via `s3cmd --configure` or environment variables).

import subprocess

# s3cmd requires prior configuration (e.g., via `s3cmd --configure` or AWS environment variables/IAM roles).
# This example assumes s3cmd is configured to access an S3 bucket.

# Example: List S3 buckets
try:
    # Using check=True will raise CalledProcessError if s3cmd exits with a non-zero status
    result = subprocess.run(['s3cmd', 'ls'], check=True, capture_output=True, text=True)
    print("S3 Buckets:\n", result.stdout)
except subprocess.CalledProcessError as e:
    print(f"Error listing buckets: {e.stderr}")
    print("Ensure s3cmd is correctly configured (e.g., with valid credentials in ~/.s3cfg).")
except FileNotFoundError:
    print("s3cmd command not found. Ensure it's installed and in your system's PATH.")

view raw JSON →