gsutil

5.36 · maintenance · verified Sat Apr 11

gsutil is a Python application that serves as a command-line tool for interacting with Google Cloud Storage. It allows users to perform a wide range of bucket and object management tasks, such as creating and deleting buckets, uploading, downloading, and deleting objects, and managing ACLs. While `gsutil` is functional and widely documented, Google now recommends using `gcloud storage` commands from the Google Cloud CLI for Cloud Storage interactions due to better performance and support for newer features. The current version is 5.36, and it sees regular maintenance releases.

Warnings

breaking `gsutil` is a legacy Cloud Storage CLI and minimally maintained. Google strongly recommends migrating to `gcloud storage` commands for all Cloud Storage interactions.
Fix: Transition scripts and workflows to use `gcloud storage` commands. Refer to the 'Transition from gsutil to gcloud storage' documentation.
gotcha There is no official Python client library for `gsutil` itself. Programmatic interaction from Python is typically achieved by invoking `gsutil` commands via `subprocess` calls. For a native Python API, use `google-cloud-storage`.
Fix: For direct integration into Python applications, use the `google-cloud-storage` client library (e.g., `pip install google-cloud-storage`). If `gsutil` CLI functionality is strictly required, use Python's `subprocess` module to execute `gsutil` commands.
breaking `gsutil` dropped support for older Python versions. Versions 5.35 and later require Python 3.9 to 3.13.
Fix: Ensure your environment uses a compatible Python version (3.9-3.13 for recent `gsutil` versions). Upgrade Python if necessary.
gotcha Scripts that parse `gsutil` command output may break when transitioning to `gcloud storage` due to differences in output formatting, error messages, and data listings.
Fix: Thoroughly review and update any automated scripts that rely on parsing `gsutil` output if migrating to `gcloud storage`.
gotcha When transferring a large number of files or large single files, `gsutil`'s performance might require manual optimization, whereas `gcloud storage` is engineered for speed by default.
Fix: For optimal performance with large transfers, especially for new projects, consider using `gcloud storage`. If sticking with `gsutil`, explore options like the `-m` (multi-threaded/multi-processing) flag for `cp` and `rsync` operations, or adjusting `parallel_process_count` and `parallel_thread_count` settings in the `.boto` configuration.

Install

pip install gsutil Install via pip (Note: Google recommends installing via Google Cloud SDK)
gcloud components install gsutil Recommended: Install as part of Google Cloud CLI

Quickstart

This quickstart demonstrates how to interact with `gsutil` from Python by calling it as a subprocess. This is the common pattern for programmatic use, as `gsutil` is primarily a command-line interface. It includes creating a bucket, uploading a file, listing contents, downloading a file, and cleaning up resources.

import subprocess
import os

# Ensure gcloud is authenticated (e.g., via `gcloud auth login` in terminal)
# For programmatic use, consider setting GOOGLE_APPLICATION_CREDENTIALS
# os.environ['GOOGLE_APPLICATION_CREDENTIALS'] = '/path/to/keyfile.json'

bucket_name = os.environ.get('GSUTIL_TEST_BUCKET', 'my-test-bucket-12345')
file_name = 'test_file.txt'
file_content = 'Hello, Cloud Storage!'

# Create a local file to upload
with open(file_name, 'w') as f:
    f.write(file_content)

try:
    # Create a bucket
    print(f"Creating bucket gs://{bucket_name}...")
    subprocess.run(['gsutil', 'mb', f'gs://{bucket_name}'], check=True, capture_output=True)
    print(f"Bucket gs://{bucket_name} created.")

    # Upload a file
    print(f"Uploading {file_name} to gs://{bucket_name}/{file_name}...")
    subprocess.run(['gsutil', 'cp', file_name, f'gs://{bucket_name}/{file_name}'], check=True, capture_output=True)
    print("File uploaded.")

    # List bucket contents
    print(f"Listing contents of gs://{bucket_name}:")
    result = subprocess.run(['gsutil', 'ls', f'gs://{bucket_name}'], check=True, capture_output=True, text=True)
    print(result.stdout)

    # Download the file
    downloaded_file_name = 'downloaded_test_file.txt'
    print(f"Downloading gs://{bucket_name}/{file_name} to {downloaded_file_name}...")
    subprocess.run(['gsutil', 'cp', f'gs://{bucket_name}/{file_name}', downloaded_file_name], check=True, capture_output=True)
    print("File downloaded.")

    with open(downloaded_file_name, 'r') as f:
        content = f.read()
        print(f"Downloaded content: {content}")

except subprocess.CalledProcessError as e:
    print(f"gsutil command failed: {e}")
    print(f"STDOUT: {e.stdout.decode()}")
    print(f"STDERR: {e.stderr.decode()}")
finally:
    # Clean up (delete file and bucket)
    print(f"Cleaning up: deleting objects from gs://{bucket_name}...")
    subprocess.run(['gsutil', '-m', 'rm', '-r', f'gs://{bucket_name}/**'], check=False, capture_output=True)
    print(f"Deleting bucket gs://{bucket_name}...")
    subprocess.run(['gsutil', 'rb', f'gs://{bucket_name}'], check=False, capture_output=True)
    print("Cleanup complete.")
    if os.path.exists(file_name):
        os.remove(file_name)
    if os.path.exists(downloaded_file_name):
        os.remove(downloaded_file_name)

view raw JSON →