Snakemake Storage Plugin GCS

1.1.4 verified Sat May 09 auth: no python

A Snakemake storage plugin providing read/write access to Google Cloud Storage (GCS). Version 1.1.4, compatible with Snakemake ≥7.x plugin interface. Release cadence is irregular, tied to Snakemake plugin API updates.

pip install snakemake-storage-plugin-gcs

Common errors

error ModuleNotFoundError: No module named 'snakemake_storage_plugin_gcs' ↓

cause Plugin not installed or installed in wrong environment.

fix

Run pip install snakemake-storage-plugin-gcs in the same Python environment as Snakemake.

error google.api_core.exceptions.Forbidden: 403 GET https://storage.googleapis.com/... ↓

cause Missing or invalid GCS credentials for the bucket/resource.

fix

Ensure your service account key has 'Storage Object Viewer' (read) or 'Storage Object Creator' (write) permissions, and set GOOGLE_APPLICATION_CREDENTIALS to the correct key JSON file.

error snakemake.exceptions.WorkflowError: TypeError: StorageProviderGCS.__init__() got an unexpected keyword argument 'fallback' ↓

cause Incompatibility between plugin version and Snakemake version.

fix

Update Snakemake to >=7.0.0 or install a compatible plugin version.

Warnings

gotcha GCS authentication uses Application Default Credentials (ADC) by default. OAuth2 tokens from ADC may expire during long runs, causing failures. ↓

fix Use a service account key and set GOOGLE_APPLICATION_CREDENTIALS environment variable, or use workload identity federation for long-running workflows.

breaking Plugin version 1.x requires Snakemake >=7.0.0 plugin API. Not compatible with older Snakemake versions that used the legacy storage API (pre-7.0). ↓

fix Upgrade Snakemake to 7.x or later, or use an older storage plugin if locked to Snakemake 6.x (not available).

gotcha The plugin does not support GCS object versioning or generation numbers. If you need to retrieve a specific version, you must handle it separately. ↓

fix Use gsutil or google-cloud-storage API directly for versioned access.

Imports

StorageProviderGCS
```
from snakemake_storage_plugin_gcs import StorageProviderGCS
```
Direct import is rarely needed; plugin auto-registers when installed.

Quickstart

Basic usage: install plugin, set up GCS authentication, and use gs:// URIs in Snakemake rules. The plugin auto-detects gs:// prefixes.

import os
# Set authentication (uses Application Default Credentials)
os.environ['GOOGLE_APPLICATION_CREDENTIALS'] = '/path/to/service-account-key.json'
# Or use implicit default credentials
# In your Snakefile, reference files with gs:// prefix
# Example rule:
rule download:
    input:
        "gs://my-bucket/data.txt"
    output:
        "data.txt"
    shell:
        "cp {input} {output}"