{"id":7197,"library":"dvc-gs","title":"dvc-gs: Google Cloud Storage Plugin for DVC","description":"dvc-gs is the official Google Cloud Storage plugin for DVC (Data Version Control). It enables DVC to store and retrieve data artifacts from Google Cloud Storage buckets, allowing users to version large files and models in the cloud. The current version is 3.0.2, and it typically follows DVC's release cadence, with frequent updates to align with core DVC features and bug fixes.","status":"active","version":"3.0.2","language":"en","source_language":"en","source_url":"https://github.com/iterative/dvc-gs","tags":["dvc","data version control","google cloud storage","cloud storage","plugin","mlops","gcs"],"install":[{"cmd":"pip install dvc-gs","lang":"bash","label":"Install plugin only"},{"cmd":"pip install dvc[gs]","lang":"bash","label":"Install DVC with GS support"}],"dependencies":[{"reason":"This is a plugin for DVC, providing the Google Cloud Storage backend.","package":"dvc","optional":false},{"reason":"Provides the underlying filesystem interface for Google Cloud Storage. Automatically installed with `dvc-gs` or `dvc[gs]`.","package":"gcsfs","optional":false}],"imports":[{"note":"Users typically interact with dvc-gs functionality indirectly through DVC's CLI or dvc.api, rather than direct Python imports from the dvc_gs package itself. dvc-gs provides the backend filesystem implementation.","symbol":"dvc.api","correct":"import dvc.api\n# dvc-gs functionality is implicitly used by dvc.api calls\n# when a Google Cloud Storage remote is configured."}],"quickstart":{"code":"import os\nfrom dvc.repo import Repo\n\n# Initialize DVC in a new directory\nos.makedirs('my_project', exist_ok=True)\nos.chdir('my_project')\nrepo = Repo.init()\n\n# Configure a Google Cloud Storage remote\n# Replace 'your-gcs-bucket' with your actual bucket name\n# Ensure GOOGLE_APPLICATION_CREDENTIALS points to a service account key or use gcloud auth\nif not os.environ.get('GOOGLE_APPLICATION_CREDENTIALS'):\n    print(\"Warning: GOOGLE_APPLICATION_CREDENTIALS not set. Ensure gcloud is authenticated or anonymous access is allowed for the bucket.\")\n\nrepo.remote.add(\n    name='my_gs_remote',\n    url=f'gs://{os.environ.get(\"GCS_BUCKET_NAME\", \"your-gcs-bucket\")}'\n)\n\n# Create a dummy data file\nwith open('data.txt', 'w') as f:\n    f.write('hello dvc-gs')\n\n# Add the file to DVC and push to GCS\nrepo.add('data.txt')\nrepo.push('data.txt')\nprint(\"data.txt added and pushed to GCS.\")\n\n# To verify, you can pull the data into another location or check your GCS bucket\n# For example, in a new directory:\n# os.chdir('..')\n# os.makedirs('another_project', exist_ok=True)\n# os.chdir('another_project')\n# Repo.clone('path/to/my_project', 'another_project')\n# new_repo = Repo('.')\n# new_repo.pull('data.txt')","lang":"python","description":"This quickstart demonstrates how to initialize a DVC repository, configure a Google Cloud Storage remote using `dvc-gs`, add a data file, and push it to the GCS bucket. It assumes DVC is installed and you have appropriate GCS credentials configured (e.g., `GOOGLE_APPLICATION_CREDENTIALS` environment variable or `gcloud` authenticated). Replace `your-gcs-bucket` with your actual bucket name."},"warnings":[{"fix":"Consult the DVC 3.0 migration guide for updated API usage. For example, `dvc.api.get_url()` now requires `url` and `path` arguments.","message":"DVC 3.0.0 (and consequently dvc-gs 3.0.0) introduced significant changes to the DVC API, particularly affecting programmatic usage with `dvc.api.get_url()`, `dvc.api.open()`, and `dvc.api.read()`. Code written for DVC 2.x will likely break with DVC 3.x.","severity":"breaking","affected_versions":"3.0.0+"},{"fix":"Ensure your environment is correctly authenticated with Google Cloud. For programmatic access, `GOOGLE_APPLICATION_CREDENTIALS` pointing to a service account key JSON file is recommended. For CLI, `gcloud auth login` and `gcloud auth application-default login` are typical.","message":"DVC-GS relies on Google Cloud credentials. Without proper authentication, operations will fail with permission errors. Common methods include setting `GOOGLE_APPLICATION_CREDENTIALS` or using `gcloud auth application-default login`.","severity":"gotcha","affected_versions":"All"},{"fix":"Check your `dvc-gs` version. If older than 3.0.2, upgrade. Always verify bucket permissions and access policies on Google Cloud Platform, even for publicly accessible buckets, to ensure DVC has the necessary read/write permissions.","message":"While `dvc-gs` version 3.0.2 introduced support for anonymous GCS login, previous versions or specific bucket configurations might still require explicit authentication even for public data, leading to unexpected `AccessDenied` errors.","severity":"gotcha","affected_versions":"<3.0.2, some 3.0.2+ configurations"}],"env_vars":null,"last_verified":"2026-04-16T00:00:00.000Z","next_check":"2026-07-15T00:00:00.000Z","problems":[{"fix":"Install the plugin: `pip install dvc-gs` or `pip install dvc[gs]`.","cause":"The dvc-gs plugin is not installed or DVC cannot find it. This usually happens if you install `dvc` but forget `dvc-gs` or `dvc[gs]`.","error":"ERROR: Plugin 'gs' is not found. Check if the plugin is installed."},{"fix":"Set the `GOOGLE_APPLICATION_CREDENTIALS` environment variable to the path of your service account key file, or run `gcloud auth application-default login` to set up user credentials.","cause":"DVC-GS (via gcsfs) could not find any Google Cloud credentials in the environment.","error":"gcsfs.exceptions.NoCredentialsError: NoCredentialsError: Could not determine credentials. Please set GOOGLE_APPLICATION_CREDENTIALS, provide explicit credentials, or load them from gcloud config."},{"fix":"Double-check the GCS bucket URL for typos. Verify that the bucket exists and that the configured credentials have appropriate read/write access to it.","cause":"This generic error often indicates issues with the remote configuration (e.g., typo in URL, bucket doesn't exist, or permissions issues even if credentials are found).","error":"dvc.api.DvcException: 'gs://your-gcs-bucket' is not a valid DVC remote or cannot be accessed."}]}