Google Cloud Dataform

0.9.0 · active · verified Sat Mar 28

Google Cloud Dataform is a service for developing, creating, documenting, testing, and updating curated tables in BigQuery. This Python client library (version 0.9.0) provides an interface to interact with the Dataform API. Google Cloud client libraries typically follow a regular release cadence, often coinciding with API updates or bug fixes, and are generally well-maintained.

Warnings

breaking As of January 19, 2026, Dataform workflows, BigQuery notebooks, pipelines, and data preparations enforce strict act-as mode at the project level. This requires using custom service accounts instead of the default Dataform service agent across all repositories and granting the Service Account User role (`roles/iam.serviceAccountUser`) to the default Dataform service agent and relevant principals to avoid failures.
Fix: Migrate to using custom service accounts for Dataform repositories and ensure `roles/iam.serviceAccountUser` is granted to them. Refer to Google Cloud documentation on 'Use strict act-as mode' and 'Verify act-as permissions'.
deprecated Legacy Dataform was deprecated on February 26, 2024. Projects running on legacy Dataform cannot be accessed after this date.
Fix: Migrate any existing legacy Dataform projects to Dataform in Google Cloud Platform. Refer to the official migration guide.
gotcha The `google-cloud-dataform` library is currently in `0.x.x` version series, indicating that its API surface might still undergo backward-incompatible changes between minor versions (e.g., 0.8.0 to 0.9.0). While Google aims for stability, pre-1.0 releases do not strictly adhere to semantic versioning guarantees.
Fix: Pin your dependency to specific patch versions (e.g., `google-cloud-dataform==0.9.0`) to prevent unexpected breakage during upgrades, and thoroughly test minor version updates.
gotcha The library's internal logging events (occurrence, level, content) may be refined without being flagged as breaking changes. Avoid depending on the immutability of these logging events.
Fix: Do not build critical system logic solely dependent on the exact format or content of library-generated log messages. Configure standard Python logging handlers to capture logs at appropriate levels.
breaking Python versions 3.6 and below are no longer supported by Google Cloud client libraries, including `google-cloud-dataform`.
Fix: Upgrade your Python environment to version 3.7 or higher. All current active and maintenance versions of Python (>=3.7) are compatible.

Install

pip install google-cloud-dataform Install latest version

Imports

DataformClient
```
from google.cloud import dataform_v1beta
client = dataform_v1beta.DataformClient()
```
Google Cloud client libraries typically organize clients under versioned submodules (e.g., `_v1beta`). Direct import from `google.cloud.dataform` is incorrect.

Quickstart

This quickstart initializes a Dataform client and lists all Dataform repositories within a specified Google Cloud project and location. Ensure your `GOOGLE_CLOUD_PROJECT` environment variable is set or replace 'your-project-id' with your actual project ID, and set the correct location. Authentication is handled automatically via Application Default Credentials (ADC).

import os
from google.cloud import dataform_v1beta

# Set your Google Cloud Project ID and Location
# For example, 'my-project-id' and 'us-central1'
project_id = os.environ.get('GOOGLE_CLOUD_PROJECT', 'your-project-id')
location = 'us-central1' # Replace with your Dataform repository location

def list_dataform_repositories(project_id: str, location: str):
    """Lists Dataform repositories in a given project and location."""
    client = dataform_v1beta.DataformClient()
    parent = f"projects/{project_id}/locations/{location}"

    print(f"Listing Dataform repositories in {parent}:")
    try:
        for repository in client.list_repositories(parent=parent):
            print(f"- Repository: {repository.name}")
    except Exception as e:
        print(f"Error listing repositories: {e}")
        print("Please ensure the Dataform API is enabled and your account has the necessary permissions.")

if __name__ == "__main__":
    list_dataform_repositories(project_id, location)

view raw JSON →