Google Cloud Dataform
raw JSON → 0.9.0 verified Tue May 12 auth: no python install: stale quickstart: stale
Google Cloud Dataform is a service for developing, creating, documenting, testing, and updating curated tables in BigQuery. This Python client library (version 0.9.0) provides an interface to interact with the Dataform API. Google Cloud client libraries typically follow a regular release cadence, often coinciding with API updates or bug fixes, and are generally well-maintained.
pip install google-cloud-dataform Common errors
error Access Denied: Project PROJECT_ID : User does not have bigquery.jobs.create permission in project PROJECT_ID. ↓
cause The Dataform service account or the user triggering the Dataform job lacks the necessary IAM permissions to create BigQuery jobs in the specified project.
fix
Grant the Dataform service account (or your user account, if using user credentials) the
BigQuery Job User role or the specific bigquery.jobs.create permission on the Google Cloud project. error An error occurred during computation of the authentication status ↓
cause Dataform is unable to authenticate with the linked Git repository, often due to an incorrect, expired, or unauthorized SSH key or access token, or network restrictions.
fix
Verify the Git access token or SSH private key stored in Secret Manager, ensure it has the correct permissions for the repository, check for extra spaces/line breaks in the secret, and confirm no firewall rules are blocking SSH (port 22) or HTTPS traffic. If using SAML SSO, ensure the SSH key is authorized in the Git provider.
error Failed to find the transpiler exported from @dataform/core. Ensure packages are installed and upgrade to a recent version. ↓
cause The `@dataform/core` package is either missing, corrupted, or an incompatible version is installed, preventing Dataform from compiling SQLX files.
fix
Ensure that
npm install has been run in your Dataform project directory to install dependencies, and explicitly set a recent, compatible version of @dataform/core (e.g., "@dataform/core": "3.0.0" or later) in your package.json file, then reinstall packages. error Quota exceeded: Your user_method exceeded quota for concurrent api requests per user per method. ↓
cause The number of concurrent API requests that Dataform sends to BigQuery has exceeded the BigQuery API quota for your project or user, leading to throttling.
fix
Reduce the number of parallel queries in Dataform by categorizing actions with tags and running only selected tags, introducing dependencies between actions, or dividing executions of actions between different Google Cloud projects.
error Dataform encountered an error: Unexpected property "type", or property value type of "string" is incorrect. ↓
cause This error typically occurs when there is a syntax or type mismatch in the configuration of a Dataform file (e.g., `sqlx`, `yaml`, or `json`), especially when a property expects a specific type but receives another.
fix
Review the Dataform configuration files for syntax errors, ensuring that all properties (like
type, tags, etc.) have correctly formatted values and are placed in valid contexts according to Dataform's documentation. Warnings
breaking As of January 19, 2026, Dataform workflows, BigQuery notebooks, pipelines, and data preparations enforce strict act-as mode at the project level. This requires using custom service accounts instead of the default Dataform service agent across all repositories and granting the Service Account User role (`roles/iam.serviceAccountUser`) to the default Dataform service agent and relevant principals to avoid failures. ↓
fix Migrate to using custom service accounts for Dataform repositories and ensure `roles/iam.serviceAccountUser` is granted to them. Refer to Google Cloud documentation on 'Use strict act-as mode' and 'Verify act-as permissions'.
deprecated Legacy Dataform was deprecated on February 26, 2024. Projects running on legacy Dataform cannot be accessed after this date. ↓
fix Migrate any existing legacy Dataform projects to Dataform in Google Cloud Platform. Refer to the official migration guide.
gotcha The `google-cloud-dataform` library is currently in `0.x.x` version series, indicating that its API surface might still undergo backward-incompatible changes between minor versions (e.g., 0.8.0 to 0.9.0). While Google aims for stability, pre-1.0 releases do not strictly adhere to semantic versioning guarantees. ↓
fix Pin your dependency to specific patch versions (e.g., `google-cloud-dataform==0.9.0`) to prevent unexpected breakage during upgrades, and thoroughly test minor version updates.
gotcha The library's internal logging events (occurrence, level, content) may be refined without being flagged as breaking changes. Avoid depending on the immutability of these logging events. ↓
fix Do not build critical system logic solely dependent on the exact format or content of library-generated log messages. Configure standard Python logging handlers to capture logs at appropriate levels.
breaking Python versions 3.6 and below are no longer supported by Google Cloud client libraries, including `google-cloud-dataform`. ↓
fix Upgrade your Python environment to version 3.7 or higher. All current active and maintenance versions of Python (>=3.7) are compatible.
breaking The required `google-cloud-dataform` client library, or the specific version containing the `dataform_v1beta` submodule, may not be installed or is incompatible with the import statement. Importing client components directly from `google.cloud` (e.g., `from google.cloud import dataform_v1beta`) depends on the individual client library being correctly installed and compatible. ↓
fix Ensure the `google-cloud-dataform` library is installed using `pip install google-cloud-dataform`. If already installed, ensure it is up-to-date or compatible with the desired API version (`dataform_v1beta`). Verify installed packages with `pip freeze` and consult the library's documentation or changelog for module availability and import paths.
gotcha Attempting to import `dataform_v1beta` directly from `google.cloud` (e.g., `from google.cloud import dataform_v1beta`) will result in an `ImportError`. The Dataform v1beta client is typically found within its own subpackage. ↓
fix Use `from google.cloud.dataform_v1beta import DataformClient` to import the Dataform v1beta client, or `from google.cloud import dataform` for the stable client if that is desired, followed by `dataform.DataformClient`.
Install compatibility stale last tested: 2026-05-12
python os / libc status wheel install import disk
3.10 alpine (musl) - - - -
3.10 slim (glibc) - - - -
3.11 alpine (musl) - - - -
3.11 slim (glibc) - - - -
3.12 alpine (musl) - - - -
3.12 slim (glibc) - - - -
3.13 alpine (musl) - - - -
3.13 slim (glibc) - - - -
3.9 alpine (musl) - - - -
3.9 slim (glibc) - - - -
Imports
- DataformClient wrong
from google.cloud.dataform import DataformClientcorrectfrom google.cloud import dataform_v1beta client = dataform_v1beta.DataformClient()
Quickstart stale last tested: 2026-04-23
import os
from google.cloud import dataform_v1beta
# Set your Google Cloud Project ID and Location
# For example, 'my-project-id' and 'us-central1'
project_id = os.environ.get('GOOGLE_CLOUD_PROJECT', 'your-project-id')
location = 'us-central1' # Replace with your Dataform repository location
def list_dataform_repositories(project_id: str, location: str):
"""Lists Dataform repositories in a given project and location."""
client = dataform_v1beta.DataformClient()
parent = f"projects/{project_id}/locations/{location}"
print(f"Listing Dataform repositories in {parent}:")
try:
for repository in client.list_repositories(parent=parent):
print(f"- Repository: {repository.name}")
except Exception as e:
print(f"Error listing repositories: {e}")
print("Please ensure the Dataform API is enabled and your account has the necessary permissions.")
if __name__ == "__main__":
list_dataform_repositories(project_id, location)