{"id":1986,"library":"dbt-bigquery","title":"dbt-bigquery","description":"dbt-bigquery is the BigQuery adapter plugin for dbt (data build tool). It enables data analysts and engineers to transform their data in Google BigQuery using SQL-based models and software engineering best practices like version control, testing, and documentation. It is currently at version 1.11.1 and follows the dbt Core release cadence.","status":"active","version":"1.11.1","language":"en","source_language":"en","source_url":"https://github.com/dbt-labs/dbt-adapters.git#subdirectory=dbt-bigquery","tags":["adapter","bigquery","dbt","elt","google","transformation","sql","data-warehouse"],"install":[{"cmd":"pip install dbt-core dbt-bigquery","lang":"bash","label":"Install dbt-bigquery and dbt-core"}],"dependencies":[{"reason":"dbt-bigquery is an adapter for dbt Core and requires it to function.","package":"dbt-core","optional":false},{"reason":"Required for authentication (gcloud auth) and interacting with Google Cloud services like BigQuery and Cloud Storage.","package":"Google Cloud SDK","optional":false},{"reason":"These APIs must be enabled in your Google Cloud Project for full dbt-bigquery functionality, especially for Python models and BigQuery DataFrames.","package":"BigQuery API, Cloud Storage API, Compute Engine API, Dataform API, Identity and Access Management API, Vertex AI API (GCP)","optional":false}],"imports":[{"note":"dbt-bigquery is an adapter plugin for dbt Core. Users primarily interact with it through the dbt Command Line Interface (CLI) and YAML configuration files (profiles.yml, dbt_project.yml), rather than direct Python import statements in their models or scripts.","symbol":"dbt-bigquery adapter is used via dbt CLI, not direct Python imports","correct":"dbt run"}],"quickstart":{"code":"# Create a profiles.yml file in ~/.dbt/ or your dbt project directory.\n# Ensure GCP_PROJECT_ID and GOOGLE_APPLICATION_CREDENTIALS (path to service account JSON) are set as environment variables.\n\n# ~/.dbt/profiles.yml\n# ---\n# my_bigquery_project:\n#   target: dev\n#   outputs:\n#     dev:\n#       type: bigquery\n#       method: service-account\n#       project: \"{{ env_var('GCP_PROJECT_ID', 'your-gcp-project-id') }}\"\n#       dataset: \"dbt_dev_dataset\" # This dataset must exist in BigQuery\n#       keyfile: \"{{ env_var('GOOGLE_APPLICATION_CREDENTIALS', 'path/to/your/service-account.json') }}\"\n#       threads: 4\n#       location: US # Or your BigQuery dataset location (e.g., EU, asia-northeast1)\n#       job_execution_timeout_seconds: 300\n# ---\n\n# 1. Initialize a new dbt project (if you don't have one):\n# dbt init my_new_dbt_project\n# cd my_new_dbt_project\n\n# 2. Edit dbt_project.yml to reference your profile (e.g., 'my_bigquery_project'):\n# name: 'my_new_dbt_project'\n# version: '1.0.0'\n# config-version: 2\n# profile: 'my_bigquery_project' # Must match a profile name in profiles.yml\n\n# 3. Create a sample SQL model: models/my_first_model.sql\n# ---\n# -- models/my_first_model.sql\n# SELECT\n#     current_timestamp() as current_time,\n#     'Hello dbt-bigquery!' as message\n# ---\n\n# 4. Set environment variables (replace with your actual values):\n# export GCP_PROJECT_ID=\"your-gcp-project-id\"\n# export GOOGLE_APPLICATION_CREDENTIALS=\"/path/to/your/service-account.json\"\n\n# 5. Run dbt commands:\n# dbt debug   # To verify your connection to BigQuery\n# dbt run     # To execute the model and create a view/table in BigQuery\n","lang":"bash","description":"This quickstart demonstrates how to set up a dbt project to connect to Google BigQuery using a service account. It involves creating a `profiles.yml` file, initializing a dbt project, creating a simple SQL model, and then running dbt commands. Ensure you have a Google Cloud Project with BigQuery enabled and a service account key file."},"warnings":[{"fix":"Always install with `pip install dbt-core dbt-bigquery`.","message":"Starting with dbt Core v1.8, installing a dbt adapter (like `dbt-bigquery`) no longer automatically installs `dbt-core`. You must explicitly install both `dbt-core` and `dbt-bigquery` to ensure a working environment.","severity":"breaking","affected_versions":">=1.8.0"},{"fix":"Explicitly set `job_execution_timeout_seconds` in your `profiles.yml` for all targets (e.g., `job_execution_timeout_seconds: 300` for a 5-minute timeout).","message":"The `job_execution_timeout_seconds` configuration (formerly `timeout_seconds` in older versions) in `profiles.yml` is critical to prevent BigQuery queries from running indefinitely and incurring high costs. BigQuery's default behavior may not impose a strict timeout.","severity":"gotcha","affected_versions":"All"},{"fix":"Ensure your service account or user credentials have all necessary BigQuery DataFrames-related IAM roles granted in GCP.","message":"BigQuery Python models and BigQuery DataFrames require additional IAM roles beyond basic BigQuery User/Data Editor, including BigQuery Job User, BigQuery Read Session User, Notebook Runtime User, Code Creator, and `colabEnterpriseUser`.","severity":"gotcha","affected_versions":"All"},{"fix":"Use a compatible Python version (e.g., Python 3.10, 3.11, 3.12, 3.13).","message":"Python 3.14 is currently not supported for `dbt-bigquery`. Users should use Python versions 3.13.x or older.","severity":"gotcha","affected_versions":"All"},{"fix":"Be aware of this BigQuery behavior when attempting 'dry run' inserts. Consider alternative validation methods for large tables.","message":"Using `LIMIT 0` with `INSERT INTO` statements in BigQuery does not optimize the query and can still result in the scanning of the entire source table, leading to unexpected billing costs.","severity":"gotcha","affected_versions":"All"},{"fix":"Review dbt logs for new warnings and correct any non-standard or deprecated configurations in your dbt project files.","message":"With dbt Core v1.11, default warnings for non-standard YAML configurations (e.g., misspelled config keys, invalid top-level properties) are now enabled. This can expose previously overlooked issues in `dbt_project.yml` or model configurations.","severity":"deprecated","affected_versions":">=1.11.0"}],"env_vars":null,"last_verified":"2026-04-09T00:00:00.000Z","next_check":"2026-07-08T00:00:00.000Z"}