{"id":478,"library":"dbt-core","title":"dbt-core","description":"dbt-core is the open-source foundation of dbt (data build tool), empowering data analysts and engineers to transform data in their warehouses using SQL, following software engineering best practices. It's a powerful CLI tool for building, testing, documenting, and deploying data models. `dbt-core` is currently at version 1.11.7 and adheres to semantic versioning, with minor versions being backward compatible and frequent patch releases for bug fixes.","status":"active","version":"1.11.7","language":"python","source_language":"en","source_url":"https://github.com/dbt-labs/dbt-core.git","tags":["data transformation","etl","elt","data modeling","sql","analytics engineering","cli"],"install":[{"cmd":"pip install dbt-core","lang":"bash","label":"Install dbt-core"},{"cmd":"pip install dbt-core dbt-<adapter_name>","lang":"bash","label":"Install dbt-core with a specific adapter (e.g., dbt-snowflake)"}],"dependencies":[{"reason":"Required to run dbt-core. PyPI metadata lists >=3.10.","package":"python","optional":false},{"reason":"Provides the base classes and utilities for dbt adapters.","package":"dbt-adapters","optional":false},{"reason":"Provides common utilities and shared code across dbt packages.","package":"dbt-common","optional":false},{"reason":"An adapter for a specific data warehouse (e.g., dbt-snowflake, dbt-bigquery). Essential for dbt to connect to a database. Must be installed separately.","package":"dbt-<adapter_name>","optional":true}],"imports":[{"note":"This is the primary programmatic entry point for invoking dbt CLI commands within Python. Available since dbt Core v1.5.","symbol":"dbtRunner","correct":"from dbt.cli.main import dbtRunner"},{"note":"Used to inspect the results of a dbtRunner invocation.","symbol":"dbtRunnerResult","correct":"from dbt.cli.main import dbtRunnerResult"}],"quickstart":{"code":"import os\nfrom dbt.cli.main import dbtRunner, dbtRunnerResult\n\n# NOTE: For this to run, you must have a dbt project initialized\n# (e.g., with 'dbt init <project_name>') and a profiles.yml configured.\n# Set the DBT_PROFILES_DIR environment variable or ensure 'dbt_project.yml'\n# is in the current working directory or a parent directory.\n# A simple dbt_project.yml might look like:\n# name: my_dbt_project\n# version: '1.0.0'\n# config-version: 2\n# profile: my_profile\n\n# Initialize the dbt runner\ndbt = dbtRunner()\n\n# Example: Run all models in your dbt project\n# You might need to set the project directory explicitly if not in CWD\nproject_directory = os.environ.get('DBT_PROJECT_DIR', os.getcwd())\ncli_args = [\"run\", \"--project-dir\", project_directory]\n\nprint(f\"Invoking dbt run with arguments: {cli_args}\")\nres: dbtRunnerResult = dbt.invoke(cli_args)\n\nif res.success:\n    print(\"dbt run completed successfully.\")\n    if res.result:\n        for r in res.result:\n            print(f\"  Model: {r.node.name}, Status: {r.status}\")\nelse:\n    print(\"dbt run failed.\")\n    if res.exception:\n        print(f\"Exception: {res.exception}\")\n    elif res.result:\n        # Some commands might return partial results even on failure\n        print(\"Partial results or errors:\")\n        for r in res.result:\n            print(f\"  Model: {r.node.name}, Status: {r.status}, Message: {r.message}\")","lang":"python","description":"This quickstart demonstrates how to programmatically invoke dbt CLI commands using the `dbtRunner` class. It shows how to run all models in an existing dbt project and process the results. A `dbt_project.yml` and `profiles.yml` must be set up for this to execute successfully. The `dbtRunner` class was introduced in dbt Core v1.5 as the official Python entry point."},"warnings":[{"fix":"Always explicitly install both `dbt-core` and your desired adapter: `pip install dbt-core dbt-<adapter_name>`.","message":"Explicit installation of both `dbt-core` and adapter plugins is now recommended. Prior to v1.8, installing an adapter (e.g., `dbt-snowflake`) implicitly installed `dbt-core`. While this implicit behavior is maintained for backward compatibility currently, it may be removed in future versions.","severity":"breaking","affected_versions":">=1.8"},{"fix":"To achieve parallel execution safely, run dbt commands in separate processes (e.g., using `subprocess`, `Celery`) or utilize tools like the dbt CLI or dbt Cloud IDE which manage concurrency.","message":"Programmatic `dbtRunner` invocations in the same Python process do not support safe parallel execution. Running multiple dbt commands concurrently within a single process is discouraged due to potential interactions with global Python variables and data platform conflicts.","severity":"gotcha","affected_versions":">=1.5"},{"fix":"Move global config flags from `profiles.yml` to the `flags` dictionary in `dbt_project.yml`.","message":"Setting custom global config flags in `profiles.yml` has been deprecated. These flags should now be configured in `dbt_project.yml`.","severity":"deprecated","affected_versions":">=1.8"},{"fix":"Always consult the official dbt Developer Hub migration guides and changelogs before upgrading to a new major version.","message":"Major versions (e.g., v1 to v2) of dbt Core may introduce breaking changes, including the removal of deprecated functionality. Minor versions (e.g., v1.8 to v1.9) are generally backward compatible for documented features.","severity":"breaking","affected_versions":"All major version upgrades"},{"fix":"Review and address deprecation warnings promptly. Ensure `dbt_project.yml` and YAML property files adhere to recommended structures and avoid standalone anchor definitions at the top level.","message":"Beginning in v1.10, dbt Core will issue deprecation warnings for invalid dbt code, including custom inputs, duplicate YAML keys, and unexpected Jinja blocks, which will become invalid in future versions.","severity":"deprecated","affected_versions":">=1.10"},{"fix":"Be aware of your data platform's Python runtime capabilities and limitations when developing dbt Python models. Do not assume local Python environment features are available remotely.","message":"dbt Python models (`.py` files) are executed remotely on the configured data platform (e.g., Snowflake, BigQuery, Databricks), not on the local machine where `dbt-core` is running. This means any Python code must be compatible with the platform's execution environment and its limitations (e.g., external API calls might not be supported).","severity":"gotcha","affected_versions":">=1.3 (when Python models were introduced)"},{"fix":"Review the script or command executed immediately after `pip install`. Ensure all placeholders like `<adapter_name>` are correctly substituted, and all referenced files or commands exist in the execution environment. This error is external to dbt-core itself.","message":"A shell error `sh: 1: cannot open <name>: No such file` indicates that a command executed after `dbt-core` installation tried to reference a non-existent file or command. This often happens when placeholders (e.g., `<adapter_name>`) are not replaced with actual values, or when a script expects a file that is not present.","severity":"gotcha","affected_versions":"All versions"}],"env_vars":null,"last_verified":"2026-05-12T14:10:36.190Z","next_check":"2026-06-26T00:00:00.000Z","problems":[{"fix":"Carefully check the specified file for missing commas, incorrect spelling, capitalization issues, unclosed Jinja blocks (e.g., {% endmacro %}), or invalid ref/source functions. Use `dbt compile` to isolate and debug issues before running.","cause":"This error occurs when dbt cannot parse the code in your .sql or .yml files due to syntax issues in Jinja or SQL, or incorrect YAML formatting.","error":"Compilation Error"},{"fix":"Ensure a `profiles.yml` file exists in `~/.dbt/` (or the directory specified by `DBT_PROFILES_DIR`), verify the profile name in `dbt_project.yml` exactly matches a profile in `profiles.yml`, and confirm credentials are correct. Run `dbt debug --config-dir` to locate your `profiles.yml` and `dbt debug` to test the connection.","cause":"dbt cannot locate the specified profile in your profiles.yml file, or the profile: key in dbt_project.yml does not match an existing profile, or there are issues with credentials/authentication.","error":"Runtime Error Could not find profile named '...'"},{"fix":"Reinstall `dbt-adapters` and your specific adapter (e.g., `dbt-bigquery`) forcefully using `pip install --force-reinstall dbt-adapters dbt-<your_adapter>`. It's recommended to explicitly install both `dbt-core` and the desired adapter, and consider reinstalling in a fresh virtual environment if issues persist.","cause":"This typically occurs after upgrading dbt-core to version 1.8 or higher, where dbt-core and its adapters were decoupled, leading to missing or incorrectly installed adapter dependencies.","error":"ModuleNotFoundError: No module named 'dbt.adapters.factory'"},{"fix":"Navigate to the root directory of your dbt project where the `.git` folder is located. If the project is not a Git repository, initialize it with `git init` or clone an existing repository. If the `.git` folder is corrupted, you might need to reinitialize or restore it.","cause":"This error indicates that a Git command was attempted in a directory that is not part of a Git repository, or the .git folder is missing or corrupted.","error":"fatal: Not a git repository (or any of the parent directories): .git"},{"fix":"Review the SQL in the problematic model. Check the `target/compiled` directory to see the fully compiled SQL that dbt sends to the warehouse. Run this compiled SQL directly in your data warehouse's query editor to pinpoint the exact syntax issue using the warehouse's error messages.","cause":"dbt successfully compiles the Jinja and dbt-specific syntax, but the resulting SQL code contains a syntax error that the target data warehouse cannot execute. This often happens due to typos, incorrect functions for the specific database, or incompatible data types.","error":"Database Error: SQL compilation error: syntax error"}],"ecosystem":"pypi","meta_description":null,"install_score":85,"install_tag":"verified","quickstart_score":80,"quickstart_tag":"verified","pypi_latest":null,"install_checks":{"last_tested":"2026-05-12","tag":"verified","tag_description":"installs cleanly on critical runtimes, fast import, recently tested","results":[{"runtime":"python:3.10-alpine","python_version":"3.10","os_libc":"alpine (musl)","variant":"default","exit_code":0,"wheel_type":null,"failure_reason":null,"install_time_s":null,"import_time_s":7.99,"mem_mb":81.7,"disk_size":"113.7M"},{"runtime":"python:3.10-alpine","python_version":"3.10","os_libc":"alpine (musl)","variant":"default","exit_code":1,"wheel_type":null,"failure_reason":null,"install_time_s":null,"import_time_s":null,"mem_mb":null,"disk_size":null},{"runtime":"python:3.10-slim","python_version":"3.10","os_libc":"slim (glibc)","variant":"default","exit_code":0,"wheel_type":null,"failure_reason":null,"install_time_s":null,"import_time_s":4.89,"mem_mb":71.8,"disk_size":"114M"},{"runtime":"python:3.10-slim","python_version":"3.10","os_libc":"slim (glibc)","variant":"default","exit_code":1,"wheel_type":null,"failure_reason":null,"install_time_s":null,"import_time_s":null,"mem_mb":null,"disk_size":null},{"runtime":"python:3.11-alpine","python_version":"3.11","os_libc":"alpine (musl)","variant":"default","exit_code":0,"wheel_type":null,"failure_reason":null,"install_time_s":null,"import_time_s":9.81,"mem_mb":86,"disk_size":"125.6M"},{"runtime":"python:3.11-alpine","python_version":"3.11","os_libc":"alpine (musl)","variant":"default","exit_code":1,"wheel_type":null,"failure_reason":null,"install_time_s":null,"import_time_s":null,"mem_mb":null,"disk_size":null},{"runtime":"python:3.11-slim","python_version":"3.11","os_libc":"slim (glibc)","variant":"default","exit_code":0,"wheel_type":null,"failure_reason":null,"install_time_s":null,"import_time_s":6.92,"mem_mb":76.1,"disk_size":"126M"},{"runtime":"python:3.11-slim","python_version":"3.11","os_libc":"slim (glibc)","variant":"default","exit_code":1,"wheel_type":null,"failure_reason":null,"install_time_s":null,"import_time_s":null,"mem_mb":null,"disk_size":null},{"runtime":"python:3.12-alpine","python_version":"3.12","os_libc":"alpine (musl)","variant":"default","exit_code":0,"wheel_type":null,"failure_reason":null,"install_time_s":null,"import_time_s":9.28,"mem_mb":84.7,"disk_size":"115.3M"},{"runtime":"python:3.12-alpine","python_version":"3.12","os_libc":"alpine (musl)","variant":"default","exit_code":1,"wheel_type":null,"failure_reason":null,"install_time_s":null,"import_time_s":null,"mem_mb":null,"disk_size":null},{"runtime":"python:3.12-slim","python_version":"3.12","os_libc":"slim (glibc)","variant":"default","exit_code":0,"wheel_type":null,"failure_reason":null,"install_time_s":null,"import_time_s":8.08,"mem_mb":74.9,"disk_size":"116M"},{"runtime":"python:3.12-slim","python_version":"3.12","os_libc":"slim (glibc)","variant":"default","exit_code":1,"wheel_type":null,"failure_reason":null,"install_time_s":null,"import_time_s":null,"mem_mb":null,"disk_size":null},{"runtime":"python:3.13-alpine","python_version":"3.13","os_libc":"alpine (musl)","variant":"default","exit_code":0,"wheel_type":null,"failure_reason":null,"install_time_s":null,"import_time_s":7.94,"mem_mb":84.1,"disk_size":"114.9M"},{"runtime":"python:3.13-alpine","python_version":"3.13","os_libc":"alpine (musl)","variant":"default","exit_code":1,"wheel_type":null,"failure_reason":null,"install_time_s":null,"import_time_s":null,"mem_mb":null,"disk_size":null},{"runtime":"python:3.13-slim","python_version":"3.13","os_libc":"slim (glibc)","variant":"default","exit_code":0,"wheel_type":null,"failure_reason":null,"install_time_s":null,"import_time_s":6.7,"mem_mb":74.1,"disk_size":"115M"},{"runtime":"python:3.13-slim","python_version":"3.13","os_libc":"slim (glibc)","variant":"default","exit_code":1,"wheel_type":null,"failure_reason":null,"install_time_s":null,"import_time_s":null,"mem_mb":null,"disk_size":null},{"runtime":"python:3.9-alpine","python_version":"3.9","os_libc":"alpine (musl)","variant":"default","exit_code":0,"wheel_type":null,"failure_reason":null,"install_time_s":null,"import_time_s":7.87,"mem_mb":79.3,"disk_size":"112.1M"},{"runtime":"python:3.9-alpine","python_version":"3.9","os_libc":"alpine (musl)","variant":"default","exit_code":1,"wheel_type":null,"failure_reason":null,"install_time_s":null,"import_time_s":null,"mem_mb":null,"disk_size":null},{"runtime":"python:3.9-slim","python_version":"3.9","os_libc":"slim (glibc)","variant":"default","exit_code":0,"wheel_type":null,"failure_reason":null,"install_time_s":null,"import_time_s":6.42,"mem_mb":69.4,"disk_size":"112M"},{"runtime":"python:3.9-slim","python_version":"3.9","os_libc":"slim (glibc)","variant":"default","exit_code":1,"wheel_type":null,"failure_reason":null,"install_time_s":null,"import_time_s":null,"mem_mb":null,"disk_size":null}]},"quickstart_checks":{"last_tested":"2026-04-23","tag":"verified","tag_description":"quickstart runs on critical runtimes, recently tested","results":[{"runtime":"python:3.10-alpine","exit_code":0},{"runtime":"python:3.10-slim","exit_code":0},{"runtime":"python:3.11-alpine","exit_code":0},{"runtime":"python:3.11-slim","exit_code":0},{"runtime":"python:3.12-alpine","exit_code":0},{"runtime":"python:3.12-slim","exit_code":0},{"runtime":"python:3.13-alpine","exit_code":0},{"runtime":"python:3.13-slim","exit_code":0},{"runtime":"python:3.9-alpine","exit_code":0},{"runtime":"python:3.9-slim","exit_code":0}]}}