{"id":5841,"library":"airflow-dbt-python","title":"Airflow dbt Python","description":"airflow-dbt-python is a Python library providing Airflow operators, hooks, and utilities to execute dbt commands. Unlike solutions wrapping the dbt CLI, it directly interfaces with dbt-core, enabling features like using Airflow connections as dbt targets and pushing dbt artifacts to XCom. The library is currently at version 3.5.0 and actively maintained, with a focus on supporting recent versions of Airflow and dbt.","status":"active","version":"3.5.0","language":"en","source_language":"en","source_url":"https://github.com/tomasfarias/airflow-dbt-python","tags":["airflow","dbt","etl","orchestration","data transformation","python","data pipelines"],"install":[{"cmd":"pip install airflow-dbt-python","lang":"bash","label":"Base Installation"},{"cmd":"pip install airflow-dbt-python[redshift]","lang":"bash","label":"With dbt-redshift adapter"},{"cmd":"pip install airflow-dbt-python[snowflake]","lang":"bash","label":"With dbt-snowflake adapter"}],"dependencies":[{"reason":"Core dbt functionality, version 1.8 or later required.","package":"dbt-core","optional":false},{"reason":"Requires an Airflow deployment, version 3.0 or later recommended. Python version >=3.10 is also required.","package":"apache-airflow","optional":false},{"reason":"Specific dbt database adapters (e.g., dbt-redshift, dbt-snowflake) are required based on your dbt project's target database.","package":"dbt-adapters","optional":true}],"imports":[{"symbol":"DbtRunOperator","correct":"from airflow_dbt_python.operators.dbt import DbtRunOperator"},{"symbol":"DbtSeedOperator","correct":"from airflow_dbt_python.operators.dbt import DbtSeedOperator"},{"note":"For dbt-core v1.0.0+, the `data` and `schema` attributes were deprecated in favor of `singular` and `generic`.","wrong":"DbtTestOperator(data=..., schema=...)","symbol":"DbtTestOperator","correct":"from airflow_dbt_python.operators.dbt import DbtTestOperator"},{"symbol":"DbtDocsGenerateOperator","correct":"from airflow_dbt_python.operators.dbt import DbtDocsGenerateOperator"}],"quickstart":{"code":"import datetime as dt\n\nfrom airflow import DAG\nfrom airflow.utils.dates import days_ago\nfrom airflow_dbt_python.operators.dbt import (\n    DbtRunOperator,\n    DbtSeedOperator,\n    DbtTestOperator,\n)\n\ndefault_args = {\n    \"owner\": \"airflow\",\n    \"start_date\": days_ago(1),\n    \"depends_on_past\": False,\n    \"email_on_failure\": False,\n    \"email_on_retry\": False,\n    \"retries\": 1,\n}\n\nwith DAG(\n    dag_id=\"example_dbt_workflow\",\n    schedule_interval=\"0 0 * * *\",\n    catchup=False,\n    dagrun_timeout=dt.timedelta(minutes=60),\n    default_args=default_args,\n    tags=[\"dbt\", \"example\"],\n) as dag:\n    dbt_seed = DbtSeedOperator(\n        task_id=\"dbt_seed_task\",\n        project_dir=\"/path/to/my/dbt/project/\",\n        profiles_dir=\"~/.dbt/\",\n        target=\"production\",\n        profile=\"my-project\",\n    )\n\n    dbt_run = DbtRunOperator(\n        task_id=\"dbt_run_task\",\n        project_dir=\"/path/to/my/dbt/project/\",\n        profiles_dir=\"~/.dbt/\",\n        target=\"production\",\n        profile=\"my-project\",\n        select=[\"+tag:daily\"],\n        exclude=[\"tag:deprecated\"],\n        full_refresh=False,\n    )\n\n    dbt_test = DbtTestOperator(\n        task_id=\"dbt_test_task\",\n        project_dir=\"/path/to/my/dbt/project/\",\n        profiles_dir=\"~/.dbt/\",\n        target=\"production\",\n        profile=\"my-project\",\n        singular=True, # For dbt-core v1.0.0+ tests\n    )\n\n    dbt_seed >> dbt_run >> dbt_test","lang":"python","description":"This example DAG demonstrates a basic dbt workflow using airflow-dbt-python operators. It includes seeding data, running dbt models with specific tags, and executing tests. Replace `/path/to/my/dbt/project/` and `~/.dbt/` with your actual dbt project and profiles directories, or configure remote storage as needed for multi-machine/cloud environments. Ensure your Airflow connections for dbt targets are configured if not using `profiles.yml`."},"warnings":[{"fix":"Ensure `dbt-core` and its adapters are installed separately. For `DbtTestOperator`, use `singular` or `generic` arguments instead of `data` or `schema` for dbt-core v1.0.0+.","message":"With dbt-core v1.0.0 and later, the way dbt is installed changed significantly. Instead of `pip install dbt`, you now install `dbt-core` and then specific database adapters (e.g., `dbt-redshift`, `dbt-snowflake`). This also impacted how `DbtTestOperator` handled test types.","severity":"breaking","affected_versions":"dbt-core <1.0.0 and airflow-dbt-python <0.10.0"},{"fix":"Store dbt projects in remote storage (e.g., S3, GCS, Git repositories) and use `project_dir` and `profiles_dir` with URL schemes (e.g., `s3://bucket/project/`) or configure Airflow Connections for dbt targets. `airflow-dbt-python` will download files to a temporary directory for execution.","message":"In multi-machine or cloud Airflow installations (e.g., AWS MWAA, GCP Cloud Composer), workers may not have a shared local filesystem. Storing dbt project files directly on the worker is unreliable. `airflow-dbt-python` requires dbt project files to be accessible.","severity":"gotcha","affected_versions":"All versions when using multi-machine/cloud Airflow"},{"fix":"Always test new versions of Airflow, dbt-core, and `airflow-dbt-python` in a staging environment before upgrading production systems. Report any issues to the library maintainers.","message":"New versions of Apache Airflow and dbt-core may introduce breaking changes. The `airflow-dbt-python` library aims to keep up with the latest releases, but compatibility issues can arise.","severity":"breaking","affected_versions":"All versions (future updates of Airflow/dbt)"},{"fix":"Explicitly set `profiles_dir` or ensure your dbt project remote URL includes `profiles.yml`. Alternatively, define an Airflow Connection with the ID matching your dbt `target` name.","message":"When omitting `profiles_dir` in operators, `airflow-dbt-python` will first check if the `project_dir` URL includes a `profiles.yml`. If not found, it will attempt to find an Airflow Connection using the `target` argument.","severity":"gotcha","affected_versions":"All versions"}],"env_vars":null,"last_verified":"2026-04-14T00:00:00.000Z","next_check":"2026-07-13T00:00:00.000Z"}