{"id":5745,"library":"airflow-dbt","title":"Airflow dbt Integration (GoCardless)","description":"airflow-dbt is a Python package that provides Apache Airflow operators for integrating with dbt (data build tool). It allows users to orchestrate dbt commands like `seed`, `snapshot`, `run`, and `test` within Airflow DAGs by wrapping the dbt CLI. This package, currently at version 0.4.0, offers a foundational way to embed dbt transformations into Airflow workflows, with its last update in September 2021.","status":"maintenance","version":"0.4.0","language":"en","source_language":"en","source_url":"https://github.com/gocardless/airflow-dbt","tags":["airflow","dbt","etl","data-orchestration","transformation"],"install":[{"cmd":"pip install airflow-dbt","lang":"bash","label":"Install from PyPI"}],"dependencies":[{"reason":"Required for dbt CLI functionality; needs to be installed separately and accessible in the Airflow environment's PATH.","package":"dbt-core","optional":false},{"reason":"A dbt adapter (e.g., dbt-postgres, dbt-snowflake) matching your data warehouse is required by dbt-core for database connectivity.","package":"dbt-<adapter>","optional":false}],"imports":[{"symbol":"DbtSeedOperator","correct":"from airflow_dbt.operators.dbt_operator import DbtSeedOperator"},{"symbol":"DbtSnapshotOperator","correct":"from airflow_dbt.operators.dbt_operator import DbtSnapshotOperator"},{"symbol":"DbtRunOperator","correct":"from airflow_dbt.operators.dbt_operator import DbtRunOperator"},{"symbol":"DbtTestOperator","correct":"from airflow_dbt.operators.dbt_operator import DbtTestOperator"},{"note":"All operators are within the dbt_operator module.","wrong":"from airflow_dbt.operators.dbt_doc_generate_operator import DbtDocsGenerateOperator","symbol":"DbtDocsGenerateOperator","correct":"from airflow_dbt.operators.dbt_operator import DbtDocsGenerateOperator"}],"quickstart":{"code":"from airflow import DAG\nfrom airflow_dbt.operators.dbt_operator import (\n    DbtSeedOperator,\n    DbtSnapshotOperator,\n    DbtRunOperator,\n    DbtTestOperator\n)\nfrom airflow.utils.dates import days_ago\nimport os\n\ndefault_args = {\n    'dir': os.environ.get('DBT_PROJECT_DIR', '/path/to/your/dbt/project'),\n    'start_date': days_ago(0)\n}\n\nwith DAG(\n    dag_id='dbt_example_dag',\n    default_args=default_args,\n    schedule_interval='@daily',\n    tags=['dbt', 'example']\n) as dag:\n    dbt_seed = DbtSeedOperator(\n        task_id='dbt_seed',\n        profiles_dir=os.environ.get('DBT_PROFILES_DIR', '/path/to/your/.dbt') # Optional\n    )\n\n    dbt_snapshot = DbtSnapshotOperator(\n        task_id='dbt_snapshot'\n    )\n\n    dbt_run = DbtRunOperator(\n        task_id='dbt_run'\n    )\n\n    dbt_test = DbtTestOperator(\n        task_id='dbt_test',\n        retries=0 # Failing tests should fail the task, not retry\n    )\n\n    dbt_seed >> dbt_snapshot >> dbt_run >> dbt_test\n","lang":"python","description":"This quickstart demonstrates how to define a basic Airflow DAG using `airflow-dbt` operators to run a sequence of dbt commands: `seed`, `snapshot`, `run`, and `test`. Ensure your dbt project directory and profiles directory (if not default) are correctly specified, typically via environment variables or directly in the `dir` and `profiles_dir` arguments. The dbt CLI must be installed and accessible on the Airflow worker's PATH."},"warnings":[{"fix":"Ensure `dbt-core` and its relevant adapter are installed in your Airflow environment and that the `dbt` command is accessible from the Airflow worker processes. For managed services, this often means including it in `requirements.txt` and potentially configuring environment variables or plugins.","message":"This package relies on wrapping the dbt CLI. This means the dbt executable must be installed and available on the Airflow worker's PATH or explicitly set via the `dbt_bin` argument. This can be a common point of failure, especially in managed Airflow environments like AWS MWAA or GCP Cloud Composer, where managing CLI tools requires specific configurations.","severity":"gotcha","affected_versions":"<=0.4.0"},{"fix":"Review the operator's source code for supported arguments. If a required argument is missing, you might need to use Airflow's `BashOperator` to execute dbt commands directly, or consider alternatives like `airflow-dbt-python` for more comprehensive dbt integration without CLI dependency.","message":"The operators may not expose the full range of arguments available in the dbt CLI commands. For example, `DbtRunOperator` might not have an explicit `fail_fast` attribute, limiting granular control over dbt execution parameters directly from Airflow.","severity":"gotcha","affected_versions":"<=0.4.0"},{"fix":"For workflows requiring dbt artifact access, consider custom solutions using the `BashOperator` to read files from the dbt project's `target/` directory, or explore alternative integration packages that provide artifact parsing and XCom capabilities.","message":"This package does not offer direct access to dbt artifacts (e.g., `manifest.json`, `run_results.json`) generated during execution. This limitation prevents more advanced use cases such as dynamic DAG generation based on dbt's lineage or pushing artifacts to Airflow XComs for downstream processing.","severity":"gotcha","affected_versions":"<=0.4.0"},{"fix":"Thoroughly test compatibility with your specific Airflow and dbt-core versions before deployment. Consider migrating to more actively maintained alternatives like `airflow-dbt-python` (github.com/tomasfarias/airflow-dbt-python) or `dbt-airflow` (github.com/gmyrianthous/dbt-airflow) which offer more recent updates and potentially address these compatibility concerns.","message":"The package has not received updates since September 2021 (v0.4.0), making it potentially incompatible with newer versions of Apache Airflow (e.g., Airflow 2.10+) or dbt-core (e.g., dbt-core 1.8+), which may introduce breaking changes or new features not supported by this older integration. For instance, `dbt-common`'s `isodate` constraint can conflict with Airflow 2.10.3+.","severity":"deprecated","affected_versions":"All versions (<=0.4.0) with Airflow 2.10+ or dbt-core 1.8+"}],"env_vars":null,"last_verified":"2026-04-14T00:00:00.000Z","next_check":"2026-07-13T00:00:00.000Z"}