{"id":14787,"library":"openlineage-dbt","title":"OpenLineage dbt Integration","description":"The `openlineage-dbt` library provides a seamless integration between dbt (data build tool) and OpenLineage, capturing detailed data lineage for dbt models, sources, and exposures. It functions as a dbt adapter, translating dbt operations into OpenLineage events. The current version is 1.46.0 and it typically releases updates to align with new OpenLineage Python client versions and dbt-core major/minor versions, maintaining an active development cadence.","status":"active","version":"1.46.0","language":"en","source_language":"en","source_url":"https://github.com/OpenLineage/OpenLineage/tree/main/integration/dbt","tags":["dbt","openlineage","lineage","data-governance","etl","elt"],"install":[{"cmd":"pip install openlineage-dbt","lang":"bash","label":"Install core package"},{"cmd":"pip install 'openlineage-dbt[snowflake]' # or [bigquery], [postgres], etc.","lang":"bash","label":"Install with dbt adapter (example: Snowflake)"}],"dependencies":[{"reason":"Required for dbt CLI functionality and adapter integration.","package":"dbt-core","optional":false},{"reason":"Underlying client for sending OpenLineage events.","package":"openlineage-python","optional":false},{"reason":"Specific dbt adapter (e.g., dbt-snowflake, dbt-bigquery) that openlineage-dbt wraps.","package":"dbt-<adapter_name>","optional":false}],"imports":[{"note":"This library primarily integrates via dbt CLI commands and `profiles.yml` configuration, rather than direct Python imports for end-user applications. This import is provided as a technical placeholder from the package root.","symbol":"__version__","correct":"from openlineage_dbt import __version__"}],"quickstart":{"code":"# 1. Install openlineage-dbt and your target dbt adapter (e.g., dbt-snowflake)\n# pip install openlineage-dbt dbt-snowflake\n\n# 2. Configure your dbt profiles.yml\n# (e.g., ~/.dbt/profiles.yml)\n# Add an OpenLineage profile that wraps your actual adapter:\n# my_openlineage_profile:\n#   target: dev\n#   outputs:\n#     dev:\n#       type: openlineage\n#       adapter: snowflake # The actual dbt adapter you are using\n#       account: <your_snowflake_account>\n#       user: <your_snowflake_user>\n#       password: <your_snowflake_password>\n#       role: <your_snowflake_role>\n#       warehouse: <your_snowflake_warehouse>\n#       database: <your_snowflake_database>\n#       schema: <your_snowflake_schema>\n#       threads: 1\n\n# 3. Set OpenLineage URL (and optionally API Key) as environment variables\n#    or pass as CLI arguments.\nimport os\nos.environ['OPENLINEAGE_URL'] = os.environ.get('OPENLINEAGE_URL', 'http://localhost:5000') # Or your Marque URL\nos.environ['OPENLINEAGE_API_KEY'] = os.environ.get('OPENLINEAGE_API_KEY', 'your_api_key_if_needed')\n\n# 4. Run dbt with the OpenLineage-enabled profile\n# Create a dummy dbt project structure for demonstration\n# (assuming you have a dbt project initialized, e.g., 'dbt init my_project')\n# For example, create models/example.sql in your dbt project:\n# -- models/example.sql\n# -- {{ config(materialized='table') }}\n# -- select 1 as id\n\nprint(f\"Running dbt with OpenLineage at {os.environ['OPENLINEAGE_URL']}...\")\n\n# Example dbt command (execute from your dbt project root):\n# dbt build --profile my_openlineage_profile --target dev\n# Or, to ensure lineage is sent:\n# dbt build --profile my_openlineage_profile --target dev \\\n#           --openlineage-url $OPENLINEAGE_URL \\\n#           --openlineage-api-key $OPENLINEAGE_API_KEY\n\nprint(\"Please execute the dbt command from your terminal to see lineage generation.\")\nprint(\"Example: dbt build --profile my_openlineage_profile --target dev\")","lang":"python","description":"To quickly integrate `openlineage-dbt`, you need to install the package along with your specific dbt adapter (e.g., `dbt-snowflake`). Then, configure your `profiles.yml` to use `type: openlineage` and specify the `adapter` it should wrap. Finally, set the `OPENLINEAGE_URL` and `OPENLINEAGE_API_KEY` (if required) environment variables or pass them as CLI arguments when running dbt commands. Lineage events will be sent to the configured OpenLineage backend."},"warnings":[{"fix":"Check the `openlineage-dbt` PyPI page or GitHub README for supported `dbt-core` versions and upgrade/downgrade dbt components accordingly.","message":"Dbt version compatibility is crucial. `openlineage-dbt` is designed to work with specific versions of `dbt-core`. Ensure your `dbt-core` and underlying `dbt-<adapter>` versions align with the `openlineage-dbt` requirements (e.g., `dbt-core>=1.0.0,<2.0.0` for recent `openlineage-dbt` versions).","severity":"gotcha","affected_versions":"<1.x.x"},{"fix":"Ensure your `profiles.yml` has the `type: openlineage` and `adapter: <your_original_dbt_adapter_type>` fields correctly specified, along with the full connection details for the underlying adapter.","message":"The `profiles.yml` configuration requires `type: openlineage` and a nested `adapter: <your_dbt_adapter>` field. Omitting or incorrectly configuring the `adapter` field under `type: openlineage` will cause dbt to fail with profile errors.","severity":"breaking","affected_versions":"All versions"},{"fix":"Set `OPENLINEAGE_URL` and `OPENLINEAGE_API_KEY` (if required by your backend) as environment variables or pass them as CLI flags (`--openlineage-url`, `--openlineage-api-key`) during your dbt command execution.","message":"OpenLineage URL and API Key are mandatory for sending lineage events. If `OPENLINEAGE_URL` is not provided (via environment variable or `--openlineage-url` CLI flag), `openlineage-dbt` will fail to send events or cause dbt runs to error out.","severity":"gotcha","affected_versions":"All versions"}],"env_vars":null,"last_verified":"2026-04-17T00:00:00.000Z","next_check":"2026-07-16T00:00:00.000Z","problems":[{"fix":"Run `pip install openlineage-dbt` to install the package.","cause":"`openlineage-dbt` is not installed in the current Python environment.","error":"ModuleNotFoundError: No module named 'openlineage_dbt'"},{"fix":"In your `profiles.yml`, under your `type: openlineage` profile, add `adapter: <your_original_dbt_adapter_type>` (e.g., `adapter: snowflake`) along with the necessary connection details for that adapter.","cause":"Your `profiles.yml` configuration for the `openlineage` type is missing the nested `adapter` field, which specifies the actual dbt adapter (e.g., snowflake, bigquery) to wrap.","error":"dbt encountered an error: The 'adapter' field in your profile is missing or invalid. Received adapter type: openlineage"},{"fix":"Verify the `OPENLINEAGE_URL` environment variable or CLI flag is correct and reachable. Check your OpenLineage backend status. If using an API key, ensure `OPENLINEAGE_API_KEY` is correctly set.","cause":"The `OPENLINEAGE_URL` is incorrect, the OpenLineage backend is unreachable, or the `OPENLINEAGE_API_KEY` (if used) is invalid.","error":"Error: Could not connect to OpenLineage API at [URL] - Status: [HTTP Status Code]"}],"ecosystem":"pypi"}