{"id":3953,"library":"dbt-clickhouse","title":"dbt-clickhouse","description":"dbt-clickhouse is the ClickHouse plugin for dbt (data build tool), enabling data professionals to transform data in ClickHouse using SQL. It leverages dbt-core's capabilities, providing native support for ClickHouse-specific features like materialized views and database engines. Currently at version 1.10.0, the library maintains an active release cadence, frequently aligning with new dbt-core versions and introducing ClickHouse-specific enhancements.","status":"active","version":"1.10.0","language":"en","source_language":"en","source_url":"https://github.com/ClickHouse/dbt-clickhouse","tags":["dbt","data build tool","clickhouse","sql","data warehousing","etl","elt","data transformation"],"install":[{"cmd":"pip install dbt-clickhouse","lang":"bash","label":"Install dbt-clickhouse"}],"dependencies":[{"reason":"Primary dependency, dbt-clickhouse is an adapter for dbt-core.","package":"dbt-core","optional":false},{"reason":"Shared adapter logic for dbt plugins. Minimum version 1.16.7 required by dbt-clickhouse v1.9.8+.","package":"dbt-adapters","optional":false}],"imports":[],"quickstart":{"code":"import os\nimport shutil\nfrom dbt.cli.main import dbtRunner, dbtUsageException\n\n# Define paths for the quickstart project\nproject_dir = \"dbt_clickhouse_quickstart_project\"\nprofiles_dir = \".dbt\" # Default location for profiles.yml\n\n# Clean up previous runs to ensure a fresh start\nif os.path.exists(project_dir):\n    shutil.rmtree(project_dir)\nif os.path.exists(profiles_dir):\n    shutil.rmtree(profiles_dir)\n\n# Create necessary directories\nos.makedirs(project_dir)\nos.makedirs(os.path.join(project_dir, \"models\"))\nos.makedirs(profiles_dir)\n\n# --- 1. Create profiles.yml for ClickHouse connection ---\n# Using environment variables for connection details, falling back to localhost defaults\nprofiles_content = f\"\"\"\ndbt_clickhouse_quickstart:\n  target: dev\n  outputs:\n    dev:\n      type: clickhouse\n      host: {os.environ.get('DBT_CLICKHOUSE_HOST', 'localhost')}\n      port: {os.environ.get('DBT_CLICKHOUSE_PORT', '8123')}\n      user: {os.environ.get('DBT_CLICKHOUSE_USER', 'default')}\n      password: {os.environ.get('DBT_CLICKHOUSE_PASSWORD', '')}\n      database: {os.environ.get('DBT_CLICKHOUSE_DATABASE', 'default')}\n      schema: default # In ClickHouse, database acts as schema for dbt\n      interface: http # or native\n      secure: false\n      verify: false\n\"\"\"\nwith open(os.path.join(profiles_dir, 'profiles.yml'), 'w') as f:\n    f.write(profiles_content)\n\n# --- 2. Create dbt_project.yml to link to the profile ---\ndbt_project_content = f\"\"\"\nname: 'dbt_clickhouse_quickstart'\nversion: '1.0.0'\nconfig-version: 2\nprofile: 'dbt_clickhouse_quickstart'\nmodel-paths: [\"models\"]\n\"\"\"\nwith open(os.path.join(project_dir, 'dbt_project.yml'), 'w') as f:\n    f.write(dbt_project_content)\n\n# --- 3. Create a simple SQL model file ---\nmodel_content = \"\"\"\n{{ config(materialized='table') }}\n\nSELECT\n    1 as id,\n    'hello dbt clickhouse' as value\n\"\"\"\nwith open(os.path.join(project_dir, 'models', 'my_test_model.sql'), 'w') as f:\n    f.write(model_content)\n\n# --- 4. Run dbt commands programmatically ---\ndbt = dbtRunner()\n\nprint(\"\\n--- Running dbt debug to test connection ---\")\n# The debug command does not require an active ClickHouse server to run, but will report connection errors\ndebug_res = dbt.invoke([\"debug\", \"--project-dir\", project_dir, \"--profiles-dir\", profiles_dir])\nif debug_res.success:\n    print(\"dbt debug successful (connection details printed, may show connectivity errors if ClickHouse is not running).\")\nelse:\n    print(f\"dbt debug failed: {debug_res.exception}\")\n\nprint(\"\\n--- Attempting to run dbt run (requires a running ClickHouse instance) ---\")\ntry:\n    run_res = dbt.invoke([\"run\", \"--project-dir\", project_dir, \"--profiles-dir\", profiles_dir])\n    if run_res.success:\n        print(\"dbt run successful. Model 'my_test_model' should be created as a table in ClickHouse.\")\n    else:\n        print(f\"dbt run failed: {run_res.exception}. Ensure ClickHouse is running and accessible.\")\nexcept dbtUsageException as e:\n    print(f\"dbt CLI usage error: {e}\")\nexcept Exception as e:\n    print(f\"An unexpected error occurred during dbt run: {e}\")\n\n# Clean up the created files and directories\nshutil.rmtree(project_dir)\nshutil.rmtree(profiles_dir)\nprint(\"\\nCleaned up quickstart files.\")\n","lang":"python","description":"This quickstart demonstrates how to configure and run a simple dbt project with dbt-clickhouse programmatically using Python. It creates the necessary `profiles.yml`, `dbt_project.yml`, and a sample model, then attempts to execute `dbt debug` and `dbt run`. Ensure you have a running ClickHouse instance accessible via the specified host and port (defaults to localhost:8123)."},"warnings":[{"fix":"Review your materialized view configurations. If you rely on historical data backfilling during full refreshes, ensure `catchup: True` is explicitly set or remove the `catchup` configuration if `True` is the default behavior you expect.","message":"In v1.10.0, the `catchup: False` configuration flag for materialized views (`materialized='materialized_view'`) is now strictly respected during full refresh operations, preventing backfilling of historical data. Previously, this might have been ignored in some scenarios.","severity":"gotcha","affected_versions":"1.10.0+"},{"fix":"Users on version 1.9.4 experiencing this issue must upgrade to dbt-clickhouse v1.9.5 or a later version to fix the `dbt test` functionality.","message":"A regression in dbt-clickhouse v1.9.4 caused `dbt test` commands to fail with `macro 'dbt_macro__get_expected_sql' takes not more than 2 argument(s)` errors.","severity":"breaking","affected_versions":"1.9.4"},{"fix":"Always check the dbt-clickhouse release notes or official documentation for the compatible dbt-core version range when upgrading either library. Ensure your `dbt-clickhouse` and `dbt-core` versions are aligned (e.g., `pip install dbt-core==X.Y.Z dbt-clickhouse==A.B.C`).","message":"dbt-clickhouse versions have specific compatibility requirements with dbt-core versions. For instance, v1.9.2 limited dbt-core to `<1.10.X`, while v1.9.7 added official support for dbt-core 1.10. Using incompatible versions can lead to unexpected errors or features not working.","severity":"gotcha","affected_versions":"All versions"},{"fix":"When installing or upgrading dbt-clickhouse 1.9.8 or later, ensure `dbt-adapters` is also updated to `1.16.7` or a more recent compatible version. `pip install dbt-clickhouse` typically handles this by installing the latest compatible dependencies.","message":"dbt-clickhouse v1.9.8 updated its minimum requirement for `dbt-adapters` to `1.16.7` to resolve compatibility issues. Older versions of `dbt-adapters` might cause problems when used with `dbt-clickhouse 1.9.8` or newer.","severity":"gotcha","affected_versions":"1.9.8+"}],"env_vars":null,"last_verified":"2026-04-11T00:00:00.000Z","next_check":"2026-07-10T00:00:00.000Z"}