dbt-Dremio Adapter

1.10.0 · active · verified Fri Apr 17

dbt-dremio is the Dremio adapter plugin for dbt (data build tool), enabling data teams to transform data directly within Dremio using dbt's SQL-based workflows. It allows users to define models, tests, and documentation, then execute them against Dremio for efficient data transformations. The current version is 1.10.0, with new releases typically aligning with dbt-core's major version updates, often every 1-3 months.

Common errors

```
ERROR dbt.exceptions.FailedToConnectException: Could not connect to Dremio with the provided credentials.
```
cause Incorrect Dremio host, port, username, password, or schema configured in your `profiles.yml`. Network connectivity issues or firewall blocks can also cause this.

fix Double-check all connection parameters in your `profiles.yml` against your Dremio instance. Ensure Dremio is running and accessible from where dbt is executed. Verify proxy settings or firewall rules if applicable.
```
dbt.exceptions.DbtRuntimeError: Runtime Error: Compilation Error in model <model_name>... Syntax Error: <Dremio-specific error>
```
cause The SQL query in your dbt model uses syntax or functions that are not supported by the Dremio SQL dialect or your specific Dremio version.

fix Review the Dremio documentation for supported SQL functions and syntax. Refactor your SQL to comply with Dremio's dialect. Test complex queries directly in Dremio before integrating into dbt models.
```
dbt.exceptions.DbtRuntimeError: Could not find adapter for type 'dremio'.
```
cause `dbt-dremio` is either not installed or not correctly recognized by dbt-core. This can happen if dbt-core was installed in a different Python environment or if there's a conflict.

fix Ensure `dbt-dremio` is installed in the same Python environment as `dbt-core`. Run `pip install dbt-dremio`. If issues persist, try reinstalling both: `pip uninstall dbt-core dbt-dremio; pip install dbt-core dbt-dremio`.

Warnings

breaking Each major release of dbt-dremio (e.g., v1.7.0, v1.8.0, v1.9.0, v1.10.0) is designed to be compatible with a specific major version of dbt-core. Upgrading dbt-core without upgrading dbt-dremio (or vice-versa) can lead to unexpected errors.
Fix: Always check the dbt-dremio release notes for the compatible dbt-core version and upgrade both packages in tandem: `pip install --upgrade dbt-core dbt-dremio`.
gotcha Model contracts and constraints, introduced in dbt-core v1.5.0, are 'stubbed' in dbt-dremio, meaning the adapter acknowledges the feature but does not fully support enforcement within Dremio. Attempting to use these features may not result in the expected behavior or error handling.
Fix: Be aware that dbt model contracts and constraints might not be fully functional or enforced when using dbt-dremio. Consult the dbt-dremio documentation or GitHub issues for updates on future support.
gotcha Prior to v1.5.1, the `create_view_as` macro included superfluous parentheses around views, which could lead to performance issues in more complex queries.
Fix: Upgrade to dbt-dremio v1.5.1 or newer to benefit from performance improvements in view creation: `pip install --upgrade dbt-dremio`.
gotcha In versions prior to v1.5.1, fetching model data was enabled by default, which could negatively impact performance. This behavior was changed in v1.5.1 to be `false` by default.
Fix: Upgrade to dbt-dremio v1.5.1 or newer. If you must use an older version, investigate if explicitly disabling model data fetching is possible (though the official fix is to upgrade).

Install

pip install dbt-dremio Install latest version

Imports

dbt-dremio adapter
```
Configured in profiles.yml
```
dbt-dremio is a dbt adapter plugin and is not imported directly into Python scripts. It is loaded by dbt-core based on your profiles.yml configuration.

Quickstart

To get started with dbt-dremio, you need to configure your dbt `profiles.yml` to specify the connection details for your Dremio instance. Then, you can define your SQL models and run them using the dbt CLI. Ensure your dbt project is configured to use the Dremio profile. Remember to replace placeholders like `<your_dremio_host>`, `<your_dremio_username>`, `<your_dremio_password>`, and `<your_schema>` with your actual Dremio credentials and configuration.

# 1. Create a dbt project (if you don't have one)
# dbt init my_dremio_project

# 2. Configure your profiles.yml (typically at ~/.dbt/profiles.yml)
# --- profiles.yml ---
# my_dremio_profile:
#   target: dev
#   outputs:
#     dev:
#       type: dremio
#       host: <your_dremio_host> # e.g., localhost or cloud.dremio.com
#       port: 31010 # or 443 for Dremio Cloud
#       username: <your_dremio_username>
#       password: <your_dremio_password>
#       schema: <your_schema>
#       threads: 1
#       db_name: Dremio # Can be 'Dremio' for Dremio Cloud, or your specific project ID
#       catalog: <your_catalog_name> # Optional, for Dremio Enterprise Catalogs (v1.9.0+)
#       disable_ssl_verification: true # Use with caution, for self-signed certificates
# --- end profiles.yml ---

# 3. Create a sample dbt model (e.g., models/my_first_model.sql)
-- models/my_first_model.sql
SELECT
    1 as id,
    'hello' as message

# 4. Run dbt commands from your project directory
# dbt debug
# dbt run
# dbt test

view raw JSON →