Elementary Data

Warnings

• breaking The `elementary-data` Python CLI works in conjunction with the `elementary-data/elementary` dbt package. Frequent updates to the dbt package can introduce incompatibilities if the CLI and dbt package versions are not aligned. Always refer to the official documentation for recommended version pairings.
  Fix: Ensure both the `elementary-data` Python library and the `elementary-data/elementary` dbt package are updated to compatible versions as recommended in Elementary's documentation. Regularly check release notes for specific version compatibility information.
• deprecated In version 0.23.0, `datetime.utcnow()` was replaced with `datetime.now(tz=timezone.utc)` for compliance with Python's best practices regarding timezone-aware datetimes. Code directly interacting with Elementary's internal datetime handling might need updating.
  Fix: Migrate any custom code that interacts with `datetime.utcnow()` to `datetime.now(tz=timezone.utc)` to ensure timezone-aware operations and avoid future deprecation warnings.
• gotcha While v0.19.2 addressed a specific backwards compatibility issue with Pydantic, broader Pydantic major version transitions (e.g., v1 to v2) can still cause issues if other libraries in your environment have conflicting Pydantic versions. This can lead to unexpected runtime errors.
  Fix: Keep `elementary-data` updated to the latest version to benefit from Pydantic compatibility fixes. When encountering Pydantic errors, check if other installed libraries have conflicting Pydantic version requirements. Consider using tools like `pipdeptree` or `pydantic-compat` for complex dependency environments.
• gotcha Elementary updated its internal dbt usage in v0.20.0 to remove the deprecated `-m` flag. Users running older versions of `elementary-data` with newer `dbt` versions might experience unexpected behavior or errors related to dbt command execution.
  Fix: Upgrade `elementary-data` to version 0.20.0 or higher to ensure compatibility with updated dbt command structures.
• gotcha The `edr` CLI requires a correctly configured `profiles.yml` (following dbt's format) with a profile specifically named `elementary` to connect to your data warehouse. Misconfiguration of this profile is a common cause of connection issues.
  Fix: Ensure your `~/.dbt/profiles.yml` file contains a profile named `elementary` with valid credentials and connection details for your data warehouse. Refer to Elementary's documentation for required permissions and specific database configurations.
• gotcha Prior to v0.19.4, `float("inf")` (infinity) and `float("nan")` (not a number) values in your data could cause JSON serialization failures when `elementary-data` attempted to generate reports, leading to corrupted or ungeneratable output.
  Fix: Upgrade `elementary-data` to version 0.19.4 or later to resolve the JSON serialization issue for `inf` and `nan` float values.
• gotcha The `elementary_sdk` Python package (distinct from `elementary-data` CLI) is primarily for programmatically sending data quality information to Elementary Cloud and testing Python pipelines. It's not intended for generating the self-hosted reports that the `edr` CLI provides. Confusing their purposes or expecting CLI features from the SDK (or vice-versa) can lead to integration challenges.
  Fix: Understand the distinct roles: `elementary-data` (CLI, `edr`) for dbt-native report generation and alerts from your data warehouse, and `elementary_sdk` (Python API) for integrating Python-based data quality checks with Elementary Cloud. Use the appropriate tool for your specific need.

Install

• pip install elementary-data Base installation for CLI
• pip install 'elementary-data[all]' Installation with all database connectors
• pip install 'elementary-data[snowflake]' # Or bigquery, redshift, databricks, etc. Installation for specific database

Quickstart

The `elementary-data` library's primary function is accessed via its command-line interface (`edr`), which generates data observability reports and sends alerts based on dbt project artifacts. This quickstart assumes you have an active dbt project with the Elementary dbt package configured and run. The `edr report` command will then query your data warehouse (via your dbt profiles) and generate a comprehensive HTML report.

# 1. Install the Elementary dbt package (in your dbt project's packages.yml):
# packages:
#   - package: elementary-data/elementary
#     version: 0.23.0 # Use the latest version

# 2. Add configuration to your dbt_project.yml (example):
# models:
#   elementary:
#     +schema: "elementary"

# 3. Install dbt dependencies and build Elementary models:
# dbt deps
# dbt run --select elementary
# dbt test

# 4. Install the Elementary CLI:
pip install elementary-data

# 5. Ensure your dbt profiles.yml is configured with an 'elementary' profile
# (see Elementary docs for details, typically ~/.dbt/profiles.yml)

# 6. Generate the data observability report:
# The report will be saved as an HTML file in 'target/elementary_report.html'
edr report --project-dir $(pwd)

# Or to monitor and send alerts:
# edr monitor --slack-token $SLACK_TOKEN --slack-channel '#data-alerts'