Evidently AI

0.7.21 · active · verified Sat Apr 11

Evidently is an open-source Python library (currently v0.7.21) for evaluating, testing, and monitoring machine learning and LLM systems in production. It offers 100+ built-in metrics to detect data drift, model performance issues, data quality problems, and LLM-specific evaluations. The library is actively developed with frequent releases, providing both an open-source framework for offline evaluations and a UI for continuous monitoring, with additional features available through Evidently Cloud.

Warnings

breaking Evidently v0.7 introduced a major API overhaul, making the new API the default. This includes changes to how Reports are imported (`from evidently.report import Report`) and the introduction of explicit `Dataset` and `DataDefinition` objects which replace the older `column_mapping` approach. Code written for versions prior to 0.7 (especially 0.6.7 and older) will likely break.
Fix: Refer to the official 'Migration Guide' in the Evidently documentation. Update import statements, and explicitly define `Dataset` and `DataDefinition` for your data inputs.
breaking Support for Python 3.9 was dropped in Evidently v0.7.21. Users on Python 3.9 will encounter errors or compatibility issues.
Fix: Upgrade your Python environment to 3.10 or newer (Evidently requires `>=3.10`).
gotcha When using Evidently alongside DVC (Data Version Control), there can be a dependency conflict with `pathspec`. Evidently v0.7.20 explicitly locked `pathspec` to `<1` for DVC compatibility. Ensure your `pathspec` version respects this constraint if you encounter issues with DVC.
Fix: When installing with DVC, ensure `pathspec<1` is enforced, e.g., `pip install evidently 'dvc-data<3' 'pathspec<1'` or check for DVC's recommended `pathspec` version at the time.
gotcha Evidently v0.7.21 added explicit support for Pandas 3. While this is an improvement, users of older Evidently versions might experience compatibility issues or unexpected behavior when using Pandas 3. Conversely, newer Evidently features might rely on specific Pandas 3 functionalities.
Fix: Always check the `evidently` and `pandas` compatibility matrix in the official documentation. For optimal compatibility, upgrade to the latest `evidently` version and a compatible `pandas` version (e.g., Pandas 3.x with Evidently >=0.7.21).
deprecated Evidently Cloud v1 entered read-only mode after May 31, 2025, for free users. Users of Evidently Cloud must migrate to Cloud v2 and use Evidently library version `0.7.0` or newer to continue sending data and accessing features.
Fix: Upgrade your Evidently Python library to `0.7.0` or higher and migrate your Evidently Cloud projects to v2.

Install

pip install evidently Install stable version

Imports

Report
```
from evidently.report import Report
```
As of v0.7, the new API is the default. 'evidently.future' imports were for a transition period (v0.6-0.6.7) and are now considered legacy for newer versions.
DataDriftPreset
```
from evidently.metric_preset import DataDriftPreset
```
Metric presets are now located under `evidently.metric_preset` for clarity.
Dataset
```
from evidently.core.datasets import Dataset
```
Starting with v0.7, explicit `Dataset` objects are required when defining data, along with `DataDefinition`.
DataDefinition
```
from evidently.core.data_definition import DataDefinition
```
Starting with v0.7, explicit `DataDefinition` objects are required to map input columns by type and role, replacing `column_mapping`.

Quickstart

This quickstart demonstrates how to generate a Data Drift Report using Evidently. It fetches a sample dataset, creates Evidently `Dataset` and `DataDefinition` objects (essential since v0.7), and then runs a `DataDriftPreset` report. The report can be displayed interactively in a notebook or saved as an HTML file.

import pandas as pd
from sklearn import datasets

from evidently.report import Report
from evidently.metric_preset import DataDriftPreset
from evidently.options import DataDriftOptions # To demonstrate custom options
from evidently.core.datasets import Dataset
from evidently.core.data_definition import DataDefinition

# Prepare a toy dataset (Adult dataset from OpenML)
reference_data_frame = datasets.fetch_openml(name="adult", version=2, as_frame="auto").frame
current_data_frame = reference_data_frame.sample(n=5000, random_state=0)

# Define data schema using DataDefinition (required from v0.7)
data_definition = DataDefinition(
    prediction_features=["income"],
    target_names="income",
    categorical_features=[
        'workclass', 'education', 'marital-status', 'occupation',
        'relationship', 'race', 'sex', 'native-country'
    ]
)

# Create Evidently Dataset objects
reference_dataset = Dataset(reference_data_frame, data_definition)
current_dataset = Dataset(current_data_frame, data_definition)

# Create and run a Data Drift Report
data_drift_report = Report(metrics=[
    DataDriftPreset(
        # Example of custom options, e.g., for statistical tests
        # data_drift_options=DataDriftOptions(threshold=0.1)
    )
])
data_drift_report.run(reference_data=reference_dataset, current_data=current_dataset)

# To display in a Jupyter notebook or save to HTML
# data_drift_report.show()
# data_drift_report.save_html("data_drift_report.html")

print("Data drift report generated successfully!")

view raw JSON →