TruLens

2.8.0 verified Fri May 01 auth: no python

TruLens is a library for systematically tracking and evaluating LLM-based applications. It provides instrumentation, feedback functions, and a dashboard for monitoring and improving LLM app performance. Current version: 2.8.0, with monthly releases.

pip install trulens-core

Common errors

error ImportError: cannot import name 'Tru' from 'trulens' ↓

cause The old `Tru` class was removed in favor of `TruSession` starting from v2.0.

fix

Use from trulens.core import TruSession and replace Tru() with TruSession().

error AttributeError: 'OpenAI' object has no attribute 'groundedness_measure_with_cot_reasons' ↓

cause Feedback function names changed between versions. The API is version-specific.

fix

Check the correct method name in the provider's documentation. For OpenAI, use provider.groundedness_measure_with_cot_reasons (available in 2.x).

error ModuleNotFoundError: No module named 'trulens_eval' ↓

cause The old `trulens_eval` package was split into `trulens-core` and other subpackages starting from v2.0.

fix

Install trulens-core and use from trulens.core import ... instead.

Warnings

breaking In version 2.7.0, the `Feedback` class and `MetricConfig` were replaced by the unified `Metric` class. Old code using `Feedback(...).on_output().on_input()` will break. ↓

fix Migrate to `Metric` class with explicit selectors dictionary. See migration guide: https://www.trulens.org/trulens/api/metrics/

deprecated `run_dashboard_sis` is deprecated as of 2.7.2. ↓

fix Use `TruSession().run_dashboard()` instead.

gotcha Feedback functions require an LLM provider to be initialized with a valid API key. If the key is missing or invalid, they will fail silently or raise cryptic errors. ↓

fix Always check that the provider is correctly instantiated with `api_key` or `base_url` and test with a known good key.

gotcha `TruSession` is the new session class starting from 2.0, replacing the old `Tru` class. Importing `Tru` directly will raise ImportError. ↓

fix Use `from trulens.core import TruSession` instead of `from trulens import Tru`.

Quickstart

Initialize TruLens session and define a feedback function.

from trulens.core import TruSession
from trulens.feedback import Feedback
from trulens.providers.openai import OpenAI

session = TruSession()
session.reset_database()

provider = OpenAI(model_engine="gpt-4o-mini", api_key=os.environ.get('OPENAI_API_KEY', ''))
f_groundedness = Feedback(provider.groundedness_measure_with_cot_reasons, name="Groundedness")
print("Feedback defined.")
# See docs for full quickstart: https://www.trulens.org/