judgeval

1.0.5 verified Sat May 09 auth: no python

judgeval is the open-source post-building layer for Agent Behavior Monitoring. It provides tools for evaluating, tracing, and monitoring AI agent behavior. The current version is 1.0.5, requiring Python >=3.10. Release cadence is active with frequent patches.

pip install judgeval

Common errors

error ModuleNotFoundError: No module named 'judgeval' ↓

cause Package not installed or installed in a different environment.

fix

Run 'pip install judgeval' and ensure you are using the correct Python environment.

error ImportError: cannot import name 'JudgmentClient' from 'judgeval' ↓

cause Using an outdated import path from old documentation.

fix

Use 'from judgeval import JudgmentClient' (not from judgeval.client).

error judgeval.errors.AuthenticationError: Invalid API key ↓

cause Missing or incorrect API key.

fix

Set the JUDGEVAL_API_KEY environment variable or pass api_key='your_key' to the client.

Warnings

breaking In v1.0.0, the client API was overhauled. The old 'judgeval.JudgmentClient' now requires an api_key argument. If you were using an older version without api_key, your code will break. ↓

fix Update to: client = JudgmentClient(api_key='your_key')

breaking The 'judgment' decorator signature changed in v1.0.0. It no longer accepts positional arguments for scorers; use keyword arguments. ↓

fix @judgment(scorers=['hallucination']) instead of @judgment('hallucination')

gotcha The package requires Python >=3.10. Using it with Python 3.9 or earlier will fail to install. ↓

fix Upgrade your Python environment to 3.10 or later.

Imports

JudgmentClient
wrong
```
from judgeval.client import JudgmentClient
```
correct
```
from judgeval import JudgmentClient
```
Client class is directly under judgeval package since v1.0.0
judgment
```
from judgeval import judgment
```
The 'judgment' decorator/context manager for tracing

scorer

from judgeval.scorers import AnswerRelevancyScorer

Scorers are in judgeval.scorers submodule

Quickstart

Initialize the client and run a simple judgment with a scorer.

from judgeval import JudgmentClient

client = JudgmentClient(api_key=os.environ.get('JUDGEVAL_API_KEY', ''))
# Example: create a judgment
result = client.judge(
    input="What is the capital of France?",
    output="Paris",
    scorers=["answer_relevancy"]
)
print(result)