HUD Python SDK (Evaluations and RL Environments)

0.5.35 · active · verified Thu Apr 16

HUD is a platform for building Reinforcement Learning (RL) environments for AI agents. It allows users to define agent-callable tools, write evaluation scenarios, run evaluations at scale, and train models on the results. The Python SDK, currently at version 0.5.35, is actively developed with frequent releases, often multiple times a month, providing tools for creating and evaluating AI agents.

Common errors

```
ModuleNotFoundError: No module named 'hud'
```
cause The `hud-python` library is either not installed or not installed in the active Python environment.

fix Run `pip install hud-python` (or `uv tool install hud-python --python 3.12` for CLI) to install the library.
```
E0004 - Hud init ran, HUD_KEY invalid
```
cause The `HUD_API_KEY` environment variable is set but contains an invalid or empty value, preventing authentication with the HUD platform.

fix Ensure `export HUD_API_KEY="your_valid_key_here"` is set with a correct, non-empty API key obtained from `hud.ai`.
```
Did not load as HUD_ENABLE is undefined. Please set HUD_ENABLE=true to run Hud.
```
cause The `HUD_ENABLE` environment variable is not defined, which is required for the SDK to initialize.

fix Set `export HUD_ENABLE=true` in your environment or service configuration.
```
E0007 - Hud init ran, HUD_TAGS invalid JSON
```
cause The optional `HUD_TAGS` environment variable is provided but its value is not a valid JSON string, which is required for metadata tagging.

fix Ensure `HUD_TAGS` is a valid JSON-formatted string, e.g., `export HUD_TAGS='{"env":"dev", "owner":"team_x"}'`.

Warnings

gotcha The `hud-python` library (for AI agent evaluations and RL environments) is distinct from `hud-sdk` (a runtime code sensor) on PyPI. Ensure you install and import from the correct package based on your use case.
Fix: For AI agent evaluations, use `pip install hud-python` and `import hud`. For the runtime sensor, refer to `hud-sdk` documentation.
gotcha The `HUD_ENABLE` environment variable must be explicitly set to `true` (case-insensitive) for the SDK to initialize and run correctly. If undefined or set to any other value, Hud components will not be active.
Fix: Set `export HUD_ENABLE=true` in your environment or service configuration before running your application.
breaking Changes in CLI commands like `hud build` and `hud analyze` have been introduced to distinguish between HTTP mode and stdio, which might alter expected behavior or require updates to existing scripts.
Fix: Consult the latest CLI documentation for `hud build` and `hud analyze` to ensure correct usage, especially when working with HTTP mode.
gotcha The `hud sync` command was introduced in version 0.5.34 for syncing local environments and tasks to the HUD platform. Users on older versions will not have access to this functionality.
Fix: Update to `hud-python` version 0.5.34 or newer to use the `hud sync` command for managing local and platform environments/tasks.

Install

pip install hud-python Install with pip
uv tool install hud-python --python 3.12 Install CLI with uv (recommended)

Imports

Environment
```
from hud import Environment
```
Used to define RL environments and scenarios.
create_agent
```
from hud.agents import create_agent
```
Used to instantiate AI agents for evaluation.
hud.eval
```
import hud; async with hud.eval(task) as ctx:
```
Context manager for running evaluations on tasks.

Quickstart

This quickstart demonstrates defining a HUD environment with a custom tool and a scenario, then running an AI agent against that scenario using `hud.eval()`. Ensure your `HUD_API_KEY` environment variable is set for authentication.

import os
from hud import Environment
from hud.agents import create_agent

# Ensure your HUD_API_KEY is set as an environment variable
# Example: export HUD_API_KEY="your_api_key_here"
hud_api_key = os.environ.get('HUD_API_KEY', '')

# Define an environment
env = Environment("my-first-env")

# Define a scenario using a tool
@env.tool()
def add(a: int, b: int) -> int:
    """Adds two numbers."""
    return a + b

@env.scenario("sum-check")
async def sum_check(num1: int, num2: int):
    # Prompt the agent to use the 'add' tool
    answer = yield f"What is the sum of {num1} and {num2}? Use the 'add' tool."
    correct = num1 + num2
    # Score the agent's answer
    yield 1.0 if str(correct) in str(answer) else 0.0

async def run_evaluation():
    # Create a task for the scenario
    task = env("sum-check", num1=5, num2=7)

    # Create an agent (e.g., using a model via HUD's gateway)
    agent = create_agent("gpt-4o") # or "claude-sonnet-4-5", etc.

    print(f"Running task: {task.scenario_slug} with {agent.model}")
    async with hud.eval(task) as ctx:
        result = await agent.run(ctx)
        print(f"Agent response: {result.response}")
        print(f"Reward: {result.reward}")

if __name__ == "__main__":
    import asyncio
    if not hud_api_key:
        print("Error: HUD_API_KEY environment variable not set. Please set it to run the quickstart.")
    else:
        asyncio.run(run_evaluation())

view raw JSON →