Braintrust Python SDK

0.14.0 · active · verified Fri Apr 10

The Braintrust Python SDK is the official library for logging, tracing, and evaluating AI applications with Braintrust. It enables developers to instrument LLM calls, manage prompts, run evaluations, and track performance. The current version is 0.14.0, and the library is actively maintained with frequent updates. [2]

Warnings

breaking Starting with v0.4.0, Braintrust Python SDK requires Python 3.10 or higher. Older Python versions are no longer supported. [17]
Fix: Upgrade your Python environment to 3.10 or newer.
deprecated The separate `braintrust-langchain` and `braintrust-adk` packages are deprecated. Their functionality for LangChain and Google ADK integrations has been moved into the main `braintrust` package. [3]
Fix: Uninstall the deprecated packages and use the latest `braintrust` package, which includes the integration directly. E.g., `pip install "braintrust[openai-agents]"` for agent integrations. [2, 3]
gotcha Authentication to Braintrust requires an API key, which must be set as the `BRAINTRUST_API_KEY` environment variable. Without this, operations like logging traces or running evaluations will not connect to your Braintrust project. [2, 9, 12]
Fix: Set the `BRAINTRUST_API_KEY` environment variable in your execution environment. For example: `export BRAINTRUST_API_KEY="<YOUR_API_KEY>"`.
gotcha For defining custom scoring functions in evaluation workflows, the `autoevals` package is frequently used alongside `braintrust`. If you encounter 'LevenshteinScorer not found' or similar errors, it's likely due to missing `autoevals`. [7, 14]
Fix: Install `autoevals` via `pip install autoevals`.
gotcha When integrating with Temporal Workflows, the Braintrust logger (`braintrust.init()`) must be initialized *before* creating your Temporal Worker to ensure proper span connection and traceability. [12]
Fix: Call `braintrust.init_logger()` at the earliest possible point in your application startup, preceding Temporal Worker initialization.

Install

pip install braintrust Core SDK
pip install "braintrust[all]" All Optional Extras
pip install "braintrust[cli,openai-agents,otel,performance,temporal]" Specific Optional Extras

Imports

Eval
```
from braintrust import Eval
```

init_logger

import braintrust
braintrust.init_logger(...)

wrap_openai

import braintrust
client = braintrust.wrap_openai(openai.OpenAI(...))

braintrust_langchain
```
from braintrust import LangchainCallback
```
The braintrust-langchain package is deprecated; its functionality is now integrated directly into the main 'braintrust' package. [3]

Quickstart

This quickstart demonstrates how to run a simple evaluation using the `braintrust.Eval` class. It defines a basic task and a scoring function, then executes the evaluation. The `BRAINTRUST_API_KEY` environment variable is required for the results to be logged to your Braintrust dashboard. [2, 7]

import os
from braintrust import Eval

def is_equal(expected, output):
    return expected == output

# Ensure BRAINTRUST_API_KEY is set in your environment variables.
# For local execution: export BRAINTRUST_API_KEY="<YOUR_API_KEY>"
# For production: Use secrets management or similar.
api_key = os.environ.get('BRAINTRUST_API_KEY', '')

if not api_key:
    print("Warning: BRAINTRUST_API_KEY environment variable not set. Evaluation will not be logged to Braintrust.")
    # In a real scenario, you might raise an error or handle accordingly.

# The Eval function itself implicitly uses the API key from the environment
# when .run() is called or when integrated with the Braintrust CLI.
Eval(
    "Say Hi Bot",
    data=lambda: [
        {"input": "Foo", "expected": "Hi Foo"},
        {"input": "Bar", "expected": "Hello Bar"},
    ],
    task=lambda input: "Hi " + input,
    scores=[is_equal],
).run()

print("Evaluation complete. Check your Braintrust dashboard.")

view raw JSON →