LiveKit Anthropic Plugin

1.5.4 · active · verified Thu Apr 16

livekit-plugins-anthropic is an Agent Framework plugin for integrating Anthropic's Claude family of Large Language Models (LLMs) with LiveKit Agents. It enables developers to use Claude APIs as an LLM provider for building real-time voice AI agents, supporting both text-based conversations and vision input capabilities. The current version is 1.5.4, with releases tied to the livekit-agents framework's active development and rapid feature additions.

Common errors

```
ValueError: Anthropic API key is required, either as argument or set ANTHROPIC_API_KEY environmental variable
```
cause The Anthropic API key has not been provided, either through the `api_key` parameter in `anthropic.LLM` or by setting the `ANTHROPIC_API_KEY` environment variable.

fix Set the `ANTHROPIC_API_KEY` environment variable (e.g., `export ANTHROPIC_API_KEY='sk-your-key'`) or pass it directly when initializing the LLM: `anthropic.LLM(api_key='sk-your-key', ...)`.
```
anthropic.APIStatusError: Your account has hit a rate limit. (HTTP status: 429)
```
cause Your Anthropic API usage has exceeded the rate limits for your account's usage tier, measured in requests per minute, tokens per minute, or tokens per day.

fix Review Anthropic's API rate limit documentation for your specific usage tier. Implement retry logic with exponential backoff in your agent, or consider upgrading your Anthropic plan if consistent higher throughput is needed.
```
The LLM repeats the exact same stream text output after triggering a function call.
```
cause This was a known bug in earlier versions of livekit-plugins-anthropic (specifically observed before version 1.5.x) when used with `voice_assistant` agents and function calls.

fix Ensure you are using `livekit-plugins-anthropic` version 1.5.x or newer. This issue was addressed in recent updates to the library. If using an older version, update your package.

Warnings

breaking Starting with livekit-agents 1.5.0, preemptive generation is enabled by default. This alters how LLM and TTS inference begins before a user's turn ends, potentially changing latency characteristics.
Fix: To disable preemptive generation, initialize AgentSession with `AgentSession(preemptive_generation=False)`.
gotcha Older versions of livekit-plugins-anthropic (before 1.4.5) might experience issues with 'trailing assistant turns' when using Claude 4.6+ models, potentially leading to incorrect responses.
Fix: Upgrade to livekit-plugins-anthropic version 1.4.5 or newer to ensure correct handling of Claude 4.6+ models.
gotcha When using Anthropic prompt caching, the `max_tokens` parameter must be strictly greater than `budget_tokens` to avoid configuration errors. Very short system prompts or tool lists may also not qualify for caching.
Fix: Ensure `max_tokens` is set sufficiently higher than `budget_tokens` in your LLM configuration. Refer to Anthropic's documentation for minimum cacheable block sizes.
gotcha The Anthropic plugin may not generate tool schemas with strict mode constraints, even though Anthropic's API supports a `strict` field on tool definitions for guaranteed schema conformance. This could lead to the model returning incompatible types or missing required fields.
Fix: Carefully validate tool outputs from the Anthropic LLM. Monitor GitHub issues for updates on `strict` tool schema support in livekit-plugins-anthropic. If strict validation is critical, implement manual validation or consider alternative tool invocation strategies.

Install

pip install livekit-plugins-anthropic Install latest version

Imports

LLM

from livekit.plugins import anthropic
llm_instance = anthropic.LLM(...)

Quickstart

This quickstart demonstrates how to initialize the Anthropic LLM within a LiveKit AgentSession. It highlights the essential `livekit.plugins.anthropic.LLM` class and the requirement for the `ANTHROPIC_API_KEY` environment variable. A full LiveKit agent typically processes real-time audio/text, but this snippet focuses on the Anthropic LLM setup.

import os
from livekit.agents import AgentSession, JobContext
from livekit.plugins import anthropic

# Set your Anthropic API key as an environment variable
# os.environ["ANTHROPIC_API_KEY"] = "sk-your-anthropic-key"

async def my_agent(ctx: JobContext):
    # Ensure ANTHROPIC_API_KEY is set in your environment or passed directly
    anthropic_api_key = os.environ.get('ANTHROPIC_API_KEY', '')
    if not anthropic_api_key:
        raise ValueError("ANTHROPIC_API_KEY environment variable is not set.")

    print("Starting agent session with Anthropic LLM...")
    session = AgentSession(
        llm=anthropic.LLM(
            model="claude-3-5-sonnet-20241022",
            api_key=anthropic_api_key # Can be omitted if env var is set
        )
    )

    # In a real agent, you'd process audio/text input and generate replies.
    # This is a minimal example to show LLM instantiation.
    print(f"Agent session created with LLM model: {session.llm.model}")
    # Example of a simple chat interaction (requires AgentSession context for full functionality)
    # from livekit.agents.llm import ChatContext, UserMessage
    # chat_ctx = ChatContext()
    # chat_ctx.append(UserMessage(text="Hello, what can you do?"))
    # response = await session.llm.chat(chat_ctx)
    # async for chunk in response.stream:
    #     print(chunk.delta)

    await session.astop()
    print("Agent session stopped.")

# To run this, you would typically integrate it with a LiveKit server
# and an Agent runner, like:
# if __name__ == "__main__":
#     from livekit.agents import cli
#     cli.run_agent(my_agent)

view raw JSON →