LiveKit ElevenLabs Plugin

1.5.2 · active · verified Sun Apr 12

livekit-plugins-elevenlabs is a Python plugin for the LiveKit Agent Framework, providing seamless integration for voice synthesis (TTS) and speech-to-text (STT) using ElevenLabs' API. It enables real-time, multimodal AI agents to interact with users through high-quality ElevenLabs voices. The library receives frequent updates, often alongside releases of the core `livekit-agents` framework.

Warnings

breaking The ElevenLabs `eleven_v3` TTS model is currently not supported via the `livekit-plugins-elevenlabs` WebSocket endpoint. Attempts to use it will result in 403 errors, as `eleven_v3` does not support multi-stream WebSockets.
Fix: Avoid using `eleven_v3` with `livekit.plugins.elevenlabs.TTS` for real-time streaming. Consider using other supported ElevenLabs models or implementing a custom, non-streaming workaround if `eleven_v3` is essential, though this will incur higher latency. Refer to the LiveKit documentation and ElevenLabs API specifications for compatible models.
deprecated The `livekit-agents` framework, which `livekit-plugins-elevenlabs` integrates with, deprecated several keyword arguments for `AgentSession`'s endpointing and interruption settings in `v1.5.0`. These are now consolidated under the `TurnHandlingOptions` dictionary.
Fix: Update `AgentSession` initialization to use the `turn_handling` parameter with a `TurnHandlingOptions` dictionary instead of individual keyword arguments like `min_endpointing_delay` or `allow_interruptions`. Old arguments still work but will emit deprecation warnings and will be removed in future major versions. Consult the `livekit-agents` changelog for full details.

Install

pip install livekit-plugins-elevenlabs Basic Installation
pip install "livekit-agents[elevenlabs]" Install as LiveKit Agents extra

Imports

TTS
```
from livekit.plugins import elevenlabs
elevenlabs.TTS(...)
```
Plugins are typically imported from the `livekit.plugins` namespace.
STT
```
from livekit.plugins import elevenlabs
elevenlabs.STT(...)
```
Plugins are typically imported from the `livekit.plugins` namespace.

Quickstart

This quickstart demonstrates how to create a basic LiveKit voice agent using `livekit-plugins-elevenlabs` for both Text-to-Speech (TTS) and Speech-to-Text (STT) capabilities. It assumes you have `livekit-agents` and `python-dotenv` installed, and `LIVEKIT_URL`, `LIVEKIT_API_KEY`, `LIVEKIT_API_SECRET`, and `ELEVENLABS_API_KEY` (and `OPENAI_API_KEY` if using OpenAI LLM) set as environment variables. The agent connects to a LiveKit room, uses ElevenLabs for voice interaction, and can be extended with an LLM for conversational AI.

import asyncio
import os
from dotenv import load_dotenv
from livekit.agents import JobContext, WorkerOptions, cli, Agent, AgentSession
from livekit.agents.voice import VoicePipeline
from livekit.plugins import elevenlabs, openai # Assuming openai for LLM, can be replaced

load_dotenv()

async def entrypoint(ctx: JobContext):
    await ctx.connect()

    # Configure ElevenLabs TTS and STT with API key from environment variable
    elevenlabs_tts = elevenlabs.TTS(
        api_key=os.environ.get('ELEVENLABS_API_KEY', '')
    )
    elevenlabs_stt = elevenlabs.STT(
        api_key=os.environ.get('ELEVENLABS_API_KEY', ''),
        model_id="scribe_v2_realtime" # Explicitly specify STT model
    )

    # Initialize an AgentSession with ElevenLabs for speech and OpenAI for LLM (example)
    session = AgentSession(
        tts=elevenlabs_tts,
        stt=elevenlabs_stt,
        llm=openai.LLM(
            api_key=os.environ.get('OPENAI_API_KEY', ''),
            model="gpt-4-turbo", 
            instructions="You are a helpful voice AI assistant."
        )
    )

    await session.start(
        room=ctx.room,
        agent=Agent(instructions="You are a helpful voice AI assistant using ElevenLabs for speech.")
    )

    # Agent generates an initial greeting using ElevenLabs TTS
    await session.generate_reply(instructions="Hello! I am a voice assistant powered by LiveKit and ElevenLabs. How can I help you today?")

if __name__ == "__main__":
    # Set LiveKit and ElevenLabs API keys as environment variables for a runnable example
    # export LIVEKIT_URL="wss://your-livekit-instance.cloud"
    # export LIVEKIT_API_KEY="your_livekit_api_key"
    # export LIVEKIT_API_SECRET="your_livekit_api_secret"
    # export ELEVENLABS_API_KEY="your_elevenlabs_api_key"
    # export OPENAI_API_KEY="your_openai_api_key"
    cli.run_app(
        WorkerOptions(
            entrypoint_fnc=entrypoint,
            num_workers=1 
        )
    )

view raw JSON →