LiveKit Noise Cancellation Plugin

0.2.5 · active · verified Mon Apr 13

livekit-plugins-noise-cancellation is a Python plugin that provides real-time enhanced noise cancellation for inbound audio streams within LiveKit Agents. It offers models for standard noise cancellation (NC), background voice cancellation (BVC), and BVC optimized for telephony. The current version is 0.2.5, and the project maintains a healthy release cadence.

Warnings

gotcha This plugin requires a LiveKit Cloud account. Enhanced noise cancellation is a feature of paid LiveKit Cloud accounts and consumes real connection minutes during use.
Fix: Ensure you have an active LiveKit Cloud account and understand the associated usage costs.
gotcha Noise cancellation should only be applied once in the audio pathway. If you enable this plugin in your agent, you should disable any noise cancellation or Krisp filter in your frontend clients to avoid unexpected audio degradation.
Fix: Disable client-side or frontend noise cancellation features when using this plugin in LiveKit Agents.
gotcha Users have reported unexpected audio degradation with the BVC (Background Voice Cancellation) model. This might manifest as the processed audio sounding 'different' or lower quality than expected.
Fix: Experiment with different noise cancellation models (e.g., NC instead of BVC) or ensure client-side echo cancellation is disabled. Consider monitoring agent observability metrics for audio quality insights.
gotcha Crashes on AMD CPUs have been reported, potentially due to OpenBLAS's CPU detection failing. This can cause agent connection issues or silent failures.
Fix: Manually set the `OPENBLAS_CORETYPE` environment variable to a more conservative value (e.g., `Haswell`).
gotcha Plugin initialization, including `livekit-plugins-noise-cancellation`, can occasionally cause LiveKit Agents to stall or crash before connecting to a room, especially in cloud deployments.
Fix: Temporarily remove the noise cancellation plugin during debugging to identify if it's the culprit. Wrap plugin initialization in `try/except` blocks and log failures explicitly, allowing the session to continue without the plugin if it fails.

Install

pip install livekit-plugins-noise-cancellation Install with pip

Imports

noise_cancellation

from livekit.plugins import noise_cancellation

NC
```
noise_cancellation.NC()
```
Standard enhanced noise cancellation model.
BVC
```
noise_cancellation.BVC()
```
Background voice cancellation model.
BVCTelephony
```
noise_cancellation.BVCTelephony()
```
Background voice cancellation model optimized for telephony applications.

Quickstart

This quickstart demonstrates how to integrate `livekit-plugins-noise-cancellation` into a LiveKit Agent. The plugin is applied by setting the `noise_cancellation` option within `RoomInputOptions` when starting an `AgentSession`. This ensures that all inbound audio streams to the agent are processed for noise reduction using the specified model (e.g., BVC). Remember that this plugin requires LiveKit Cloud for enhanced noise cancellation services.

import os
from livekit.agents import AgentSession, JobContext, RoomOptions, RoomInputOptions
from livekit.plugins import noise_cancellation

class MyAgent:
    async def start(self, ctx: JobContext):
        # Example of applying BVC noise cancellation to inbound audio
        await ctx.connect(
            RoomOptions(auto_subscribe=RoomOptions.AutoSubscribe.AUDIO_ONLY),
            room_input_options=RoomInputOptions(
                noise_cancellation=noise_cancellation.BVC(),
            ),
        )
        print("Agent connected with BVC noise cancellation enabled.")
        # Your agent logic here, e.g., process audio from ctx.room.local_participant.audio_tracks
        # Keep the agent alive (e.g., by waiting for a signal or keeping a loop)
        # For a simple quickstart, we'll just wait indefinitely for demonstration.
        # In a real agent, you'd manage session lifecycle more robustly.
        await ctx.room.wait_for_is_connected()

async def main():
    # This quickstart assumes a LiveKit Agent environment setup
    # In a real scenario, AgentSession would be started by the LiveKit server
    # with appropriate environment variables for authentication.
    LIVEKIT_URL = os.environ.get('LIVEKIT_URL', 'wss://your-livekit-url.cloud')
    LIVEKIT_API_KEY = os.environ.get('LIVEKIT_API_KEY', 'your_api_key')
    LIVEKIT_API_SECRET = os.environ.get('LIVEKIT_API_SECRET', 'your_api_secret')
    
    # This part typically runs on a LiveKit Agent service, not as a standalone script.
    # For local testing, you might mock AgentSession or connect to a local LiveKit instance.
    print("To run this, ensure LIVEKIT_URL, LIVEKIT_API_KEY, and LIVEKIT_API_SECRET are set.")
    print("This code snippet demonstrates plugin usage within a LiveKit AgentSession.")

if __name__ == '__main__':
    # In a typical LiveKit Agents deployment, you wouldn't directly call main here.
    # The agent system would instantiate and manage MyAgent.
    # This is for illustration of the noise_cancellation plugin usage.
    import asyncio
    asyncio.run(main())

view raw JSON →