pyannote.audio

Warnings

• breaking As of `pyannote.audio` v4.x, all pre-trained models hosted on the Hugging Face Hub require an authentication token to be downloaded. This is a significant change from v3.x, where models could be downloaded without explicit authentication.
  Fix: Obtain a Hugging Face user access token (read role is sufficient) from `https://huggingface.co/settings/tokens`. Pass it via the `use_auth_token` argument to `Pipeline` or `Model` constructors, or log in using `huggingface-cli login`.
• gotcha GPU acceleration with PyTorch requires a specific `torch` installation matching your CUDA version. `pyannote.audio` itself does not install GPU-enabled `torch` by default, leading to CPU-only inference if not correctly set up.
  Fix: Manually install the correct `torch` version for your CUDA toolkit (e.g., `pip install torch torchvision torchaudio --index-url https://download.pytorch.org/whl/cu121` for CUDA 12.1) *before* installing `pyannote.audio`.
• gotcha Input audio files should ideally be mono, 16kHz sample rate, and in a commonly supported format (e.g., WAV). Issues may arise with uncommon codecs, multichannel audio, or significantly different sample rates, potentially leading to errors or suboptimal performance.
  Fix: Pre-process audio to mono, 16kHz before passing to the pipeline. `pyannote.audio` uses `torchaudio` for loading, which handles resampling and channel reduction internally but explicit pre-processing ensures consistency.
• gotcha Model versions (e.g., `pyannote/speaker-diarization-3.1` vs `pyannote/speaker-diarization@main`) can have different performance characteristics, bug fixes, or even breaking changes. Relying on `@main` can lead to unexpected behavior.
  Fix: Always pin to a specific model version (e.g., `"pyannote/speaker-diarization-3.1"`) when instantiating pipelines in production or for reproducible research. Check the Hugging Face model page for available versions.
• gotcha For CPU-only inference, the default PyTorch backend can be significantly slower than optimized runtimes like ONNX. This impacts processing time for long audio files or batch processing.
  Fix: Install `pyannote.audio` with the ONNX extra: `pip install pyannote.audio[onnx]`. The pipeline will automatically try to use ONNX runtime if available and compatible.

Install

• pip install pyannote.audio Basic Installation
• pip install pyannote.audio[onnx] CPU-Optimized (ONNX Runtime)
• pip install torch torchvision torchaudio --index-url https://download.pytorch.org/whl/cu121 pip install pyannote.audio GPU Installation (CUDA 12.1 example)

Imports

• Pipeline

  from pyannote.audio import Pipeline

  The primary Pipeline class is directly available at the top-level `pyannote.audio` package since v2.x. Accessing it from internal modules like `pyannote.audio.core.pipeline` is discouraged and might break in future versions.
• Model

  from pyannote.audio import Model

  For loading individual models directly, e.g., for embedding or Voice Activity Detection.
• Annotation

  from pyannote.core import Annotation

  The core data structure for storing diarization results is part of the `pyannote.core` library.
• Segment

  from pyannote.core import Segment

  Used for representing time segments in annotations and other audio processing tasks, also part of `pyannote.core`.

Quickstart

This quickstart demonstrates how to set up `pyannote.audio`, authenticate with the Hugging Face Hub, and run a speaker diarization pipeline on a dummy audio file. It highlights the critical step of providing an authentication token for model access, a common requirement for `pyannote.audio` models since version 4.0.

import os
import torchaudio
import torch
import numpy as np
import tempfile
import shutil

# 1. Create a dummy audio file for demonstration
duration_seconds = 5
sample_rate = 16000
t = np.linspace(0, duration_seconds, int(sample_rate * duration_seconds), endpoint=False)
audio_data = 0.5 * np.sin(2 * np.pi * 440 * t).astype(np.float32)

temp_dir = tempfile.mkdtemp()
dummy_audio_path = os.path.join(temp_dir, "dummy_audio.wav")
torchaudio.save(dummy_audio_path, torch.from_numpy(audio_data).unsqueeze(0), sample_rate)

# 2. Authenticate with Hugging Face Hub
# Get your Hugging Face token from https://huggingface.co/settings/tokens
# and set it as an environment variable `HF_TOKEN` or replace the placeholder.
hf_token = os.environ.get("HF_TOKEN", "hf_YOUR_HUGGING_FACE_TOKEN_HERE")

if hf_token == "hf_YOUR_HUGGING_FACE_TOKEN_HERE":
    print("WARNING: Please obtain a Hugging Face token from https://huggingface.co/settings/tokens ")
    print("and set the HF_TOKEN environment variable or replace the placeholder in the code.")
    print("Continuing with placeholder token; pipeline initialization might fail without proper authentication.")

# 3. Import and initialize the Pyannote.audio Pipeline
from pyannote.audio import Pipeline
pipeline = Pipeline("pyannote/speaker-diarization-3.1", use_auth_token=hf_token)

# 4. Prepare the audio input
demo_file = {"uri": "dummy_conversation", "audio": dummy_audio_path}

# 5. Run the speaker diarization
di_result = pipeline(demo_file)

# 6. Print the diarization result
print("\nDiarization Result:")
for turn, _, speaker in di_result.itertracks(yield_label=True):
    print(f"start={turn.start:.1f}s stop={turn.end:.1f}s speaker={speaker}")

# 7. Clean up the dummy audio file
shutil.rmtree(temp_dir)
print(f"\nCleaned up temporary audio directory: {temp_dir}")