NVIDIA NeMo Toolkit

2.7.2 · active · verified Sun Apr 12

NeMo is an open-source, PyTorch-based toolkit for developing state-of-the-art conversational AI models, including Automatic Speech Recognition (ASR), Text-to-Speech (TTS), Large Language Models (LLMs), and Natural Language Processing (NLP). It is currently at version 2.7.2 and receives frequent updates, typically bi-monthly or monthly, with patch releases for security and critical fixes.

Warnings

breaking The entire `nemo.collections.nlp` module was removed in NeMo v2.6.1. Any code that imports or uses classes/functions from this module will break.
Fix: Review the NeMo documentation for updated NLP capabilities or alternative approaches for versions 2.6.1 and later. Some functionalities may have been migrated or require different import paths.
gotcha NeMo requires a specific PyTorch version to be installed *before* installing NeMo itself, compatible with your CUDA version (if using GPU). Installing NeMo without pre-installing PyTorch can lead to dependency conflicts or incorrect CUDA setups.
Fix: Always install PyTorch with the correct CUDA version first, then install `nemo_toolkit[all]`. Refer to the official NeMo installation guide for specific PyTorch/CUDA version compatibility.
gotcha Some functionalities, especially for ASR, had compatibility issues with NumPy 2.0 prior to NeMo v2.6.1.
Fix: Upgrade to NeMo v2.6.1 or newer to ensure full compatibility with NumPy 2.0. If unable to upgrade, you might need to pin NumPy to an older version (e.g., `numpy<2.0`).
gotcha Users of `numba-cuda` and `cuda-python` packages (often implicit dependencies for GPU acceleration) experienced installation and usage issues in earlier versions.
Fix: Ensure you are on NeMo v2.7.2 or newer, as specific fixes for these packages were introduced in recent releases. If issues persist, verify your CUDA toolkit installation and driver versions.
gotcha NeMo models (especially large language models or pre-trained ASR/TTS models) require significant GPU memory and disk space for downloading model checkpoints. Running on CPU is possible but much slower.
Fix: Ensure you have adequate GPU VRAM (e.g., 8GB+ for many models) and sufficient disk space. For production, consider deploying on NVIDIA GPUs. For local development, be mindful of model sizes.

Install

pip install torch==2.1.2 torchvision==0.16.2 torchaudio==2.1.2 --index-url https://download.pytorch.org/whl/cu121 pip install nemo_toolkit[all] Recommended (CUDA 12.1 example)
pip install nemo_toolkit[all] Basic (requires pre-installed PyTorch)

Imports

nemo.collections.asr
```
import nemo.collections.asr as nemo_asr
```
nemo.collections.tts
```
import nemo.collections.tts as nemo_tts
```
nemo.collections.nlp
```
This module was removed in NeMo 2.6.1. Refer to NeMo documentation for alternatives.
```
The entire `nemo.collections.nlp` module was removed in NeMo v2.6.1. Code relying on it will break. Check the latest documentation for updated NLP functionalities.

Quickstart

This quickstart demonstrates how to load a pre-trained ASR model and transcribe an audio file. The model will be downloaded automatically on the first run. Ensure you have a `.wav` audio file (preferably 16kHz mono) at the specified `filepath`.

import nemo.collections.asr as nemo_asr

# This will download and load the pretrained model from NVIDIA's NGC cloud
# The first run takes time to download the model (~1.5 GB)
asr_model = nemo_asr.models.EncDecRNNTModel.from_pretrained(model_name="stt_en_fastconformer_hybrid_large_ctc_rnnt")

# Path to an audio file (replace with your own or download an example)
# For demonstration, we'll use a placeholder. In a real scenario, you'd have an actual .wav file.
# Example audio can be found in NeMo's tutorials or downloaded from public datasets.
# For example: !wget https://nemo-public.s3.us-east-2.amazonaws.com/example_samples/audio_0.wav
filepath = "./audio_0.wav" # Ensure this file exists for the code to run

# For demonstration, let's create a dummy file if it doesn't exist
import os
if not os.path.exists(filepath):
    try:
        import torchaudio
        import torch
        sample_rate = 16000
        duration_seconds = 5
        waveform = torch.sin(2 * torch.pi * 440 * torch.arange(0, sample_rate * duration_seconds) / sample_rate).unsqueeze(0)
        torchaudio.save(filepath, waveform, sample_rate)
        print(f"Created dummy audio file: {filepath}")
    except ImportError:
        print(f"Warning: '{filepath}' not found and torchaudio not installed to create a dummy file. Quickstart may fail.")


transcriptions = asr_model.transcribe([filepath])
print(f"Transcription: {transcriptions[0]}")

view raw JSON →