{"id":10086,"library":"pyctcdecode","title":"pyctcdecode","description":"pyctcdecode is a Python library that provides a standalone beam search decoder for CTC (Connectionist Temporal Classification) models. It allows for efficient decoding of CTC outputs and seamlessly integrates with KenLM language models to improve speech recognition accuracy. The current version is 0.5.0, and it follows an active release cadence, with updates addressing features, performance, and bug fixes.","status":"active","version":"0.5.0","language":"en","source_language":"en","source_url":"https://github.com/kensho-technologies/pyctcdecode","tags":["CTC","speech recognition","beam search","language model","KenLM"],"install":[{"cmd":"pip install pyctcdecode","lang":"bash","label":"Base installation"},{"cmd":"pip install pyctcdecode[kenlm]","lang":"bash","label":"Installation with KenLM support"}],"dependencies":[{"reason":"Optional, required for language model integration, which significantly improves decoding accuracy for speech recognition tasks. Its installation often requires system-level C++ build tools.","package":"kenlm","optional":true},{"reason":"Core dependency for numerical operations.","package":"numpy","optional":false},{"reason":"Core dependency for scientific computing functions.","package":"scipy","optional":false},{"reason":"Used for progress bars.","package":"tqdm","optional":false},{"reason":"Used for efficient prefix tree operations in beam search.","package":"pygtrie","optional":false}],"imports":[{"symbol":"BeamSearchDecoderCTC","correct":"from pyctcdecode import BeamSearchDecoderCTC"},{"symbol":"LanguageModel","correct":"from pyctcdecode import LanguageModel"},{"symbol":"Alphabet","correct":"from pyctcdecode.alphabet import Alphabet"},{"symbol":"BLANK_TOKEN","correct":"from pyctcdecode.alphabet import BLANK_TOKEN"},{"symbol":"get_alphabet","correct":"from pyctcdecode.alphabet import get_alphabet"}],"quickstart":{"code":"from pyctcdecode import BeamSearchDecoderCTC\nfrom pyctcdecode.alphabet import BLANK_TOKEN, get_alphabet\nimport numpy as np\n\n# Define your model's alphabet. The BLANK_TOKEN must be the first element.\n# This example uses a common alphabet for English speech recognition.\nlabels = [BLANK_TOKEN] + list(\"abcdefghijklmnopqrstuvwxyz '\")\nalphabet = get_alphabet(labels)\n\n# Create dummy CTC output (logits) for demonstration.\n# In a real scenario, these would come from your deep learning model.\n# Shape: (time_steps, alphabet_size)\ntime_steps = 50\nlogits = np.random.rand(time_steps, len(labels)).astype(np.float32)\n\n# Initialize the decoder without a language model.\n# For better accuracy, integrate with a KenLM language model (see warnings).\ndecoder = BeamSearchDecoderCTC(alphabet)\n\n# Decode the logits. The decode method returns a list of hypotheses.\n# We take the first (most probable) one.\nhypotheses = decoder.decode(logits)\ndecoded_text = hypotheses[0]\n\nprint(f\"Decoded text (example): {decoded_text}\")\n","lang":"python","description":"This quickstart demonstrates how to initialize `BeamSearchDecoderCTC` with a custom alphabet and decode dummy CTC logits. It shows the basic usage without a language model. For real-world applications, integrating with a KenLM language model is highly recommended for improved accuracy."},"warnings":[{"fix":"For common Linux distributions, ensure development headers for `boost`, `zlib`, `bzip2`, and `cmake` are installed. For example: `sudo apt-get install libboost-all-dev liblzma-dev libbz2-dev cmake`. Then retry `pip install pyctcdecode[kenlm]` or consult KenLM's specific build instructions if issues persist.","message":"The KenLM C++ library, an optional but highly recommended dependency for language model integration, can be challenging to install due to its native compilation requirements (e.g., Boost, Zlib, Bzip2, CMake).","severity":"gotcha","affected_versions":"All versions requiring KenLM (0.1.0+)"},{"fix":"Always construct your alphabet list by placing `BLANK_TOKEN` (or an empty string) as the first element. Example: `labels = [BLANK_TOKEN] + list('abc')`. Ensure the total length of the alphabet matches your model's CTC output dimension.","message":"The CTC blank token MUST be the first element in the alphabet list provided to `get_alphabet` or `Alphabet` constructor. Incorrect positioning will lead to incorrect decoding results or runtime errors.","severity":"gotcha","affected_versions":"All versions"},{"fix":"Ensure the `.arpa` file exists and is accessible. Verify its format using KenLM's tools if possible. For very large models, consider reducing the beam width or using a smaller language model for initial testing to mitigate performance issues.","message":"Loading a KenLM language model requires a valid `.arpa` file. Incorrect file paths, malformed `.arpa` files, or extremely large language models can cause `FileNotFoundError`, `MemoryError`, or excessively slow initialization.","severity":"gotcha","affected_versions":"All versions using KenLM"}],"env_vars":null,"last_verified":"2026-04-17T00:00:00.000Z","next_check":"2026-07-16T00:00:00.000Z","problems":[{"fix":"Install with KenLM support using `pip install pyctcdecode[kenlm]`. If this fails, review system dependencies for KenLM (Boost, Zlib, Bzip2, CMake) as described in the warnings.","cause":"The `pyctcdecode[kenlm]` extra was not installed, or its installation failed during compilation, preventing the import of the `kenlm` module required for language model functionality.","error":"ModuleNotFoundError: No module named 'kenlm'"},{"fix":"Install Boost development headers on your system (e.g., `sudo apt-get install libboost-all-dev` on Debian/Ubuntu, or `brew install boost` on macOS). Ensure `cmake` is also installed. If Boost is in a non-standard location, you might need to manually set environment variables like `BOOST_ROOT` or `BOOST_INCLUDEDIR`.","cause":"During `pyctcdecode[kenlm]` installation, the underlying KenLM C++ compilation process failed to locate the necessary Boost library, which is a critical dependency.","error":"error: Boost library not found or not configured. Set BOOST_ROOT or BOOST_INCLUDEDIR to point to the Boost install."},{"fix":"Verify that your `labels` list used to create the `alphabet` has the `BLANK_TOKEN` (or an empty string) as its first element, and that the total length of your alphabet exactly matches the last dimension of your CTC model's output logits.","cause":"This often occurs when the `alphabet` passed to the decoder does not correctly map to the CTC model's output dimension, or more commonly, if the blank token is not positioned as the very first element in the alphabet list.","error":"IndexError: list index out of range"}]}