{"id":5276,"library":"kaldiio","title":"Kaldi-ark loading and writing module","description":"kaldiio is a pure Python module for reading and writing Kaldi ark and scp files. It provides utilities for handling various Kaldi object types, including binary/text matrices, vectors, and compressed matrices, by interacting with NumPy arrays. The library also facilitates I/O operations through Unix pipes and supports some extended ark formats, including those used for NumPy, Pickle, WAV, and FLAC files. It is actively maintained with frequent minor and patch releases.","status":"active","version":"2.18.1","language":"en","source_language":"en","source_url":"https://github.com/nttcslab-sp/kaldiio","tags":["audio","speech-recognition","kaldi","data-io","ark","scp","numpy"],"install":[{"cmd":"pip install kaldiio","lang":"bash","label":"Install stable version"}],"dependencies":[{"reason":"Fundamental for handling matrix and vector data read from/written to Kaldi archives. While not a direct PyPI dependency, it's implicitly required for almost all practical usage.","package":"numpy","optional":false}],"imports":[{"symbol":"load_ark","correct":"from kaldiio import load_ark"},{"symbol":"load_scp","correct":"from kaldiio import load_scp"},{"note":"save_ark is a function directly exposed in the top-level package for convenience, not typically accessed as a submodule.","wrong":"import kaldiio.save_ark","symbol":"save_ark","correct":"from kaldiio import save_ark"},{"symbol":"ReadHelper","correct":"from kaldiio import ReadHelper"},{"symbol":"WriteHelper","correct":"from kaldiio import WriteHelper"},{"symbol":"open_like_kaldi","correct":"from kaldiio import open_like_kaldi"}],"quickstart":{"code":"import numpy as np\nfrom kaldiio import load_scp, ReadHelper, WriteHelper, save_ark\n\n# Example: Create dummy data for writing\nutt_ids = ['utt1', 'utt2', 'utt3']\nfeatures = {\n    'utt1': np.random.rand(100, 40).astype(np.float32),\n    'utt2': np.random.rand(120, 40).astype(np.float32),\n    'utt3': np.random.rand(90, 40).astype(np.float32)\n}\n\n# Write features to an ARK file and generate an SCP file\noutput_ark_path = 'output.ark'\noutput_scp_path = 'output.scp'\nsave_ark(output_ark_path, features, scp=output_scp_path)\nprint(f\"Written to {output_ark_path} and {output_scp_path}\")\n\n# Read SCP file using ReadHelper (sequential access, supports pipes)\nprint(\"\\nReading via ReadHelper (scp:)...\")\nwith ReadHelper(f'scp:{output_scp_path}') as reader:\n    for key, matrix in reader:\n        print(f\"Key: {key}, Matrix shape: {matrix.shape}\")\n\n# Read SCP file using load_scp (random access, returns dict-like object)\nprint(\"\\nReading via load_scp (random access)...\")\nloaded_features = load_scp(output_scp_path)\nprint(f\"Loaded 'utt2' shape: {loaded_features['utt2'].shape}\")\n\n# Cleanup (optional)\nimport os\nos.remove(output_ark_path)\nos.remove(output_scp_path)","lang":"python","description":"This quickstart demonstrates how to write a dictionary of NumPy arrays to a Kaldi .ark file and generate a corresponding .scp file using `save_ark`. It then shows two common methods for reading data: `ReadHelper` for sequential iteration (supporting Kaldi-style specifiers and pipes) and `load_scp` for random access to features by utterance ID."},"warnings":[{"fix":"Ensure you are importing directly from `kaldiio` (`from kaldiio import ...`) and that it is the installed package, not an indirect dependency or a different library.","message":"Users often confuse `kaldiio` with older or related libraries like `kaldi_io` (from Karel Vesely) or `torchaudio.kaldi_io`. The `torchaudio.kaldi_io` module, in particular, is deprecated since version 2.8 and will be removed in 2.9, potentially leading to breaking changes if not using `kaldiio` directly.","severity":"gotcha","affected_versions":"All versions"},{"fix":"For WAV file I/O, use the dedicated `kaldiio.wavio` functions directly or leverage `kaldiio`'s support for reading WAV/FLAC files via extended ark formats if applicable. Review the `ReadHelper` documentation for current functionality.","message":"The `wav` option, previously available in `ReadHelper` for WAV file processing, was removed in version 2.11.0. Code relying on this option will break.","severity":"breaking","affected_versions":">=2.11.0"},{"fix":"Upgrade `kaldiio` to the latest version to ensure compatibility with all supported `.ark` formats, especially if you are working with recently generated Kaldi data.","message":"Version 2.16.0 introduced support for an 'extended ark format'. Older versions of `kaldiio` (prior to 2.16.0) might not be able to correctly read or fully utilize `.ark` files created with these extended formats by newer `kaldiio` versions or other Kaldi tools.","severity":"breaking","affected_versions":"<2.16.0 (reading newer formats)"},{"fix":"Carefully construct your specifiers. For debugging pipe issues, try running the pipe command directly in the shell to isolate the problem. `kaldiio`'s `open_like_kaldi` and `ReadHelper` are designed to abstract away some of this complexity, but understanding Kaldi's I/O conventions is beneficial.","message":"Kaldi's I/O often involves complex `rspecifiers` and `wspecifiers` which can include UNIX pipes (e.g., `ark:gunzip -c file.ark.gz |`). Incorrect formatting or environment issues with pipe commands are common pitfalls. `kaldiio` explicitly supports 'pipe fashion' for functions like `load_scp` (since v2.15.0) and `ReadHelper`.","severity":"gotcha","affected_versions":"All versions"}],"env_vars":null,"last_verified":"2026-04-13T00:00:00.000Z","next_check":"2026-07-12T00:00:00.000Z"}