{"id":8505,"library":"pynini","title":"Pynini","description":"Pynini is a Python library providing efficient Python bindings for the OpenFst C++ library, enabling the construction, manipulation, and compilation of finite-state transducers (FSTs) and finite-state acceptors (FSAs). It's widely used for tasks in natural language processing (NLP) such as text normalization, phonology, and speech recognition. The current version is 2.1.7, with active development and regular releases.","status":"active","version":"2.1.7","language":"en","source_language":"en","source_url":"https://github.com/kylebgorman/pynini","tags":["NLP","FSM","finite-state","grammar","compiler","transducer","acceptor"],"install":[{"cmd":"pip install pynini","lang":"bash","label":"Install with pip"}],"dependencies":[{"reason":"Pynini is a Python binding for the OpenFst C++ library. Pre-compiled wheels typically bundle OpenFst, but on some systems or for custom builds, OpenFst may need to be installed separately.","package":"OpenFst","optional":true}],"imports":[{"note":"Access as pynini.Fst to create or manipulate finite-state transducers.","symbol":"Fst","correct":"import pynini"},{"note":"Access as pynini.string_file to compile OpenFst text files into FSTs.","symbol":"string_file","correct":"import pynini"},{"note":"Access as pynini.union to create an FST that accepts the union of input FSTs or strings.","symbol":"union","correct":"import pynini"},{"note":"Access as pynini.accep to create an FSA (acceptor) from a string.","symbol":"accep","correct":"import pynini"},{"note":"Access as pynini.compose to compose two FSTs.","symbol":"compose","correct":"import pynini"}],"quickstart":{"code":"import pynini as pn\n\n# Create a simple acceptor for 'hello'\nhello_fst = pn.accep(\"hello\")\n\n# Create an acceptor for 'world'\nworld_fst = pn.accep(\"world\")\n\n# Concatenate them with a space acceptor using the overloaded '+' operator\nhello_world_fst = hello_fst + pn.accep(\" \") + world_fst\n\nprint(f\"Hello World FST: {hello_world_fst}\")\nprint(f\"Shortest path for Hello World: {pn.shortestpath(hello_world_fst).string()}\")\n\n# Create a simple transducer mapping 'a' to 'b'\na_to_b_map = pn.string_map([(\"a\", \"b\"), (\"c\", \"d\")])\n\n# Input string as an acceptor\ninput_string_fst = pn.accep(\"apple_cake\")\n\n# Compose the input with the transducer\noutput_fst = pn.compose(input_string_fst, a_to_b_map)\n\n# Get the shortest path (result string)\nif not output_fst.empty():\n    print(f\"'apple_cake' -> '{pn.shortestpath(output_fst).string()}'\")\nelse:\n    print(\"No valid output path found.\")","lang":"python","description":"This quickstart demonstrates creating basic finite-state acceptors and transducers, concatenating FSTs, and composing an input string with a mapping transducer to transform text."},"warnings":[{"fix":"Ensure all FSTs involved in an operation use the same weight semiring. You can explicitly set the semiring when creating FSTs (e.g., `pn.Fst(weight_type=\"log\")`) or convert them using `pn.cast()`.","message":"Pynini FSTs operate within specific weight semirings (e.g., Tropical, Log, Probability). The default is Tropical. Operations (like composition, union) on FSTs with different semirings will result in a runtime error.","severity":"gotcha","affected_versions":"All versions"},{"fix":"Always assign the result of FST operations to a new variable or reassign to the original variable to capture the modified FST (e.g., `my_fst = pn.compose(my_fst, another_fst)`).","message":"Pynini FST objects are generally immutable. Operations like `compose()`, `union()`, `closure()`, `concat()` return new FST objects rather than modifying the original in place.","severity":"gotcha","affected_versions":"All versions"},{"fix":"Ensure your system has necessary build tools (g++, make, etc.). For persistent issues, consider installing OpenFst manually according to its documentation and then compiling Pynini from source, ensuring it links to your specific OpenFst installation.","message":"Despite `pip install pynini` usually working, Pynini is a C++ library binding. On some Linux distributions, macOS versions, or for advanced compilation flags, you might encounter issues with the bundled OpenFst leading to linker errors or missing symbols.","severity":"gotcha","affected_versions":"All versions"},{"fix":"Optimize grammar design by minimizing states/arcs where possible. Consider breaking down complex operations into smaller, manageable steps. Profile your code to identify bottlenecks and explore Pynini's various optimization flags or algorithms (e.g., `pn.optimize()`)","message":"Working with very large grammars or performing complex FST operations can be memory and CPU intensive, potentially leading to slow execution or out-of-memory errors.","severity":"gotcha","affected_versions":"All versions"}],"env_vars":null,"last_verified":"2026-04-16T00:00:00.000Z","next_check":"2026-07-15T00:00:00.000Z","problems":[{"fix":"Convert the Python string into a finite-state acceptor using `pynini.accep()` before concatenation. Example: `my_fst + pynini.accep(\"literal_string\")`.","cause":"Attempting to concatenate a `pynini.Fst` object directly with a Python string.","error":"TypeError: unsupported operand type(s) for +: 'pynini.Fst' and 'str'"},{"fix":"Carefully examine the input string, text file, or arguments passed to functions like `pynini.string_map()` or `pynini.string_file()`. Ensure correct OpenFst FST text format, proper character escaping, and valid UTF-8 encoding. Test with smaller, simpler inputs to isolate the issue.","cause":"This error indicates that the underlying OpenFst library failed to compile the provided grammar or string. Common causes include malformed FST text syntax, invalid UTF-8 characters, or exceeding OpenFst's internal limits.","error":"pynini.exceptions.FstCompilerException: OpenFst compilation failed for ..."},{"fix":"Verify that the file path is absolutely correct and that the file exists at that location. Ensure the Python process has the necessary read permissions for the file. Use an absolute path or ensure the file is in the current working directory.","cause":"Typically occurs when `pynini.string_file()` is called with a path to a non-existent file, or the Python process lacks read permissions for the specified file.","error":"FileNotFoundError: No such file or directory: '...' (OpenFst)"}]}