Pynini

Common errors

• TypeError: unsupported operand type(s) for +: 'pynini.Fst' and 'str'

  cause Attempting to concatenate a `pynini.Fst` object directly with a Python string.
  fix Convert the Python string into a finite-state acceptor using `pynini.accep()` before concatenation. Example: `my_fst + pynini.accep("literal_string")`.

• pynini.exceptions.FstCompilerException: OpenFst compilation failed for ...

  cause This error indicates that the underlying OpenFst library failed to compile the provided grammar or string. Common causes include malformed FST text syntax, invalid UTF-8 characters, or exceeding OpenFst's internal limits.
  fix Carefully examine the input string, text file, or arguments passed to functions like `pynini.string_map()` or `pynini.string_file()`. Ensure correct OpenFst FST text format, proper character escaping, and valid UTF-8 encoding. Test with smaller, simpler inputs to isolate the issue.

• FileNotFoundError: No such file or directory: '...' (OpenFst)

  cause Typically occurs when `pynini.string_file()` is called with a path to a non-existent file, or the Python process lacks read permissions for the specified file.
  fix Verify that the file path is absolutely correct and that the file exists at that location. Ensure the Python process has the necessary read permissions for the file. Use an absolute path or ensure the file is in the current working directory.

Warnings

• gotcha Pynini FSTs operate within specific weight semirings (e.g., Tropical, Log, Probability). The default is Tropical. Operations (like composition, union) on FSTs with different semirings will result in a runtime error.
  Fix: Ensure all FSTs involved in an operation use the same weight semiring. You can explicitly set the semiring when creating FSTs (e.g., `pn.Fst(weight_type="log")`) or convert them using `pn.cast()`.
• gotcha Pynini FST objects are generally immutable. Operations like `compose()`, `union()`, `closure()`, `concat()` return new FST objects rather than modifying the original in place.
  Fix: Always assign the result of FST operations to a new variable or reassign to the original variable to capture the modified FST (e.g., `my_fst = pn.compose(my_fst, another_fst)`).
• gotcha Despite `pip install pynini` usually working, Pynini is a C++ library binding. On some Linux distributions, macOS versions, or for advanced compilation flags, you might encounter issues with the bundled OpenFst leading to linker errors or missing symbols.
  Fix: Ensure your system has necessary build tools (g++, make, etc.). For persistent issues, consider installing OpenFst manually according to its documentation and then compiling Pynini from source, ensuring it links to your specific OpenFst installation.
• gotcha Working with very large grammars or performing complex FST operations can be memory and CPU intensive, potentially leading to slow execution or out-of-memory errors.
  Fix: Optimize grammar design by minimizing states/arcs where possible. Consider breaking down complex operations into smaller, manageable steps. Profile your code to identify bottlenecks and explore Pynini's various optimization flags or algorithms (e.g., `pn.optimize()`)

Install

• pip install pynini Install with pip

Imports

• Fst

  import pynini

  Access as pynini.Fst to create or manipulate finite-state transducers.
• string_file

  import pynini

  Access as pynini.string_file to compile OpenFst text files into FSTs.
• union

  import pynini

  Access as pynini.union to create an FST that accepts the union of input FSTs or strings.
• accep

  import pynini

  Access as pynini.accep to create an FSA (acceptor) from a string.
• compose

  import pynini

  Access as pynini.compose to compose two FSTs.

Quickstart

This quickstart demonstrates creating basic finite-state acceptors and transducers, concatenating FSTs, and composing an input string with a mapping transducer to transform text.

import pynini as pn

# Create a simple acceptor for 'hello'
hello_fst = pn.accep("hello")

# Create an acceptor for 'world'
world_fst = pn.accep("world")

# Concatenate them with a space acceptor using the overloaded '+' operator
hello_world_fst = hello_fst + pn.accep(" ") + world_fst

print(f"Hello World FST: {hello_world_fst}")
print(f"Shortest path for Hello World: {pn.shortestpath(hello_world_fst).string()}")

# Create a simple transducer mapping 'a' to 'b'
a_to_b_map = pn.string_map([("a", "b"), ("c", "d")])

# Input string as an acceptor
input_string_fst = pn.accep("apple_cake")

# Compose the input with the transducer
output_fst = pn.compose(input_string_fst, a_to_b_map)

# Get the shortest path (result string)
if not output_fst.empty():
    print(f"'apple_cake' -> '{pn.shortestpath(output_fst).string()}'")
else:
    print("No valid output path found.")