Rust Stemmers for Python

Warnings

• gotcha When installing from source or in environments like Docker/CI without pre-built wheels, `py-rust-stemmers` requires the Rust toolchain and `maturin` to be present for compilation. This can add complexity to build pipelines.
  Fix: Ensure Rust and `maturin` are installed in your build environment, or rely on pre-built wheels where available. For example: `curl --proto '=https' --tlsv1.2 -sSf https://sh.rustup.rs | sh` followed by `pip install maturin`.
• gotcha Snowball stemming algorithms aim to reduce words to a common root form, which may not always be a dictionary word or a true lemma. If your application requires actual dictionary words or full lemmatization, Snowball stemmers (and thus `py-rust-stemmers`) may not be the appropriate solution.
  Fix: Understand the distinction between stemming and lemmatization. If full lemmatization is needed, consider libraries like spaCy or NLTK's WordNetLemmatizer, which use linguistic knowledge bases.
• gotcha Errors originating from the underlying Rust code that result in a `panic!` in Rust will typically manifest as a `pyo3_runtime.PanicException` in Python. This might be less specific than expected Python exceptions, making granular error handling challenging without explicit Rust-to-Python exception mapping.
  Fix: Implement robust error handling in Rust (using `Result` types and mapping to specific `PyErr` types like `PyValueError`) if more precise Python exceptions are required. Handle `PanicException` as a general fallback for unrecoverable Rust errors.

Install

• pip install py-rust-stemmers Install latest version

Imports

• SnowballStemmer

  from py_rust_stemmers import SnowballStemmer

Quickstart

Initialize a `SnowballStemmer` for a specific language and then use `stem_word`, `stem_words`, or `stem_words_parallel` for single word, batch, or parallel stemming, respectively.

from py_rust_stemmers import SnowballStemmer

# Initialize the stemmer for the English language
s = SnowballStemmer('english')

text = """This stem form is often a word itself, but this is not always the case as this is not a requirement for text search systems, which are the intended field of use. We also aim to conflate words with the same meaning, rather than all words with a common linguistic root (so awe and awful don't have the same stem), and over-stemming is more problematic than under-stemming so we tend not to stem in cases that are hard to resolve. If you want to always reduce words to a root form and/or get a root form which is itself a word then Snowball's stemming algorithms likely aren't the right answer."""
words = text.split()

# Stem a single word
stemmed_word = s.stem_word(words[0])
print(f"Stemmed word: {stemmed_word}")

# Stem a list of words
stemmed_words = s.stem_words(words)
print(f"Stemmed words: {stemmed_words}")

# Stem words in parallel (for larger text sequences)
stemmed_words_parallel = s.stem_words_parallel(words)
print(f"Stemmed words (parallel): {stemmed_words_parallel}")