stop-words

Common errors

• Words like 'The' or 'Is' are not removed, even when 'the' or 'is' are in the stop word list.

  cause Input text words are not converted to lowercase before comparison with the (lowercase) stop word list. [8]
  fix Ensure all words in your input text are converted to lowercase (e.g., `word.lower()`) before checking them against the stop words.

• Words with trailing punctuation (e.g., 'example.') are not being filtered out by the stop word removal process.

  cause Punctuation attached to words prevents an exact match with the clean stop words in the list. [8]
  fix Strip punctuation from words (e.g., using `str.strip(string.punctuation)` or regular expressions) before checking them against the stop word list.

• I modified `english.txt` (or another language file) in the installed package, but `get_stop_words('en')` still returns the old list.

  cause The library employs internal caching, and direct modification of files within the installed package doesn't trigger a cache refresh. This is generally an anti-pattern for package customization. [1, 9, 21]
  fix Instead of modifying package files, get the list programmatically and add/remove items (e.g., `my_stop_words = set(get_stop_words('en')) | {'my_new_word'}`). If absolutely necessary, clear the cache using `from stop_words import STOP_WORDS_CACHE; STOP_WORDS_CACHE.clear()`.

• Calling `get_stop_words('unsupported_lang_code')` returns an empty list, but I expected an error or the language to be supported.

  cause The requested language code or name is either misspelled or genuinely not included in the library's collection. `get_stop_words` silently returns an empty list for unsupported languages. [1, 9]
  fix Verify the language code/name against the library's available languages (often listed in the GitHub README or PyPI page). If the language is truly unsupported, you'll need to provide your own stop word list.

Warnings

• gotcha The library caches stop word lists by default for performance. Directly modifying the raw text files (e.g., `english.txt`) within the installed package directory will likely not update the loaded stop words unless the cache is cleared or the application is restarted. [1, 9, 21]
  Fix: Interact with the returned list (e.g., `my_list = get_stop_words('en'); my_list.append('custom_word')`) or explicitly clear the cache (`from stop_words import STOP_WORDS_CACHE; STOP_WORDS_CACHE.clear()`).
• gotcha The `get_stop_words()` function returns an empty list if the requested language is not supported or recognized, rather than raising an error. This can lead to silent failures if not explicitly handled or checked. [1, 9]
  Fix: Always check if the returned list is empty, or use `safe_get_stop_words()` if an empty list is the desired fallback for unsupported languages. Consult documentation for available language codes.
• gotcha Stop word lists are typically in lowercase. Input text containing capitalized words (e.g., 'The') or words with punctuation (e.g., 'word.') will not match their lowercase, punctuation-free counterparts in the stop word list, leading to them not being filtered. [8]
  Fix: Always normalize your input text by converting words to lowercase (e.g., `.lower()`) and stripping punctuation before comparing them to the stop word list.

Install

• pip install stop-words Install latest version

Imports

• get_stop_words

  from stop_words import get_stop_words

  Primary function to retrieve stop words for a specified language.
• safe_get_stop_words

  from stop_words import safe_get_stop_words

  A safer alternative that handles unsupported languages gracefully (returns empty list).
• STOP_WORDS_CACHE

  from stop_words import STOP_WORDS_CACHE

  Access the internal cache for advanced control, e.g., clearing or inspecting cached languages.

Quickstart

Demonstrates how to fetch stop words for English and Spanish and apply them to a simple text string. It highlights the importance of lowercasing and punctuation removal for effective filtering. [1, 9]

from stop_words import get_stop_words

# Get English stop words
english_stop_words = get_stop_words('en')
print(f"English stop words (first 5): {english_stop_words[:5]}")

# Get Spanish stop words using full name
spanish_stop_words = get_stop_words('spanish')
print(f"Spanish stop words (first 5): {spanish_stop_words[:5]}")

# Example usage in text processing
text = "This is a sample sentence, demonstrating stop word removal."
filtered_words = [word.lower() for word in text.replace(',', '').replace('.', '').split() if word.lower() not in english_stop_words]
print(f"Filtered text: {' '.join(filtered_words)}")