text-unidecode

1.3 · active · verified Sat Mar 28

text-unidecode is a basic and lightweight Python port of the Perl Text::Unidecode library. It converts Unicode text into a 'good enough' 7-bit ASCII representation by performing context-free, character-by-character transliteration. The current version is 1.3, released in 2019, indicating a stable but infrequently updated project.

Warnings

gotcha text-unidecode performs basic, context-free character-by-character transliteration. This can lead to inaccurate or unexpected results for languages with complex writing systems (e.g., Japanese, Thai) or for characters with language-specific transliteration conventions (e.g., German umlauts, which are mapped to 'A', 'O', 'U' instead of 'Ae', 'Oe', 'Ue'). It's not designed for high-quality, language-aware transliteration.
Fix: For critical applications, evaluate the output carefully or pre-process strings for specific language rules. For better transliteration quality, consider language-specific libraries or the alternative 'unidecode' package if its license is acceptable.
breaking The transliteration tables within the library can be updated in future versions to improve mappings or fix inconsistencies. This means that the ASCII output for a given Unicode input might change between versions, potentially breaking systems that rely on consistent, stable ASCII representations (e.g., for generating unique URL slugs or identifiers).
Fix: If output stability is crucial (e.g., for URL slugs), either store the generated ASCII string in a database or pin the 'text-unidecode' dependency to a specific version to prevent unexpected changes.
gotcha The `text-unidecode` project's own documentation suggests an alternative library, `unidecode`, which is also a port of Text::Unidecode. It states that `unidecode` offers 'better memory usage and better transliteration quality' but is GPL-licensed. Users should evaluate `unidecode` if their project is compatible with GPL and requires potentially superior results.
Fix: Review project licensing requirements and compare `text-unidecode` with the `unidecode` library to choose the best fit for quality and license compatibility.

Install

pip install text-unidecode Install latest version

Imports

unidecode
```
from text_unidecode import unidecode
```

Quickstart

Demonstrates how to import the `unidecode` function and use it to transliterate a Unicode string into a basic ASCII equivalent.

from text_unidecode import unidecode

unicode_text = "Héllø Wörld! Како си? 北亰"
ascii_text = unidecode(unicode_text)

print(f"Original: {unicode_text}")
print(f"ASCII: {ascii_text}")

# Example with specific characters
unicode_german = "Äpfel, Öfen, Übermut"
ascii_german = unidecode(unicode_german)
print(f"German Original: {unicode_german}")
print(f"German ASCII: {ascii_german}")

view raw JSON →