Unidecode

Warnings

• breaking The output of `unidecode()` is not guaranteed to be stable across different versions of the library. Improvements to transliteration tables can cause the ASCII approximation for certain Unicode characters to change in new releases.
  Fix: If using `unidecode()` to generate persistent identifiers like URL slugs, either lock your `unidecode` dependency to a specific version or generate the slug once and store it in your database, rather than re-generating on the fly.
• gotcha Unidecode performs a context-free, character-by-character mapping and is not language-specific. This means transliterations may not align with linguistic rules or cultural expectations for all languages (e.g., German umlauts are 'a', 'o', 'u' instead of 'ae', 'oe', 'ue'; East Asian languages may have simplified mappings).
  Fix: For language-specific or more sophisticated transliterations (especially for Japanese, Chinese, Korean), consider using libraries designed for those specific languages or implement pre-processing rules before using `unidecode()`.
• gotcha Unidecode requires a Python build with 'wide' Unicode characters (UCS-4 build) to correctly handle characters outside the Basic Multilingual Plane (BMP). 'Narrow' Python builds using surrogate pair encoding are not supported, which can lead to incorrect transliterations for mathematical symbols, emojis, etc.
  Fix: Ensure your Python environment is built with 'wide' Unicode support (typically `sys.maxunicode > 0xffff`). This is usually the default for Python 3.7+ builds but can vary by system configuration.
• gotcha The `unidecode` function expects a Unicode string (Python 3 `str`) as input. Passing byte data (e.g., from reading a file in binary mode) will result in a `TypeError` or incorrect output.
  Fix: Always ensure your input is a properly decoded Unicode string before passing it to `unidecode()`. If reading from a file, open it in text mode with the correct encoding (e.g., `open('file.txt', 'r', encoding='utf-8')`).
• gotcha The output of `unidecode` is a 'lossy' approximation. Since some characters map to `''` (empty string) or generic characters (like `?`), and due to its non-linguistic approach, the transliterated output should not be directly exposed to users without careful consideration, as it may be perceived as offensive or simply incorrect.
  Fix: Use `unidecode` primarily for internal system identifiers, search indexing, or compatibility with ASCII-only systems, not as a user-facing display mechanism. Always consider the context and potential user perception of the transliterated text.

Install

• pip install unidecode Install latest version

Imports

• unidecode

  from unidecode import unidecode

Quickstart

Demonstrates how to import and use the `unidecode` function for basic transliteration and a common use case like generating URL slugs.

from unidecode import unidecode

# Basic transliteration
text_unicode = 'Łódź, 北京, Español'
text_ascii = unidecode(text_unicode)
print(f"Original: {text_unicode}")
print(f"Transliterated: {text_ascii}")

# Example for URL slug generation (common use case)
article_title = 'The "Café" where you can find "Piñatas"!'
slug = unidecode(article_title).replace(' ', '-').lower()
print(f"\nURL Slug: {slug}")