win-unicode-console

Common errors

• UnicodeEncodeError: 'charmap' codec can't encode character '\uXXXX' in position Y: character maps to <undefined>

  cause The default Windows console encoding (e.g., CP437 or CP1252) cannot represent the Unicode character you are trying to print. This is a common issue for Python versions prior to 3.6 on Windows.
  fix Install and call `win_unicode_console.enable()` at the beginning of your script. Additionally, ensure your console font supports the required Unicode characters.

• Unicode characters display as '?' or squares in the Windows console output.

  cause This typically indicates either that `win-unicode-console` is not active, or the selected console font does not support the specific Unicode characters being displayed.
  fix Make sure `win_unicode_console.enable()` is called. Then, change your console font to 'Lucida Console' or 'Consolas' through the console window's properties to ensure full Unicode glyph support.

• Scripts launched with `python -m run script.py` (win-unicode-console runner) fail or behave unexpectedly with custom readline hooks or Python 2 `raw_input`.

  cause Prior to version 0.4, `win-unicode-console` used a custom REPL. Changes in 0.4 and later versions refined how the runner works and deprecated certain direct manipulations of `raw_input`. Python 2's `raw_input` returns bytes, which needs careful handling when Unicode is expected.
  fix Review the `win-unicode-console` README for specific `runner` usage (e.g., `py -i -m run` for interactive mode). For Python 2, if `raw_input` issues persist with Unicode, you might need `win_unicode_console.enable(raw_input__return_unicode=False)` if your IPython version is older than 4.

Warnings

• breaking This package is not necessary for Python 3.6 and newer versions because PEP 528 introduced native UTF-8 support for the Windows console, resolving the underlying issues that `win-unicode-console` addresses. Using it with Python 3.6+ is generally harmless but redundant.
  Fix: Remove `win-unicode-console` from your dependencies for projects targeting Python 3.6 or later.
• gotcha Even with `win-unicode-console`, the Windows console must be configured to use a font that supports Unicode characters (e.g., 'Lucida Console' or 'Consolas') to display them correctly. Without a suitable font, characters may still appear as squares or question marks.
  Fix: Right-click the console window title bar, select 'Properties' or 'Defaults', navigate to the 'Font' tab, and choose a Unicode-capable font like 'Lucida Console' or 'Consolas'.
• breaking Version 0.4 changed the signature of `streams.enable` and removed `streams.enable_reader`, `streams.enable_writer`, and `streams.enable_error_writer`. Code using these older stream-specific functions will break.
  Fix: Migrate to the new `streams.enable` or `streams.enable_only` interface, passing stream objects as keyword arguments (e.g., `streams.enable_only(stdin=streams.stdin_text_transcoding)`). For generic use, `win_unicode_console.enable()` should suffice.
• gotcha Manually setting the console code page to `chcp 65001` (UTF-8) is often problematic on Windows. It can cause issues with `sys.stdin`, result in `IOError`s, or not fully resolve Unicode display due to limitations of the standard console APIs. `win-unicode-console` is designed to work around these limitations without requiring `chcp 65001`.
  Fix: Rely on `win_unicode_console.enable()` rather than `chcp 65001` for Unicode support in Python scripts. If you encounter issues, consider using `PYTHONIOENCODING=utf-8` as an environment variable, but be aware of its own potential caveats.

Install

• pip install win-unicode-console Install latest version

Imports

• enable

  import win_unicode_console
  win_unicode_console.enable()

  Call `enable()` at the very beginning of your script, or via the `python -m run` runner.

Quickstart

This quickstart demonstrates how to enable Unicode support and then print and (for Python 3) read Unicode text in the Windows console. It's crucial to call `win_unicode_console.enable()` early in your script. Ensure your console font (e.g., Lucida Console) supports the characters you wish to display.

import win_unicode_console

# Enable Unicode console support (should be called early)
win_unicode_console.enable()

# Now, print Unicode characters
print('こんにちは世界 (Hello world in Japanese)')
print('Français: bonjour, ça va? (French)')

# Try reading Unicode input (on Python 2, this might still need custom readline hook or specific configurations)
# For simple testing, focus on output initially.
# For Python 3, input() typically handles Unicode after enable().
# Example for Python 3:
try:
    user_input = input('Enter some Unicode text: ')
    print(f'You entered: {user_input}')
except Exception as e:
    print(f'Input error: {e}')