Multi-byte String Decoder

Common errors

• UnicodeDecodeError: 'utf-8' codec can't decode byte 0xXX in position Y: invalid start byte

  cause This error occurs when attempting to decode a byte string using an incorrect encoding (e.g., UTF-8) when the actual byte sequence does not conform to that encoding's rules. While `mbstrdecoder` is designed to handle this, users might still encounter it if the library cannot determine a suitable encoding or if they bypass the decoder's functionality.
  fix Ensure you are passing a `bytes` object to `MultiByteStrDecoder` and check its `unicode_str` attribute. If an error still occurs, `mbstrdecoder` may not be able to automatically detect the encoding, in which case you might need to manually specify a `codec` when instantiating `MultiByteStrDecoder` if you know the correct encoding, or use its `get_decodable_codec_list()` to inspect available options. For example: ```python from mbstrdecoder import MultiByteStrDecoder malformed_bytes = b'\xed\xa0\x80' decoder = MultiByteStrDecoder(malformed_bytes) if not decoder.is_decodable(): print(f"Could not decode: {decoder.unicode_str} using codec {decoder.codec}") # Attempt with a known codec if available # decoder_latin1 = MultiByteStrDecoder(malformed_bytes, codec='latin-1') # print(decoder_latin1.unicode_str) else: print(decoder.unicode_str) ```

• ModuleNotFoundError: No module named 'mbstrdecoder'

  cause This error indicates that the `mbstrdecoder` library has not been installed in the current Python environment or the Python interpreter cannot find it.
  fix Install the library using pip: ```bash pip install mbstrdecoder ```

• AttributeError: module 'mbstrdecoder' has no attribute 'decode'

  cause Developers might mistakenly try to call a top-level `decode` function directly from the `mbstrdecoder` module, similar to how Python's built-in `bytes` objects have a `decode()` method. However, `mbstrdecoder` requires instantiating the `MultiByteStrDecoder` class first.
  fix Instantiate `MultiByteStrDecoder` with the byte string, then access the decoded string via its `unicode_str` attribute: ```python from mbstrdecoder import MultiByteStrDecoder encoded_bytes = b'hello world' decoder = MultiByteStrDecoder(encoded_bytes) decoded_string = decoder.unicode_str print(decoded_string) ```

Warnings

• breaking Python 3.7 and 3.8 support was dropped in version 1.1.4. Users on these Python versions must upgrade their Python environment or pin mbstrdecoder to an older version (e.g., <1.1.4).
  Fix: Upgrade to Python 3.9 or newer, or specify `mbstrdecoder<1.1.4` in your project dependencies.
• gotcha Prior to version 1.1.4, there were reported issues where `UnicodeDecodeError` exceptions might not be sent (or propagated) correctly during decoding attempts, leading to silent failures or unexpected behavior in error handling logic.
  Fix: Upgrade to version 1.1.4 or newer to ensure correct `UnicodeDecodeError` exception handling and propagation.
• gotcha The `chardet` library is a mandatory dependency. While it was optional in very early versions, it became mandatory from v0.8.2 onwards and has specific version requirements (`chardet>=3.0.2,<6.0.0`). Ensure it's installed alongside `mbstrdecoder` to avoid import errors related to encoding detection.
  Fix: Ensure `chardet` is part of your project's `pip install` commands. `pip install mbstrdecoder` should handle this automatically for recent versions.

Install

• pip install mbstrdecoder Install latest version

Imports

• MbStrDecoder

  from mbstrdecoder import MbStrDecoder

Quickstart

Initialize `MbStrDecoder` with a byte string. You can optionally provide an initial `encoding` hint or let it auto-detect. The `unicode_str` attribute holds the decoded string, and `detected_encoding` shows the encoding used.

from mbstrdecoder import MbStrDecoder

# Example 1: Decode a byte string with known encoding
decoder1 = MbStrDecoder(b"hello\xc2\xa3world", encoding="utf-8")
print(f"Decoded (UTF-8 known): {decoder1.unicode_str}, Encoding: {decoder1.detected_encoding}")

# Example 2: Decode a byte string with unknown encoding (chardet will detect)
decoder2 = MbStrDecoder(b"\xa3123.45") # Assuming some non-UTF8 locale, chardet will try
print(f"Decoded (auto-detect): {decoder2.unicode_str}, Encoding: {decoder2.detected_encoding}")

# Example 3: Handling undecodable bytes gracefully (if any)
decoder3 = MbStrDecoder(b'\xed\xa0\x80some invalid bytes', errors='replace')
print(f"Decoded (replace errors): {decoder3.unicode_str}, Encoding: {decoder3.detected_encoding}")