ormsgpack

Common errors

• TypeError: Type is not msgpack serializable: YourCustomObject

  cause ormsgpack encountered an object type that it does not natively know how to serialize into MessagePack format. This often occurs with custom classes, certain datetime objects, or other non-primitive types.
  fix Provide a `default` callable to `ormsgpack.packb()` that explicitly handles the serialization of the unsupported type. This function should return a natively serializable type or raise a TypeError if the object cannot be handled.

• ModuleNotFoundError: No module named 'ormsgpack'

  cause The 'ormsgpack' package is not installed in the currently active Python environment, or the environment in which it was installed is not the one being used.
  fix Install the package using pip: `pip install ormsgpack`. If using virtual environments, ensure you activate the correct environment before installation.

• ormsgpack.MsgpackDecodeError: unpack requires a bytes object

  cause The input provided to `ormsgpack.unpackb()` is not a `bytes` object, which is required for deserialization.
  fix Ensure that the data passed to `ormsgpack.unpackb()` is a `bytes` object. For example, convert a string to bytes using `.encode('utf-8')` if it represents MessagePack data.

• error: subprocess-exited-with-error ... Cargo, the Rust package manager, is not installed or is not on PATH.

  cause ormsgpack is a Rust-based library. If a pre-built wheel (binary distribution) is not available for your specific system or Python version, pip will attempt to build it from source, which requires the Rust toolchain (including Cargo) to be installed on your system.
  fix Install Rust and Cargo development tools before attempting to install ormsgpack. The recommended way is via `rustup.rs` or your system's package manager (e.g., `apt install rust-all` on Debian/Ubuntu, `brew install rust` on macOS).

• TypeError: Integer exceeds 64-bit range

  cause Attempting to serialize a Python integer that is larger than the 64-bit integer range supported by MessagePack without enabling the appropriate option for handling large integers.
  fix Use the `ormsgpack.OPT_PASSTHROUGH_BIG_INT` option with `packb()` and provide a `default` function to handle these very large integers, typically by converting them to strings or a custom extension type.

Warnings

• breaking ormsgpack dropped support for Python 3.9 in version 1.12.0 and Python 3.8 in version 1.7.0. Users on older Python versions must use an earlier ormsgpack version.
  Fix: Upgrade Python to 3.10+ or pin ormsgpack to a compatible version (e.g., `<1.12.0` for Python 3.9, `<1.7.0` for Python 3.8).
• breaking `packb` started rejecting dictionary keys that are nested dataclasses or Pydantic models in version 1.8.0. This change was implemented to prevent potential issues with complex key serialization.
  Fix: Ensure dictionary keys are simple types (strings, numbers, basic Python objects) or use a custom `default` hook to convert complex keys to supported types before serialization.
• gotcha When providing a `default` callable to `packb` for custom type serialization, it must explicitly raise an exception (e.g., `TypeError`) if it cannot handle a given type. If it implicitly returns `None`, `ormsgpack` will serialize `None` as a valid value, potentially leading to unexpected data.
  Fix: Always explicitly raise an exception in the `default` callable for types it doesn't handle, for example: `raise TypeError(f"Object of type {type(obj).__name__} is not msgpack serializable")`.
• gotcha Using the `OPT_NON_STR_KEYS` option for dictionary keys can lead to duplicate keys if different non-string objects serialize to the same string representation. For example, `{'1970-01-01T00:00:00+00:00': True, datetime.datetime(1970, 1, 1, 0, 0, 0): False}` could result in a single key after serialization.
  Fix: Be aware of potential key collisions when using `OPT_NON_STR_KEYS`. Consider standardizing keys to strings or ensuring unique string representations for all keys.
• deprecated While not a breaking change in current behavior, prior to version 1.12.0, serializing strings containing surrogate code points (e.g., ill-formed UTF-8) required the `OPT_REPLACE_SURROGATES` option to prevent errors. Version 1.12.0 added this option and improved handling.
  Fix: For versions <1.12.0, explicitly use `option=ormsgpack.OPT_REPLACE_SURROGATES` if you expect to serialize strings with surrogate code points. Upgrading to 1.12.0 or newer is recommended for improved performance and fixes.
• breaking The script failed because the 'numpy' package was not found. This indicates that 'numpy' was not installed in the environment.
  Fix: Ensure 'numpy' is listed as a dependency in your `requirements.txt` or `setup.py` and is installed in the environment before running the script. For example, add `numpy` to your `pip install` command.
• breaking The 'numpy' module is not found, causing a 'ModuleNotFoundError'. This indicates that a required external dependency is missing from the environment.
  Fix: Install the 'numpy' package using pip (e.g., `pip install numpy`) or ensure that 'numpy' is included in the project's dependency management system (e.g., `requirements.txt`).

Install

• pip install ormsgpack Install stable version

Imports

• packb

  import ormsgpack
  packed_data = ormsgpack.packb(data)

  Main function for serializing Python objects to MessagePack bytes.
• unpackb

  import ormsgpack
  unpacked_data = ormsgpack.unpackb(packed_data)

  Main function for deserializing MessagePack bytes to Python objects.
• OPT_SERIALIZE_NUMPY

  import ormsgpack
  packed_data = ormsgpack.packb(data, option=ormsgpack.OPT_SERIALIZE_NUMPY)

  An example of an option constant used to modify serialization behavior.

Quickstart

This quickstart demonstrates how to serialize a Python dictionary containing a datetime object and a NumPy array into MessagePack format, and then deserialize it back. It shows the use of `packb` with an option for NumPy serialization and `unpackb` for deserialization.

import ormsgpack
import datetime
import numpy

# Example data including datetime and numpy array
event = {
    "type": "put",
    "time": datetime.datetime(1970, 1, 1, tzinfo=datetime.timezone.utc),
    "uid": 1,
    "data": numpy.array([1, 2, 3], dtype=numpy.int64),
}

# Serialize the data with an option to handle NumPy arrays
packed_data = ormsgpack.packb(event, option=ormsgpack.OPT_SERIALIZE_NUMPY)
print(f"Packed data: {packed_data}")

# Deserialize the data
unpacked_data = ormsgpack.unpackb(packed_data)
print(f"Unpacked data: {unpacked_data}")

# Verify types (Note: NumPy arrays deserialize to lists by default unless a custom hook is used)
assert isinstance(unpacked_data, dict)
assert isinstance(unpacked_data['time'], str) # datetime serializes to ISO 8601 string by default
assert isinstance(unpacked_data['data'], list)
assert unpacked_data['data'] == [1, 2, 3]