toons: High-Performance TOON Parser and Serializer

0.5.4 · active · verified Thu Apr 09

toons is a high-performance Python library for parsing and serializing TOON (Token Oriented Object Notation), implemented in Rust for speed. It offers an API that mirrors Python's built-in `json` module, making it easy to convert Python objects to TOON strings and vice-versa. TOON is a token-efficient data serialization format specifically designed to reduce token counts for Large Language Models. The current version is 0.5.4, with frequent minor releases addressing fixes and specification compliance.

Warnings

breaking Version 0.5.0 introduced compliance with TOON Specification v3.0, potentially causing breaking changes for users relying on data formatted under older TOON specifications (e.g., v2.0 from versions 0.3.x). Ensure your TOON data adheres to v3.0 when upgrading.
Fix: Review the TOON Specification v3.0 for any changes relevant to your data structures and update your TOON inputs/outputs accordingly.
gotcha Older versions (prior to 0.5.3) had issues with deserialization of special unicode characters and (prior to 0.5.2) with serialization of datetime objects. If you're handling complex strings or datetimes, ensure you are on a recent version.
Fix: Upgrade to `toons` version 0.5.3 or later to ensure correct handling of unicode and datetime types.
gotcha By default, `toons` enforces strict compliance with the TOON v3.0 specification. Malformed TOON strings (e.g., those with incorrect indentation, invalid escape sequences, or blank lines in arrays) will raise errors during parsing. For lenient parsing, strict mode can be explicitly disabled.
Fix: Ensure your TOON input strictly adheres to the v3.0 specification, or pass `strict=False` to `toons.loads()` for more tolerant parsing, e.g., `toons.loads(toon_str, options={'strict': False})`.

Install

pip install toons Install `toons`

Imports

loads
```
from toons import loads
```
dumps
```
from toons import dumps
```

Quickstart

This quickstart demonstrates how to serialize a Python dictionary and a list of dictionaries (tabular data) into TOON format, and then parse a TOON string back into a Python object using `toons.dumps` and `toons.loads`, mirroring the standard `json` library API.

import toons

# Sample Python data
data = {
    "name": "Alice",
    "age": 30,
    "tags": ["python", "rust", "toon"]
}

# Serialize Python data to TOON string
toon_string = toons.dumps(data)
print("--- TOON String ---")
print(toon_string)
# Expected output (simplified, actual format may vary slightly based on spec):
# name: Alice
# age: 30
# tags[3]: python,rust,toon

# Parse TOON string back to Python data
parsed_data = toons.loads(toon_string)
print("\n--- Parsed Python Data ---")
print(parsed_data)
# Expected output:
# {'name': 'Alice', 'age': 30, 'tags': ['python', 'rust', 'toon']}

# Example with tabular data (common for LLMs)
users = [
    {"id": 1, "name": "Bob", "active": True},
    {"id": 2, "name": "Charlie", "active": False}
]
tabular_toon = toons.dumps(users)
print("\n--- Tabular TOON String ---")
print(tabular_toon)
# Expected output (simplified):
# [2]{id,name,active}:
# 1,Bob,true
# 2,Charlie,false

view raw JSON →