Python TOON (Token-Oriented Object Notation)

0.1.3 · deprecated · verified Thu Apr 16

python-toon is a Python library for encoding and decoding TOON (Token-Oriented Object Notation), a compact data format optimized for Large Language Models (LLMs). It provides bidirectional JSON-to-TOON conversion, aiming to reduce token costs by 30-60% compared to JSON. The current version is 0.1.3, featuring alignment with the official TOON specification (v1.2 for decoding) and command-line interface support. However, users should note that the original `xaviviro/python-toon` repository is officially deprecated in favor of `toon-format/toon-python` as the main community-driven implementation.

Common errors

```
ModuleNotFoundError: No module named 'toon'
```
cause The Python package `python-toon` is not installed, or the import statement `from toon import ...` is attempting to import from a non-existent module name.

fix Ensure the package is installed: `pip install python-toon`. If using the new official implementation, it might be `from toon_format import ...` after installing `toon-format/toon-python`.
```
SyntaxError: invalid syntax (during decode)
```
cause The TOON string provided to `decode()` has fundamental syntax errors, such as missing required elements, malformed structures, or incorrect character usage that violates the TOON specification.

fix Carefully inspect the TOON string for syntax errors. Refer to the TOON specification for correct format. If the error is with programmatically generated TOON, verify the encoding logic. Ensure no common Python syntax errors are accidentally introduced if the TOON string is hardcoded.
```
ValueError: Invalid TOON input: ...
```
cause The TOON string is syntactically valid but semantically incorrect or violates data integrity checks (e.g., array length mismatches, invalid delimiters) especially when strict mode is enabled during decoding.

fix Review the TOON string for consistency, particularly array lengths (`[N]`) matching actual item counts and correct delimiter usage. If lenient parsing is acceptable, check the `decode` options for a `strict` parameter and set it to `False`.

Warnings

breaking The `xaviviro/python-toon` library is officially deprecated. Users are advised to migrate to the `toon-format/toon-python` repository for future development and support, which may involve changes to package name and import paths.
Fix: Switch to `pip install git+https://github.com/toon-format/toon-python.git` (or the PyPI version when available) and update imports from `from toon import encode, decode` to `from toon_format import encode, decode` as seen in the new library's quickstart.
gotcha The new, recommended `toon-format/toon-python` library is currently in beta (v0.9.x), and its API may change before the 1.0.0 stable release. This means that users should expect potential breaking changes in minor versions.
Fix: Monitor the official GitHub repository for release notes and changes. Pin dependencies to exact versions to prevent unexpected breakage in production environments.
gotcha TOON, like YAML, is sensitive to indentation. Incorrect or inconsistent indentation (e.g., mixing tabs and spaces) can lead to parsing errors when manually writing or manipulating TOON strings.
Fix: Use a consistent indentation strategy (e.g., 2 or 4 spaces) and configure your editor to automatically convert tabs to spaces to avoid `IndentationError` during `decode` operations.
gotcha Decoding TOON in `strict` mode can lead to `ValueError` or `SyntaxError` if the input TOON string contains minor non-conformances, such as invalid escape sequences, missing colons, malformed headers, or array length/delimiter mismatches.
Fix: Ensure TOON input strictly adheres to the TOON specification. For more lenient parsing, check if the `decode` function offers a `strict=False` option (as seen in `toon-format/toon-python` documentation for `decode` options).

Install

pip install python-toon Install stable version (deprecated)
pip install toon-format/toon-python Install recommended replacement (beta)

Imports

encode
wrong:
```
import toon.encode
```
correct:
```
from toon import encode
```
The `encode` and `decode` functions are directly importable from the `toon` package. The package name on PyPI is `python-toon` but the import path is `toon`.
decode
```
from toon import decode
```
The `encode` and `decode` functions are directly importable from the `toon` package. The package name on PyPI is `python-toon` but the import path is `toon`.

Quickstart

This quickstart demonstrates how to use `encode` to convert Python dictionaries and lists into TOON format, showcasing both simple objects and tabular arrays. It also shows how to use `decode` to convert a TOON string back into a Python object.

from toon import encode, decode

# Example 1: Encode a simple object
data_object = {"name": "Alice", "age": 30}
toon_output_obj = encode(data_object)
print("Encoded object:\n" + toon_output_obj)
# Expected output for data_object:
# name: Alice
# age: 30

# Example 2: Encode a tabular array (list of uniform objects)
data_list = [
    {"id": 1, "name": "Alice", "active": True},
    {"id": 2, "name": "Bob", "active": False},
    {"id": 3, "name": "Charlie", "active": True}
]
toon_output_list = encode(data_list)
print("\nEncoded tabular list:\n" + toon_output_list)
# Expected output for data_list:
# [3,]{id,name,active}:
# 1,Alice,true
# 2,Bob,false
# 3,Charlie,true

# Example 3: Decode TOON back to Python
toon_string = """users[3]{id,name,role}:
1,Alice,admin
2,Bob,user
3,Charlie,user"""
decoded_data = decode(toon_string)
print("\nDecoded data:")
print(decoded_data)
# Expected output for decoded_data:
# {'users': [{'id': 1, 'name': 'Alice', 'role': 'admin'}, {'id': 2, 'name': 'Bob', 'role': 'user'}, {'id': 3, 'name': 'Charlie', 'role': 'user'}]}

view raw JSON →