TOON Format for Python

0.9.0-beta.1 · active · verified Thu Apr 16

TOON (Token-Oriented Object Notation) is a compact, human-readable data format designed for LLM prompts to reduce token usage by 30-60% compared to JSON. It achieves this by eliminating redundant punctuation and using a tabular format for uniform data structures. The Python implementation, currently in beta (v0.9.0-beta.1), provides encoding and decoding functionalities, aiming for full compliance with the TOON specification.

Common errors

```
ModuleNotFoundError: No module named 'toon_format'
```
cause You likely installed the package using `pip install toon-format`, but the Python import path uses underscores, not hyphens.

fix Ensure your import statement is `from toon_format import encode, decode` (using an underscore in `toon_format`) after installing `pip install toon-format`.
```
SyntaxError: Invalid TOON format: Array length mismatch. Expected 3 items, got 2.
```
cause The TOON string you are trying to decode has an array length declared (e.g., `items[3]`) that does not match the actual number of items provided in the data rows.

fix Correct the TOON string so that the declared array length `[N]` precisely matches the number of data rows, or set `strict=False` during decoding (e.g., `decode(toon_string, options={'strict': False})`) to allow lenient parsing.
```
ValueError: Invalid TOON format: Indentation error.
```
cause TOON strictly enforces 2-space indentation for nested objects, similar to YAML. Inconsistent or incorrect indentation will result in a parsing error.

fix Review your TOON string to ensure all nested objects use exactly 2 spaces per indentation level and are consistent throughout the document. Avoid using tabs for indentation.

Warnings

breaking The library is currently in beta (v0.9.x), and its API may change significantly before the 1.0.0 stable release.
Fix: Review the GitHub repository for the latest API changes before upgrading to new beta versions or the eventual 1.0.0 release. Pin specific beta versions if stability is critical.
gotcha TOON is primarily optimized for uniform arrays of objects (tabular data) and may not always yield significant token savings or be ideal for deeply nested or highly non-uniform JSON structures.
Fix: Evaluate TOON's token efficiency for your specific data structures. For deeply nested or irregular data, standard (compacted) JSON might still be more efficient or appropriate. Use TOON primarily for structured, tabular data fed to LLMs.
gotcha There are multiple Python implementations related to TOON (e.g., `py-toon-format`, `python-toon`, `toons`), leading to a fragmented ecosystem. Ensure you are using the official `toon-format/toon-python` implementation for the most up-to-date and spec-compliant version.
Fix: Always install `toon-format` via `pip install toon-format` or directly from the official GitHub repository (`pip install git+https://github.com/toon-format/toon-python.git`) to guarantee you're using the intended library. Verify import paths (`from toon_format import ...`) match the official documentation.
gotcha The `decode` function operates in 'strict' mode by default, which means it will raise errors for any syntax inconsistencies, array length mismatches, or delimiter issues in the TOON string.
Fix: If you need to parse potentially malformed or lenient TOON, use `decode(toon_string, options={'strict': False})`. However, be aware that disabling strict mode might lead to unexpected parsing results for invalid TOON.

Install

pip install toon-format Standard Installation
pip install toon-format[openai] With OpenAI Integration (for token counting)
pip install git+https://github.com/toon-format/toon-python.git Install from GitHub (for latest beta)

Imports

encode
wrong:
```
from toon_format.encoder import encode
```
correct:
```
from toon_format import encode
```
The `encode` function is directly available from the top-level `toon_format` package.
decode
wrong:
```
from toon_format.decoder import decode
```
correct:
```
from toon_format import decode
```
The `decode` function is directly available from the top-level `toon_format` package.
load, dump
wrong:
```
from toon_format.io import load, dump
```
correct:
```
from toon_format import load, dump
```
File I/O functions are exposed at the top level.

Quickstart

This quickstart demonstrates how to encode Python dictionaries and lists into TOON format and decode TOON strings back into Python objects using `toon_format.encode` and `toon_format.decode` functions.

from toon_format import encode, decode

# Encode a simple Python dictionary to TOON
data_object = {"name": "Alice", "age": 30}
toon_object = encode(data_object)
print(f"\nEncoded Object:\n{toon_object}")
# Expected output:
# name: Alice
# age: 30

# Encode a list of uniform dictionaries (tabular array)
data_list = [
    {"id": 1, "name": "Widget", "price": 9.99},
    {"id": 2, "name": "Gadget", "price": 19.99}
]
toon_list = encode(data_list)
print(f"\nEncoded List:\n{toon_list}")
# Expected output:
# [2]{id,name,price}:
# 1,Widget,9.99
# 2,Gadget,19.99

# Decode TOON back to Python objects
toon_string_to_decode = """
items[2]: apple,banana
"""
decoded_data = decode(toon_string_to_decode)
print(f"\nDecoded Data:\n{decoded_data}")
# Expected output: {'items': ['apple', 'banana']}

view raw JSON →