FlatBuffers

Warnings

• gotcha FlatBuffers uses an 'inside-out' or 'depth-first' construction rule: any nested objects (strings, vectors, other tables, or structs within tables/vectors) must be serialized first, and their offsets obtained, before they can be added to their parent object. Attempting to add an uncreated nested object will result in an invalid buffer.
  Fix: Always serialize leaf-level data (strings, raw scalar vectors, structs) and then child tables before their parent tables. Pass the returned offsets (e.g., from `builder.CreateString()` or `ChildTableEnd()`) to the parent's `Add` methods.
• gotcha When deserializing string fields in Python, they are returned as `bytes` objects. You must explicitly call `.decode('utf-8')` (or another appropriate encoding) to convert them into standard Python `str` objects.
  Fix: After accessing a string field, apply `.decode('utf-8')`. Example: `monster.Name().decode('utf-8')`.
• gotcha FlatBuffers are designed for efficient reads and compact storage, not for easy in-place modification or creation of data in Python. If you need to change data in a FlatBuffer, you typically have to rebuild the entire buffer, which can be computationally expensive for frequent updates.
  Fix: For mutable data, consider using a standard Python data structure, converting it to FlatBuffers for storage/transmission, and then rebuilding it entirely if changes are needed. FlatBuffers shine when data is mostly static or read-heavy.
• breaking Schema evolution requires careful management to maintain forward and backward compatibility. Removing fields is prohibited; instead, mark them `deprecated`. New fields must be added at the end of a table definition unless explicit `id` attributes are used for all fields. Changing types or default values can also break compatibility.
  Fix: Consult the FlatBuffers 'Evolution' documentation. Use the `deprecated` attribute for unused fields and always append new fields. If `id` attributes are used, ensure they are consistent across schema versions.
• gotcha The `flatc` compiler (written in C++) is essential for generating the Python classes from `.fbs` schema files. It needs to be installed and accessible in your system's PATH. Version mismatches between the `flatc` compiler used to generate the code and the `flatbuffers` Python runtime library can lead to hard-to-debug issues.
  Fix: Ensure `flatc` is installed and in your PATH. Ideally, use a `flatc` compiler version that is consistent with or close to the `flatbuffers` Python library version you are using. Check the FlatBuffers GitHub releases for `flatc` binaries and the changelog for versioning notes.

Install

• pip install flatbuffers Install FlatBuffers Python library

Imports

• Builder

  import flatbuffers
  builder = flatbuffers.Builder(0)

  The core builder class for serialization.
• Generated Classes (e.g., Monster, MonsterAddName, GetRootAsMonster)

  from MyGame.Sample import Monster, MonsterAddName, GetRootAsMonster

  Schema-defined classes (e.g., `Monster`, `Vec3`, and their associated builder functions like `MonsterStart`, `MonsterAddName`, `MonsterEnd`) are generated by the `flatc` compiler into a Python module (e.g., `MyGame.Sample.py`). You import them from this generated module, not directly from the `flatbuffers` package.

Quickstart

This quickstart demonstrates how to define a FlatBuffers schema, compile it using `flatc` to generate Python classes, and then use these generated classes along with the `flatbuffers.Builder` to serialize a 'Monster' object into a byte buffer. It also shows how to deserialize the buffer and access its fields. Note that you must first save the schema to a file (e.g., `monster.fbs`) and run the `flatc --python monster.fbs` command in your terminal for the Python imports to work. The `flatc` compiler needs to be installed separately.

# 1. Define schema in 'monster.fbs' (run 'flatc --python monster.fbs' first)
# monster.fbs content:
# namespace MyGame.Sample;
# struct Vec3 {
#   x:float;
#   y:float;
#   z:float;
# }
# table Monster {
#   pos:Vec3;
#   hp:short = 100;
#   name:string;
#   inventory:[ubyte];
# }
# root_type Monster;

# Assuming 'flatc --python monster.fbs' has been run,
# which generates MyGame/Sample/Monster.py in the current directory.

import flatbuffers
from MyGame.Sample import Monster, Vec3

# --- Serialization ---
builder = flatbuffers.Builder(0) # Initial buffer size, will grow as needed

# 1. Create string for name (must be done before creating the Monster table)
name_offset = builder.CreateString('Orc')

# 2. Create Vector for inventory (must be done before creating the Monster table)
inventory_data = [0, 1, 2, 3, 4]
Monster.MonsterStartInventoryVector(builder, len(inventory_data))
for x in reversed(inventory_data): # FlatBuffers builds backwards
    builder.PrependByte(x)
inventory_offset = builder.EndVector()

# 3. Create the Vec3 struct (can be done inline or before Monster table)
Vec3.CreateVec3(builder, 1.0, 2.0, 3.0)
pos_offset = builder.EndStruct()

# 4. Start and populate the Monster table
Monster.MonsterStart(builder)
Monster.MonsterAddPos(builder, pos_offset)
Monster.MonsterAddHp(builder, 80) # Override default hp=100
Monster.MonsterAddName(builder, name_offset)
Monster.MonsterAddInventory(builder, inventory_offset)
monster_offset = Monster.MonsterEnd(builder)

# 5. Finish the buffer
builder.Finish(monster_offset)

buffer = builder.Output()
print(f"Serialized buffer (bytes): {buffer}")

# --- Deserialization ---
# Get a 'view' of the root monster from the buffer
monster = Monster.GetRootAsMonster(buffer, 0)

# Access fields
print(f"Monster Name: {monster.Name().decode('utf-8')}") # Strings need decoding
print(f"Monster HP: {monster.Hp()}")

pos = monster.Pos()
print(f"Monster Position: ({pos.X()}, {pos.Y()}, {pos.Z()})")

inventory = []
for i in range(monster.InventoryLength()):
    inventory.append(monster.Inventory(i))
print(f"Monster Inventory: {inventory}")