Google Protocol Buffers Tools

0.47.0 · active · verified Fri Apr 17

`pbtools` is a Python library for working with Google Protocol Buffers. It provides tools to read, write, encode, and decode Protocol Buffers messages (both binary and text) directly from `.proto` definitions, *without* requiring generated Python code. This simplifies field access and enables decoding of unknown messages. The current version is 0.47.0, and it maintains an active release cadence with incremental updates.

Common errors

```
ModuleNotFoundError: No module named 'pbtools'
```
cause The `pbtools` package is not installed in the current Python environment, or the environment is not activated.

fix Run `pip install pbtools` to install the library. If using a virtual environment, ensure it is activated.
```
pbtools.error.Error: Failed to find message 'MyMessage'.
```
cause The message type specified (e.g., `database.MyMessage`) does not exist in any of the `.proto` files provided to `pbtools.build_database()`, or there is a typo in the message name.

fix Verify that the message name exactly matches the definition in your `.proto` file and that all necessary `.proto` files are included in the list passed to `build_database()`.
```
TypeError: 'bytes' object cannot be interpreted as an integer
```
cause Attempting to assign a Python type that is incompatible with the declared Protobuf field type (e.g., passing a `bytes` object to an `int32` field), or trying to decode malformed binary data.

fix Ensure that the Python data types used when creating a message instance match the corresponding Protobuf field types (e.g., `int` for `int32`, `str` for `string`, `list` for `repeated`). Verify the integrity of binary data before decoding.
```
TypeError: argument of type 'Database' is not iterable
```
cause The `Database` object returned by `build_database()` is a container for message *types*, not a collection of message *instances*. It cannot be directly iterated over.

fix Access specific message types as attributes of the `Database` object (e.g., `database.MyMessage`) to create or decode message instances.

Warnings

gotcha `pbtools` explicitly avoids generating Python code, which is a common workflow for `protoc`. Users accustomed to `protoc`-generated classes might find this approach different. `pbtools` parses `.proto` files at runtime.
Fix: Understand that `pbtools` dynamically works with message definitions by parsing `.proto` files at runtime, offering a different paradigm than compiled protobuf code. It's designed for direct data manipulation.
gotcha Although `pbtools` does not generate code, it still relies on the core `protobuf` library for its underlying mechanisms. Ensure the `protobuf` package is installed and compatible with your `pbtools` version (e.g., `pbtools` 0.47.0 requires `protobuf>=4.0.0`).
Fix: Usually handled automatically by `pip install pbtools`. If issues arise, check `pip check` and ensure there are no conflicting `protobuf` installations or manual downgrades of `protobuf`.
gotcha When accessing fields on a decoded message, if a field is not present in the binary data, `pbtools` will return its default value (e.g., `0` for integers, `''` for strings, empty list for repeated fields) rather than raising an `AttributeError`. This can mask missing data if not explicitly checked.
Fix: Be aware of Protobuf's default value behavior. For optional fields or fields that might genuinely be missing, consider checking if the field was 'set' if such a feature becomes available or design your `.proto` schema accordingly.

Install

pip install pbtools Install pbtools

Imports

build_database
```
from pbtools import build_database
```
Database
```
from pbtools import Database
```
Message
```
from pbtools import Message
```
Field
```
from pbtools import Field
```

Quickstart

This quickstart demonstrates how to define a Protocol Buffer message, build a database from its `.proto` definition, create a message instance, encode it to binary, and then decode it back using `pbtools`. It highlights the direct interaction with messages without code generation.

import pbtools
import os

# Example .proto file content
proto_file_content = """
syntax = "proto3";

message MyMessage {
    int32 id = 1;
    string name = 2;
    repeated int32 values = 3;
}
"""
# For simplicity, write to a temporary file
proto_path = "my_message.proto"
with open(proto_path, "w") as f:
    f.write(proto_file_content)

# Build a database from .proto files
database = pbtools.build_database([proto_path])

# Get the message type from the database
MyMessage = database.MyMessage

# Create a message instance
message = MyMessage(id=1, name="hello", values=[10, 20, 30])
print(f"Original message: {message}")

# Encode to binary
encoded = message.encode()
print(f"Encoded bytes: {encoded}")

# Decode from binary
decoded = MyMessage.decode(encoded)
print(f"Decoded message: {decoded}")

# Access fields
print(f"Decoded ID: {decoded.id}")
print(f"Decoded Name: {decoded.name}")
print(f"Decoded Values: {decoded.values}")

# Clean up the temporary file
os.remove(proto_path)

view raw JSON →