Apache Thrift Python Bindings

raw JSON →
0.22.0 verified Tue May 12 auth: no python install: verified quickstart: stale

Apache Thrift is a lightweight, language-independent RPC framework designed to enable seamless cross-language communication between services. It provides abstractions for data transport, serialization, and application-level processing. The Python bindings allow developers to build interoperable RPC clients and servers from a common Interface Definition Language (IDL). The current version is 0.22.0, with releases typically occurring a few times a year.

pip install thrift
error ModuleNotFoundError: No module named 'your_service_name.ttypes'
cause The Python interpreter cannot find the modules generated by the `thrift` compiler because the directory containing them (e.g., `gen-py`) is not in `sys.path`.
fix
Add the directory containing the generated Python modules to sys.path at runtime or ensure it's part of your PYTHONPATH environment variable before importing. Example: import sys; sys.path.append('./gen-py'); from your_service_name.ttypes import MyStruct
error sh: thrift: command not found
cause The Apache Thrift compiler executable is not installed on the system or is not present in the system's PATH environment variable.
fix
Install the Apache Thrift compiler using your system's package manager (e.g., sudo apt-get install thrift-compiler on Debian/Ubuntu, brew install thrift on macOS) or by building it from source.
error thrift.transport.TTransportException: Could not connect to host localhost:9090
cause The Thrift client failed to establish a connection with the Thrift server, typically because the server is not running, is listening on a different host/port, or a firewall is blocking the connection.
fix
Ensure the Thrift server is running and listening on the specified host and port, and verify network connectivity and firewall rules.
error thrift.protocol.TProtocolException: Bad version in readMessageBegin
cause The Thrift client and server are configured to use different protocol implementations, leading to a mismatch in how messages are serialized and deserialized.
fix
Ensure both the Thrift client and server are configured to use the exact same protocol factory (e.g., both use TBinaryProtocol.TBinaryProtocolFactory or both use TCompactProtocol.TCompactProtocolFactory).
breaking Apache Thrift 0.22.0 introduced significant breaking changes for some language bindings (e.g., .NET Standard namespace and class name changes like `TServerSocket` to `TServerSocketTransport`, `TSimpleServer` to `TSimpleAsyncServer`). While the core Python API hasn't seen as many direct breaking changes in naming within the `thrift` package itself for this version, applications might still need adjustments, especially if depending on specific compiler flag behaviors or generated code specifics that were updated. Always re-generate Python code with the latest compiler for consistency.
fix Re-generate Python code from `.thrift` files using the `thrift` compiler version 0.22.0 or newer. Review generated code and update client/server implementations to match any altered patterns or class names in the generated stubs or core library.
gotcha The `pip install thrift` package provides only the Python runtime libraries. It DOES NOT include the `thrift` compiler (a C++ executable) which is absolutely necessary to generate Python client/server stub code from your `.thrift` IDL files. This compiler must be installed separately (e.g., via a package manager like `apt`, `brew`, or by building from source).
fix Install the Apache Thrift compiler executable (`thrift`) separately from the Python library. Consult the official Apache Thrift documentation for instructions on installing the compiler for your operating system.
gotcha Do not confuse the official `thrift` library (which requires code generation) with `thriftpy` or `thriftpy2`. `thriftpy`/`thriftpy2` are alternative, pure Python implementations that can dynamically load `.thrift` files at runtime without a separate compiler step. Attempting to use `thriftpy`'s dynamic loading or API (`thriftpy.load()`) with the official `thrift` library will lead to `AttributeError` or `ImportError`.
fix Choose one implementation (`thrift` or `thriftpy`/`thriftpy2`) and stick to its API and workflow. If using the official `thrift`, always use the `thrift --gen py` command to generate code before importing.
gotcha When importing generated Thrift code, examples often use `sys.path.append('gen-py')` to make the generated modules discoverable. While functional for quickstarts, this is not a best practice for production environments as it bypasses standard Python packaging mechanisms and can lead to module resolution issues.
fix For production, consider installing the generated Python code as a proper Python package (e.g., using `setuptools` and defining the generated directory as a package) or ensuring it's part of your application's installable modules that are correctly added to `PYTHONPATH`.
python os / libc status wheel install import disk
3.10 alpine (musl) sdist - 0.02s 20.0M
3.10 alpine (musl) - - 0.02s 20.0M
3.10 slim (glibc) sdist 2.3s 0.01s 20M
3.10 slim (glibc) - - 0.01s 20M
3.11 alpine (musl) sdist - 0.05s 22.6M
3.11 alpine (musl) - - 0.05s 22.6M
3.11 slim (glibc) sdist 2.4s 0.05s 23M
3.11 slim (glibc) - - 0.04s 23M
3.12 alpine (musl) sdist - 0.03s 12.5M
3.12 alpine (musl) - - 0.04s 12.5M
3.12 slim (glibc) sdist 3.3s 0.04s 13M
3.12 slim (glibc) - - 0.04s 13M
3.13 alpine (musl) sdist - 0.03s 12.2M
3.13 alpine (musl) - - 0.04s 12.1M
3.13 slim (glibc) sdist 3.0s 0.04s 13M
3.13 slim (glibc) - - 0.04s 13M
3.9 alpine (musl) sdist - 0.02s 19.6M
3.9 alpine (musl) - - 0.02s 19.6M
3.9 slim (glibc) sdist 2.8s 0.02s 20M
3.9 slim (glibc) - - 0.02s 20M

To use Apache Thrift in Python, you first define your service in a `.thrift` IDL file. Then, you use the Apache Thrift compiler (a C++ executable, not `pip install thrift`) to generate Python client and server stubs from this IDL. The generated code lives in a `gen-py` directory. The example demonstrates a simple `Calculator` service with `add` and `ping` methods, showing basic server and client setup.

# 1. Define your service in 'calculator.thrift'
# namespace py tutorial
# service Calculator {
#     i32 add(1:i32 num1, 2:i32 num2),
#     void ping()
# }

# 2. Generate Python code using the Thrift compiler:
#    thrift --gen py calculator.thrift
# This creates a 'gen-py' directory with 'tutorial' module.

import sys
import os
import time

# Ensure 'gen-py' is on the path to import generated code
current_dir = os.path.dirname(os.path.abspath(__file__))
sys.path.insert(0, os.path.join(current_dir, 'gen-py'))

# Generated code imports
from tutorial import Calculator
from tutorial.ttypes import *

# Thrift core library imports
from thrift.transport import TSocket, TTransport
from thrift.protocol import TBinaryProtocol
from thrift.server import TServer

# --- Server Implementation ---
class CalculatorHandler:
    def __init__(self):
        self.log = {}

    def ping(self):
        print('Server: ping()')

    def add(self, num1, num2):
        print(f'Server: add({num1}, {num2})')
        return num1 + num2

def start_server():
    handler = CalculatorHandler()
    processor = Calculator.Processor(handler)
    transport = TSocket.TServerSocket(host='127.0.0.1', port=9090)
    tfactory = TTransport.TBufferedTransportFactory()
    pfactory = TBinaryProtocol.TBinaryProtocolFactory()

    server = TServer.TSimpleServer(processor, transport, tfactory, pfactory)
    print('Starting the server on port 9090...')
    # In a real application, you might run this in a separate process or thread
    # For this quickstart, we'll simulate a long-running server
    # server.serve() # This blocks
    return server # Return server for demonstration, typically would call serve()

# --- Client Implementation ---
def run_client():
    # Make socket
    transport = TSocket.TSocket('localhost', 9090)
    # Buffering is critical. Raw sockets are very slow
    transport = TTransport.TBufferedTransport(transport)
    # Wrap in a protocol
    protocol = TBinaryProtocol.TBinaryProtocol(transport)
    # Create a client to use the protocol encoder
    client = Calculator.Client(protocol)

    # Connect!
    try:
        transport.open()
        print('Client: Connected to server.')
        client.ping()
        print('Client: ping() sent.')

        sum_result = client.add(5, 7)
        print(f'Client: 5 + 7 = {sum_result}')

    except TException as tx:
        print(f'Client: Error: {tx.message}')
    finally:
        transport.close()
        print('Client: Connection closed.')

if __name__ == '__main__':
    # This part demonstrates server setup and a client call in sequence.
    # For actual usage, server and client would typically run in separate processes.
    server_instance = start_server()
    # In a real scenario, you would start the server in a background thread or process.
    # For simplicity, we'll run the client immediately assuming the server is ready.
    # You might need a small delay for the server to fully start in a real async scenario.
    print("Quickstart: Server started (not blocking), running client now...")
    run_client()
    print("Quickstart: Client finished. If server was blocking, it would still be running.")
    # If server_instance.serve() was called, you'd need a way to stop it.