Apache Thrift Python Bindings
Apache Thrift is a lightweight, language-independent RPC framework designed to enable seamless cross-language communication between services. It provides abstractions for data transport, serialization, and application-level processing. The Python bindings allow developers to build interoperable RPC clients and servers from a common Interface Definition Language (IDL). The current version is 0.22.0, with releases typically occurring a few times a year.
Warnings
- breaking Apache Thrift 0.22.0 introduced significant breaking changes for some language bindings (e.g., .NET Standard namespace and class name changes like `TServerSocket` to `TServerSocketTransport`, `TSimpleServer` to `TSimpleAsyncServer`). While the core Python API hasn't seen as many direct breaking changes in naming within the `thrift` package itself for this version, applications might still need adjustments, especially if depending on specific compiler flag behaviors or generated code specifics that were updated. Always re-generate Python code with the latest compiler for consistency.
- gotcha The `pip install thrift` package provides only the Python runtime libraries. It DOES NOT include the `thrift` compiler (a C++ executable) which is absolutely necessary to generate Python client/server stub code from your `.thrift` IDL files. This compiler must be installed separately (e.g., via a package manager like `apt`, `brew`, or by building from source).
- gotcha Do not confuse the official `thrift` library (which requires code generation) with `thriftpy` or `thriftpy2`. `thriftpy`/`thriftpy2` are alternative, pure Python implementations that can dynamically load `.thrift` files at runtime without a separate compiler step. Attempting to use `thriftpy`'s dynamic loading or API (`thriftpy.load()`) with the official `thrift` library will lead to `AttributeError` or `ImportError`.
- gotcha When importing generated Thrift code, examples often use `sys.path.append('gen-py')` to make the generated modules discoverable. While functional for quickstarts, this is not a best practice for production environments as it bypasses standard Python packaging mechanisms and can lead to module resolution issues.
Install
-
pip install thrift
Imports
- TSocket
from thrift.transport import TSocket
- TBufferedTransport
from thrift.transport import TBufferedTransport
- TBinaryProtocol
from thrift.protocol import TBinaryProtocol
- TSimpleServer
from thrift.server import TServer
- TException
from thrift import Thrift; Thrift.TException
Quickstart
# 1. Define your service in 'calculator.thrift'
# namespace py tutorial
# service Calculator {
# i32 add(1:i32 num1, 2:i32 num2),
# void ping()
# }
# 2. Generate Python code using the Thrift compiler:
# thrift --gen py calculator.thrift
# This creates a 'gen-py' directory with 'tutorial' module.
import sys
import os
import time
# Ensure 'gen-py' is on the path to import generated code
current_dir = os.path.dirname(os.path.abspath(__file__))
sys.path.insert(0, os.path.join(current_dir, 'gen-py'))
# Generated code imports
from tutorial import Calculator
from tutorial.ttypes import *
# Thrift core library imports
from thrift.transport import TSocket, TTransport
from thrift.protocol import TBinaryProtocol
from thrift.server import TServer
# --- Server Implementation ---
class CalculatorHandler:
def __init__(self):
self.log = {}
def ping(self):
print('Server: ping()')
def add(self, num1, num2):
print(f'Server: add({num1}, {num2})')
return num1 + num2
def start_server():
handler = CalculatorHandler()
processor = Calculator.Processor(handler)
transport = TSocket.TServerSocket(host='127.0.0.1', port=9090)
tfactory = TTransport.TBufferedTransportFactory()
pfactory = TBinaryProtocol.TBinaryProtocolFactory()
server = TServer.TSimpleServer(processor, transport, tfactory, pfactory)
print('Starting the server on port 9090...')
# In a real application, you might run this in a separate process or thread
# For this quickstart, we'll simulate a long-running server
# server.serve() # This blocks
return server # Return server for demonstration, typically would call serve()
# --- Client Implementation ---
def run_client():
# Make socket
transport = TSocket.TSocket('localhost', 9090)
# Buffering is critical. Raw sockets are very slow
transport = TTransport.TBufferedTransport(transport)
# Wrap in a protocol
protocol = TBinaryProtocol.TBinaryProtocol(transport)
# Create a client to use the protocol encoder
client = Calculator.Client(protocol)
# Connect!
try:
transport.open()
print('Client: Connected to server.')
client.ping()
print('Client: ping() sent.')
sum_result = client.add(5, 7)
print(f'Client: 5 + 7 = {sum_result}')
except TException as tx:
print(f'Client: Error: {tx.message}')
finally:
transport.close()
print('Client: Connection closed.')
if __name__ == '__main__':
# This part demonstrates server setup and a client call in sequence.
# For actual usage, server and client would typically run in separate processes.
server_instance = start_server()
# In a real scenario, you would start the server in a background thread or process.
# For simplicity, we'll run the client immediately assuming the server is ready.
# You might need a small delay for the server to fully start in a real async scenario.
print("Quickstart: Server started (not blocking), running client now...")
run_client()
print("Quickstart: Client finished. If server was blocking, it would still be running.")
# If server_instance.serve() was called, you'd need a way to stop it.