{"library":"pykafka","title":"PyKafka","description":"PyKafka is a full-featured, pure-Python client for Apache Kafka, which optionally uses a C extension (librdkafka) for improved performance. It supports Kafka versions 0.8.2 and newer, providing Pythonic implementations of Kafka producers and consumers. The library aims to offer a similar abstraction level to the JVM Kafka client. PyKafka is actively maintained, with the current stable version being 2.8.0, and typically releases updates as needed.","language":"python","status":"active","last_verified":"Mon May 18","install":{"commands":["pip install pykafka"],"cli":null},"imports":["from pykafka import KafkaClient","from pykafka.common import OffsetType","from pykafka.producer import Producer","from pykafka.simpleconsumer import SimpleConsumer","from pykafka.balancedconsumer import BalancedConsumer"],"auth":{"required":false,"env_vars":[]},"quickstart":{"code":"import os\nimport time\nfrom pykafka import KafkaClient\nfrom pykafka.common import OffsetType\n\nKAFKA_HOSTS = os.environ.get('KAFKA_HOSTS', 'localhost:9092')\nTOPIC_NAME = os.environ.get('KAFKA_TOPIC', 'test-topic').encode('utf-8')\nCONSUMER_GROUP = os.environ.get('KAFKA_CONSUMER_GROUP', 'my-consumer-group').encode('utf-8')\n\n# 1. Connect to Kafka\nclient = KafkaClient(hosts=KAFKA_HOSTS)\ntopic = client.topics[TOPIC_NAME]\n\n# 2. Produce a message\nwith topic.get_producer() as producer:\n    print(f\"Producing message to {TOPIC_NAME.decode()}...\")\n    for i in range(5):\n        message_value = f\"test message {i}\".encode('utf-8')\n        producer.produce(message_value)\n        print(f\"Sent: {message_value.decode()}\")\n    print(\"Finished producing messages.\")\n\n# Give Kafka a moment to process\ntime.sleep(2)\n\n# 3. Consume messages\nprint(f\"\\nConsuming messages from {TOPIC_NAME.decode()} (group: {CONSUMER_GROUP.decode()})...\")\nconsumer = topic.get_simple_consumer(\n    consumer_group=CONSUMER_GROUP,\n    auto_commit_enable=True,\n    auto_offset_reset=OffsetType.EARLIEST, # Start from the beginning if no offset is committed\n    reset_offset_on_start=True # Force reset on start, useful for quickstart\n)\n\nfor i, message in enumerate(consumer):\n    if message is not None:\n        print(f\"Received: Offset={message.offset}, Value={message.value.decode()}\")\n    if i >= 4: # Consume the 5 messages we sent\n        break\n\nconsumer.stop()\nprint(\"Finished consuming messages.\")\n","lang":"python","description":"This quickstart demonstrates how to initialize a PyKafka client, produce a few messages to a topic, and then consume them using a `SimpleConsumer`. It's configured to connect to a local Kafka instance and resets the consumer offset to the earliest available for demonstration purposes. Ensure `KAFKA_HOSTS`, `KAFKA_TOPIC`, and `KAFKA_CONSUMER_GROUP` environment variables are set or default to `localhost:9092` and `test-topic`, `my-consumer-group` respectively.","tag":null,"tag_description":null,"last_tested":null,"results":[]},"compatibility":{"tag":null,"tag_description":null,"last_tested":"2026-05-18","installed_version":"2.8.0","pypi_latest":"2.8.0","is_stale":false,"summary":{"python_range":"3.10–3.9","success_rate":100,"avg_install_s":2.9,"avg_import_s":0.43,"wheel_type":"sdist"},"results":[{"runtime":"python:3.10-alpine","python_version":"3.10","os_libc":"alpine (musl)","variant":"pykafka","exit_code":0,"wheel_type":"sdist","failure_reason":null,"import_side_effects":"noisy","install_time_s":null,"import_time_s":0.41,"mem_mb":10.2,"disk_size":"22.3M"},{"runtime":"python:3.10-slim","python_version":"3.10","os_libc":"slim (glibc)","variant":"pykafka","exit_code":0,"wheel_type":"sdist","failure_reason":null,"import_side_effects":"noisy","install_time_s":2.5,"import_time_s":0.27,"mem_mb":10.2,"disk_size":"23M"},{"runtime":"python:3.11-alpine","python_version":"3.11","os_libc":"alpine (musl)","variant":"pykafka","exit_code":0,"wheel_type":"sdist","failure_reason":null,"import_side_effects":"noisy","install_time_s":null,"import_time_s":0.61,"mem_mb":11.6,"disk_size":"25.4M"},{"runtime":"python:3.11-slim","python_version":"3.11","os_libc":"slim (glibc)","variant":"pykafka","exit_code":0,"wheel_type":"sdist","failure_reason":null,"import_side_effects":"noisy","install_time_s":2.6,"import_time_s":0.55,"mem_mb":11.6,"disk_size":"26M"},{"runtime":"python:3.12-alpine","python_version":"3.12","os_libc":"alpine (musl)","variant":"pykafka","exit_code":0,"wheel_type":"sdist","failure_reason":null,"import_side_effects":"broken","install_time_s":null,"import_time_s":null,"mem_mb":null,"disk_size":"15.0M"},{"runtime":"python:3.12-slim","python_version":"3.12","os_libc":"slim (glibc)","variant":"pykafka","exit_code":0,"wheel_type":"sdist","failure_reason":null,"import_side_effects":"broken","install_time_s":3.5,"import_time_s":null,"mem_mb":null,"disk_size":"16M"},{"runtime":"python:3.13-alpine","python_version":"3.13","os_libc":"alpine (musl)","variant":"pykafka","exit_code":0,"wheel_type":"sdist","failure_reason":null,"import_side_effects":"broken","install_time_s":null,"import_time_s":null,"mem_mb":null,"disk_size":"14.7M"},{"runtime":"python:3.13-slim","python_version":"3.13","os_libc":"slim (glibc)","variant":"pykafka","exit_code":0,"wheel_type":"sdist","failure_reason":null,"import_side_effects":"broken","install_time_s":3.3,"import_time_s":null,"mem_mb":null,"disk_size":"15M"},{"runtime":"python:3.9-alpine","python_version":"3.9","os_libc":"alpine (musl)","variant":"pykafka","exit_code":0,"wheel_type":"sdist","failure_reason":null,"import_side_effects":"noisy","install_time_s":null,"import_time_s":0.38,"mem_mb":10,"disk_size":"21.9M"},{"runtime":"python:3.9-slim","python_version":"3.9","os_libc":"slim (glibc)","variant":"pykafka","exit_code":0,"wheel_type":"sdist","failure_reason":null,"import_side_effects":"noisy","install_time_s":2.8,"import_time_s":0.37,"mem_mb":10,"disk_size":"22M"}]}}