{"id":1607,"library":"opentelemetry-instrumentation-celery","title":"OpenTelemetry Celery Instrumentation","description":"This library provides OpenTelemetry instrumentation for Celery, enabling comprehensive monitoring of Celery applications by automatically collecting telemetry data such as task execution times, worker performance, and error rates. It also captures distributed traces across your task pipeline. The current version is `0.61b0`, and packages within the `opentelemetry-python-contrib` repository typically follow a monthly release cadence for beta versions.","status":"active","version":"0.61b0","language":"en","source_language":"en","source_url":"https://github.com/open-telemetry/opentelemetry-python-contrib","tags":["opentelemetry","celery","instrumentation","tracing","distributed tracing","monitoring","asynchronous tasks"],"install":[{"cmd":"pip install opentelemetry-instrumentation-celery opentelemetry-sdk","lang":"bash","label":"Install package"}],"dependencies":[{"reason":"The library instruments Celery; Celery must be installed separately.","package":"celery","optional":false},{"reason":"Provides the core OpenTelemetry SDK components required for tracing and exporting telemetry data.","package":"opentelemetry-sdk","optional":false}],"imports":[{"symbol":"CeleryInstrumentor","correct":"from opentelemetry.instrumentation.celery import CeleryInstrumentor"}],"quickstart":{"code":"import os\nfrom opentelemetry import trace\nfrom opentelemetry.sdk.resources import Resource\nfrom opentelemetry.sdk.trace import TracerProvider\nfrom opentelemetry.sdk.trace.export import BatchSpanProcessor, ConsoleSpanExporter\nfrom opentelemetry.exporter.otlp.proto.grpc.trace_exporter import OTLPSpanExporter\nfrom opentelemetry.instrumentation.celery import CeleryInstrumentor\nfrom celery import Celery\nfrom celery.signals import worker_process_init\n\n# Configure OpenTelemetry Tracer Provider\n# For local development, ConsoleSpanExporter is useful. For production, OTLPSpanExporter.\nresource = Resource.create({\"service.name\": os.environ.get('OTEL_SERVICE_NAME', 'celery-app')})\nprovider = TracerProvider(resource=resource)\n\n# Choose an exporter. For OTLP, ensure an OTLP endpoint is available (e.g., OpenTelemetry Collector).\n# Exporter endpoint can be configured via environment variable OTEL_EXPORTER_OTLP_ENDPOINT\nif os.environ.get('OTEL_EXPORTER_OTLP_ENDPOINT'):\n    exporter = OTLPSpanExporter()\nelse:\n    exporter = ConsoleSpanExporter()\n\nprocessor = BatchSpanProcessor(exporter)\nprovider.add_span_processor(processor)\ntrace.set_tracer_provider(provider)\n\n# Initialize Celery app\napp = Celery(\n    'my_celery_app',\n    broker=os.environ.get('CELERY_BROKER_URL', 'redis://localhost:6379/0'),\n    backend=os.environ.get('CELERY_RESULT_BACKEND', 'redis://localhost:6379/0')\n)\n\n# Crucial: Initialize instrumentation AFTER Celery worker process is initialized\n@worker_process_init.connect(weak=False)\ndef init_celery_tracing(*args, **kwargs):\n    CeleryInstrumentor().instrument()\n    print(\"OpenTelemetry Celery instrumentation initialized.\")\n\n# Define a simple task\n@app.task\ndef add(x, y):\n    with trace.get_current_tracer().start_as_current_span(\"add.task.execution\") as span:\n        result = x + y\n        span.set_attribute(\"sum\", result)\n        print(f\"Task 'add' executed: {x} + {y} = {result}\")\n        return result\n\nif __name__ == '__main__':\n    # Example of how to send a task (typically done from a separate producer process)\n    print(\"Sending task...\")\n    task = add.delay(1, 2)\n    print(f\"Task ID: {task.id}\")\n    print(f\"Task result: {task.get(timeout=10)}\")\n\n    # To run the worker (from your terminal):\n    # OTEL_SERVICE_NAME=\"celery-worker\" OTEL_EXPORTER_OTLP_ENDPOINT=\"http://localhost:4317\" celery -A your_module_name worker -l info","lang":"python","description":"This example demonstrates how to set up OpenTelemetry tracing for a Celery application. It configures a `TracerProvider` with a `BatchSpanProcessor` and an `OTLPSpanExporter` (or `ConsoleSpanExporter` as a fallback). Crucially, the `CeleryInstrumentor` is initialized within the `worker_process_init` signal handler to ensure proper tracing context propagation across Celery's prefork worker model. Remember to set `OTEL_SERVICE_NAME` and `OTEL_EXPORTER_OTLP_ENDPOINT` environment variables for OTLP export."},"warnings":[{"fix":"Monitor official OpenTelemetry Python releases for stable versions and semantic convention updates. Review changelogs carefully before upgrading.","message":"OpenTelemetry Python contrib packages, including Celery instrumentation, are currently in beta. This means APIs and behavior may change in future versions, and they are generally not recommended for production environments without careful consideration.","severity":"gotcha","affected_versions":"All versions up to 0.61b0"},{"fix":"Always connect `CeleryInstrumentor().instrument()` to the `worker_process_init` signal, as shown in the quickstart example.","message":"The `CeleryInstrumentor` must be initialized within the `celery.signals.worker_process_init` signal handler when using Celery's prefork worker model. Failing to do so can lead to incorrect or missing traces, as child processes may not inherit the parent's OpenTelemetry state properly.","severity":"breaking","affected_versions":"All versions"},{"fix":"Explicitly configure and set a global `TracerProvider` and add a `SpanProcessor` with a chosen `Exporter` during application startup, preferably before Celery tasks are defined or run, and within `worker_process_init` for workers.","message":"Telemetry data (traces, metrics) will not be collected or visible without a properly configured OpenTelemetry SDK and an exporter (e.g., OTLP, Jaeger, Console). Ensure `TracerProvider`, `SpanProcessor`, and an `Exporter` are set up.","severity":"gotcha","affected_versions":"All versions"},{"fix":"Stay informed about OpenTelemetry semantic convention updates and consult the `opentelemetry-python-contrib` documentation for specific instrumentation status and migration guides.","message":"OpenTelemetry Python is adopting a semantic convention migration plan. While currently focused on HTTP-related instrumentations, this may eventually affect all types of instrumentations, including Celery, and could introduce breaking changes to attribute names or span structures.","severity":"gotcha","affected_versions":"Future stable versions (post-beta)"},{"fix":"Identify all external libraries used by your Celery application (broker, backend, databases, HTTP clients) and install their corresponding OpenTelemetry instrumentations where available.","message":"The `opentelemetry-instrumentation-celery` package only instruments Celery itself. Depending on your Celery broker (e.g., Redis, RabbitMQ) and result backend, you might need to install additional OpenTelemetry instrumentations (e.g., `opentelemetry-instrumentation-redis`, `opentelemetry-instrumentation-amqp`).","severity":"gotcha","affected_versions":"All versions"}],"env_vars":null,"last_verified":"2026-04-09T00:00:00.000Z","next_check":"2026-07-08T00:00:00.000Z"}