OpenTelemetry OpenAI Instrumentation

Warnings

• gotcha There are two distinct OpenTelemetry OpenAI instrumentation packages: `opentelemetry-instrumentation-openai` (this community package by Traceloop/OpenLLMetry) and `opentelemetry-instrumentation-openai-v2` (the official OpenTelemetry Contrib package). Ensure you install and use the intended library for your project as they have different origins and might have subtle differences in implementation or features.
  Fix: Verify which package you intend to use. For new projects, the `opentelemetry-instrumentation-openai-v2` is often recommended as the official OpenTelemetry implementation.
• gotcha By default, this instrumentation does not capture the full content of prompts and completions due to privacy concerns. Only metadata like token counts and model names are recorded.
  Fix: To enable capturing message content as log events or span attributes, set the environment variable `OTEL_INSTRUMENTATION_GENAI_CAPTURE_MESSAGE_CONTENT=true` before running your application. Be mindful of potential PII in your telemetry data if you enable this.
• breaking The OpenTelemetry GenAI Semantic Conventions are under active development. Recent versions of this library (`0.55.0` and above) implement `OpenTelemetry GenAI Semantic Conventions 0.5.0` or later, which may change attribute names or span structures. Existing instrumentations using older versions (e.g., v1.36.0 or prior) might not emit the latest conventions by default.
  Fix: To opt into the latest experimental GenAI conventions, set the environment variable `OTEL_SEMCONV_STABILITY_OPT_IN=gen_ai_latest_experimental`. Review the OpenTelemetry GenAI semantic conventions documentation for specific changes.
• gotcha Using this instrumentation with pre-forking servers (e.g., Gunicorn with multiple workers) can lead to issues with metric generation and inconsistent telemetry due to how OpenTelemetry SDK components handle background threads and locks after process forking.
  Fix: When using pre-forking servers, consider using programmatic auto-instrumentation within each worker process, or configure the server to use a single worker for telemetry-sensitive processes. Alternatively, explore alternative deployment strategies that avoid process forking after instrumentation setup.
• gotcha Some users have reported significant performance degradation (reduced throughput) when this instrumentation is enabled, particularly under load. While improvements have been made, it's essential to benchmark your application.
  Fix: Regularly update the library to the latest version, as performance optimizations are ongoing. Profile your application with and without instrumentation to understand the overhead. If performance remains an issue, consider selective instrumentation or alternative tracing strategies.

Install

• pip install opentelemetry-instrumentation-openai openai opentelemetry-sdk opentelemetry-exporter-otlp Install core and OpenAI dependencies

Imports

• OpenAIInstrumentor

  from opentelemetry.instrumentation.openai import OpenAIInstrumentor

Quickstart

This quickstart demonstrates how to set up OpenTelemetry with the `opentelemetry-instrumentation-openai` library. It initializes a basic `TracerProvider` with a `ConsoleSpanExporter` to print traces to the console, then instruments the OpenAI client. Any subsequent OpenAI API calls will automatically generate spans.

import os
from opentelemetry import trace
from opentelemetry.sdk.trace import TracerProvider
from opentelemetry.sdk.trace.export import ConsoleSpanExporter, SimpleSpanProcessor
from opentelemetry.instrumentation.openai import OpenAIInstrumentor
from openai import OpenAI

# Configure OpenTelemetry Tracer Provider
provider = TracerProvider()
processor = SimpleSpanProcessor(ConsoleSpanExporter())
provider.add_span_processor(processor)
trace.set_tracer_provider(provider)

# Instrument the OpenAI SDK
OpenAIInstrumentor().instrument()

# Set your OpenAI API key (replace with actual key or use env var)
os.environ['OPENAI_API_KEY'] = os.environ.get('OPENAI_API_KEY', 'sk-YOUR_OPENAI_API_KEY')

# Initialize OpenAI client and make a call
client = OpenAI()

try:
    print("Making an OpenAI chat completion call...")
    response = client.chat.completions.create(
        model="gpt-3.5-turbo",
        messages=[
            {"role": "system", "content": "You are a helpful assistant."},
            {"role": "user", "content": "Tell me a short story about a brave knight."}
        ]
    )
    print("OpenAI response received.")
    print(f"Story: {response.choices[0].message.content}")
except Exception as e:
    print(f"An error occurred: {e}")
    print("Ensure OPENAI_API_KEY is set and valid.")

# You should see OpenTelemetry traces printed to the console.