OpenInference Instrumentation

Warnings

• gotcha OpenInference auto-instrumentation (e.g., for CrewAI, LiteLLM) may not inherit `OTEL_RESOURCE_ATTRIBUTES` (like `langfuse.environment`) unless the `TracerProvider` is explicitly configured with these attributes *before* importing and initializing the instrumentors. If the instrumentor is imported first, it might create a default `TracerProvider`, causing traces to default to a 'default' environment.
  Fix: Ensure `opentelemetry.sdk.trace.TracerProvider` is configured with desired `Resource` attributes and set as the global `TracerProvider` *before* importing or instantiating any OpenInference instrumentors. Use `opentelemetry.sdk.resources.Resource.create({'langfuse.environment': 'your_env'})`.
• gotcha The `openinference-instrumentation-openai` instrumentor might not fully respect the OpenTelemetry `suppress_instrumentation` context flag. Spans might still be created for OpenAI API calls even when `suppress_instrumentation=True` is active in the context.
  Fix: As of the report, no direct fix in the library, a workaround might involve more manual control over span creation or conditionally calling the OpenAI API outside the instrumented scope if suppression is critical.
• gotcha To correctly obtain token counts when streaming with OpenAI, `openai>=1.26` is required, and `stream_options={'include_usage': True}` must be explicitly passed to the `client.chat.completions.create` method. Without this, token counts for streaming responses may be missing.
  Fix: Upgrade OpenAI SDK to `openai>=1.26` and pass `stream_options={'include_usage': True}` when making streaming chat completion calls.
• gotcha The base `openinference-instrumentation` package provides core utilities like context managers (`using_session`, `using_metadata`). However, for auto-instrumentation of specific LLM frameworks or SDKs (e.g., OpenAI, LangChain, LlamaIndex), you must install and import the corresponding `openinference-instrumentation-<framework>` sub-package. Installing only the base package will not provide framework-specific auto-instrumentation.
  Fix: Always install the specific instrumentation package for the framework you are using, e.g., `pip install openinference-instrumentation-openai`.
• gotcha Older versions of `openinference-instrumentation-openai-agents` might not log the tools configured on an agent as part of the agent's input. Instead, tools are only logged if they appear in a response (i.e., when a tool is actually called). This can lead to an incomplete view of agent capabilities in the trace UI.
  Fix: Check for updates to `openinference-instrumentation-openai-agents` or consider manual instrumentation to ensure all relevant tool information is captured in agent spans.

Install

• pip install openinference-instrumentation openinference-instrumentation-openai openai opentelemetry-sdk opentelemetry-exporter-otlp Install core utilities and OpenAI instrumentation
• pip install openinference-instrumentation # For core utilities like context managers Install core library only

Imports

• using_session

  from openinference.instrumentation.span_data import using_session

  Context manager for tracking user sessions across requests.
• using_user

  from openinference.instrumentation.span_data import using_user

  Context manager for associating traces with specific users.
• using_metadata

  from openinference.instrumentation.span_data import using_metadata

  Context manager for adding custom metadata to traces.
• OpenAIInstrumentor

  from openinference.instrumentation.openai import OpenAIInstrumentor

  Example of an instrumentation class from a framework-specific sub-package.

Quickstart

This quickstart demonstrates how to set up OpenInference instrumentation for OpenAI, configure an OpenTelemetry tracer, and use OpenInference context managers for adding session, user, and custom metadata. The traces are exported to an OTLP collector (e.g., Arize Phoenix).

import os
import openai
from openinference.instrumentation.openai import OpenAIInstrumentor
from openinference.instrumentation.span_data import using_session, using_user, using_metadata
from opentelemetry.exporter.otlp.proto.http.trace_exporter import OTLPSpanExporter
from opentelemetry.sdk import trace as trace_sdk
from opentelemetry.sdk.trace.export import SimpleSpanProcessor

# 1. Configure OpenTelemetry Tracer Provider
# Traces will be sent to an OTLP collector, e.g., Arize Phoenix (default at http://127.0.0.1:6006/v1/traces)
# Ensure your collector is running before executing this code.
endpoint = os.environ.get("OTEL_EXPORTER_OTLP_ENDPOINT", "http://127.0.0.1:6006/v1/traces")
tracer_provider = trace_sdk.TracerProvider()
tracer_provider.add_span_processor(SimpleSpanProcessor(OTLPSpanExporter(endpoint)))

# 2. Instrument your application with OpenAIInstrumentor
OpenAIInstrumentor().instrument(tracer_provider=tracer_provider)

# 3. Set OpenAI API Key (replace with your actual key or environment variable)
os.environ["OPENAI_API_KEY"] = os.environ.get("OPENAI_API_KEY", "sk-YOUR_OPENAI_API_KEY")

# 4. Use OpenInference context managers and make an LLM call
client = openai.OpenAI()

with using_session("user_session_abc"), \
     using_user("test_user_123"), \
     using_metadata(key="deployment_env", value="staging"): 

    print("Making OpenAI chat completion call...")
    response = client.chat.completions.create(
        model="gpt-3.5-turbo",
        messages=[
            {"role": "user", "content": "What is the capital of France?"}
        ]
    )
    print(f"Response: {response.choices[0].message.content}")

print("Traces should now be visible in your OpenTelemetry collector.")