OpenInference LlamaIndex Instrumentation

Common errors

• openai.error.AuthenticationError: No API key provided.

  cause The OpenAI API key is not correctly set or is not accessible in the environment where LlamaIndex is being initialized, or it's being imported before the key is set.
  fix Ensure `os.environ["OPENAI_API_KEY"] = "your_api_key"` is called *before* any `llama_index` or `openai` imports, or that the `OPENAI_API_KEY` environment variable is set in your shell/deployment environment.

• WARNING Unhandled event of type WorkflowStepOutputEvent

  cause The OpenInference instrumentation for LlamaIndex has encountered an event type (like `WorkflowStepOutputEvent` from LlamaIndex Workflows) that it doesn't currently explicitly handle, leading to a gap in tracing.
  fix Update `openinference-instrumentation-llama-index` to the latest version. The OpenInference team frequently adds support for new LlamaIndex features. Check the project's GitHub issues or releases for specific updates.

• ImportError: cannot import name 'LlamaIndexInstrumentor' from 'openinference.instrumentation.llama_index'

  cause The `openinference-instrumentation-llama-index` package is not installed, or there is a typo in the import statement.
  fix Verify the package is installed correctly using `pip show openinference-instrumentation-llama-index`. Correct the import statement to `from openinference.instrumentation.llama_index import LlamaIndexInstrumentor` if necessary.

Warnings

• breaking OpenInference LlamaIndex Instrumentation versions have specific compatibility requirements with LlamaIndex versions. Upgrading LlamaIndex without checking compatibility with the instrumentation can lead to runtime errors or incomplete tracing.
  Fix: Refer to the official OpenInference LlamaIndex documentation for the exact compatibility matrix. Ensure your `llama-index` version aligns with the `openinference-instrumentation-llama-index` version you are using.
• gotcha By default, OpenInference LlamaIndex instrumentation logs sensitive data such as prompts, completions, and embeddings to span attributes. This can be a privacy concern or lead to large trace payloads.
  Fix: To disable this logging, set the `TRACELOOP_TRACE_CONTENT` environment variable to `false`. Consult OpenInference documentation for more granular control over data masking.
• gotcha When running LlamaIndex instrumentation within Python threads (e.g., in a FastAPI application), traces might appear 'malformed' or incomplete if context propagation is not properly handled.
  Fix: Ensure OpenTelemetry context is properly propagated across threads. This often involves using a context manager or explicit context passing mechanisms provided by OpenTelemetry, or consulting the OpenInference documentation for best practices with async/threaded applications.
• gotcha The OpenInference LlamaIndex instrumentation may not handle all LlamaIndex event types, especially newer ones introduced in LlamaIndex Workflows, leading to 'Unhandled event of type' warnings and missing trace details.
  Fix: Keep `openinference-instrumentation-llama-index` updated to the latest version. If using custom event types or advanced LlamaIndex workflows, check for specific support in the instrumentation's changelog or consider contributing to add support.

Install

• pip install openinference-instrumentation-llama-index Install core package
• pip install "llama-index>=0.12.3" openinference-instrumentation-llama-index opentelemetry-sdk opentelemetry-exporter-otlp Install with compatible LlamaIndex and OpenTelemetry dependencies

Imports

• LlamaIndexInstrumentor

  from openinference.instrumentation.llama_index import LlamaIndexInstrumentor

  Main class to initialize LlamaIndex instrumentation.
• TracerProvider

  from opentelemetry.sdk.trace import TracerProvider

  Standard OpenTelemetry tracer provider setup.
• OTLPSpanExporter

  from opentelemetry.exporter.otlp.proto.http.trace_exporter import OTLPSpanExporter

  Standard OpenTelemetry exporter for OTLP over HTTP.
• SimpleSpanProcessor

  from opentelemetry.sdk.trace.export import SimpleSpanProcessor

  Standard OpenTelemetry span processor.
• Resource

  from opentelemetry.sdk.resources import Resource

  Used to add resource attributes like project name to traces.

Quickstart

This quickstart demonstrates how to instrument a basic LlamaIndex query engine with OpenInference. It sets up an OpenTelemetry `TracerProvider` and `OTLPSpanExporter` to send traces to a collector (e.g., Arize Phoenix). It then initializes the `LlamaIndexInstrumentor` and performs a simple query using LlamaIndex with OpenAI as the LLM, showing how operations like document loading, indexing, and querying are automatically traced.

import os
from openinference.instrumentation.llama_index import LlamaIndexInstrumentor
from openinference.semconv.resource import ResourceAttributes
from opentelemetry import trace as trace_api
from opentelemetry.exporter.otlp.proto.http.trace_exporter import OTLPSpanExporter
from opentelemetry.sdk.resources import Resource
from opentelemetry.sdk.trace import TracerProvider
from opentelemetry.sdk.trace.export import SimpleSpanProcessor

from llama_index.core import VectorStoreIndex, SimpleDirectoryReader
from llama_index.llms.openai import OpenAI
from llama_index.embeddings.openai import OpenAIEmbedding
from llama_index.core import Settings

# Set up OpenTelemetry for tracing
def setup_tracing():
    collector_endpoint = os.environ.get("COLLECTOR_ENDPOINT", "http://localhost:6006/v1/traces")
    resource = Resource(attributes={ResourceAttributes.PROJECT_NAME: "llama-index-demo"})
    tracer_provider = TracerProvider(resource=resource)
    span_exporter = OTLPSpanExporter(endpoint=collector_endpoint)
    span_processor = SimpleSpanProcessor(span_exporter=span_exporter)
    tracer_provider.add_span_processor(span_processor=span_processor)
    trace_api.set_tracer_provider(tracer_provider=tracer_provider)
    LlamaIndexInstrumentor().instrument()
    print("🔭 OpenInference LlamaIndex instrumentation enabled.")

# Main application logic
def main():
    setup_tracing()

    # Ensure OPENAI_API_KEY is set
    if not os.environ.get("OPENAI_API_KEY"):
        print("Please set the OPENAI_API_KEY environment variable.")
        return

    # Configure LLM and embeddings for LlamaIndex
    Settings.llm = OpenAI(model="gpt-3.5-turbo", temperature=0.1)
    Settings.embed_model = OpenAIEmbedding(model="text-embedding-3-small")

    # Create a dummy data directory and file for demonstration
    if not os.path.exists("data"):
        os.makedirs("data")
    with open("data/sample.txt", "w") as f:
        f.write("The quick brown fox jumps over the lazy dog. This is a sample document for LlamaIndex.")

    # Load documents and create an index
    documents = SimpleDirectoryReader('data').load_data()
    index = VectorStoreIndex.from_documents(documents)

    # Query the index
    query_engine = index.as_query_engine()
    response = query_engine.query("What did the fox do?")
    print(f"Response: {response}")

    print("\nVisit your OpenTelemetry collector (e.g., Phoenix at http://localhost:6006) to see traces.")

if __name__ == "__main__":
    main()