OpenTelemetry Ollama Instrumentation

Warnings

• breaking Recent versions (e.g., v0.53.0 onwards) have migrated to align with the OpenTelemetry GenAI Semantic Conventions (0.5.0). This change introduces new attribute names and structures for LLM-related spans, potentially breaking existing dashboards, alerts, or custom processors relying on older conventions.
  Fix: Review the OpenTelemetry GenAI Semantic Conventions documentation for updated attribute names. Update your trace processors, dashboards, and any custom logic to reflect the new conventions. Consider using environment variables like `OTEL_SEMCONV_STABILITY_OPT_IN` for a phased migration if supported by your OpenTelemetry SDK.
• gotcha By default, this instrumentation captures and logs prompts, completions, and embeddings to span attributes. This data may contain highly sensitive user information.
  Fix: To disable logging of sensitive content for privacy reasons or to reduce trace size, set the `TRACELOOP_TRACE_CONTENT` environment variable to `false` (e.g., `export TRACELOOP_TRACE_CONTENT=false`).
• gotcha This library instruments the `ollama` Python client. For the instrumentation to be effective, the `ollama` client library must be installed separately, and an Ollama server instance must be running and accessible with the required models pulled.
  Fix: Ensure you have installed `ollama` via `pip install ollama`. Start your Ollama server (e.g., `ollama serve`) and pull any necessary models (e.g., `ollama pull llama2`) before running your instrumented application.

Install

• pip install opentelemetry-instrumentation-ollama ollama opentelemetry-sdk Core Installation
• pip install opentelemetry-distro Recommended for Auto-Instrumentation

Imports

• OllamaInstrumentor

  from opentelemetry.instrumentation.ollama import OllamaInstrumentor

Quickstart

This quickstart demonstrates how to set up a basic OpenTelemetry `TracerProvider` that exports traces to the console, and then instruments the `ollama` client to automatically capture traces for chat and generate operations. Ensure you have the `ollama` client library installed and an Ollama server running locally with the `llama2` model pulled.

import os
from opentelemetry import trace
from opentelemetry.sdk.resources import Resource
from opentelemetry.sdk.trace import TracerProvider
from opentelemetry.sdk.trace.export import ConsoleSpanExporter, BatchSpanProcessor
from opentelemetry.instrumentation.ollama import OllamaInstrumentor
import ollama

# 1. Set up OpenTelemetry Tracer Provider
# This simple setup exports traces to the console.
resource = Resource.create(attributes={"service.name": "ollama-app"})
provider = TracerProvider(resource=resource)
processor = BatchSpanProcessor(ConsoleSpanExporter())
provider.add_span_processor(processor)
trace.set_tracer_provider(provider)

# 2. Instrument Ollama
OllamaInstrumentor().instrument()

# 3. Use Ollama client (ensure ollama server is running and model pulled)
try:
    print("\n--- Making an Ollama chat request ---")
    chat_response = ollama.chat(
        model='llama2',
        messages=[{'role': 'user', 'content': 'Why is the sky blue?'}]
    )
    print("Ollama Chat Response:")
    print(chat_response['message']['content'])
    print("\n--- Trace for chat request should be visible above ---")

    print("\n--- Making an Ollama generate request (streaming) ---")
    stream = ollama.generate(
        model='llama2',
        prompt='Tell me a short story about a space-faring cat.',
        stream=True
    )
    full_response = ""
    print("Streaming Ollama Response:")
    for chunk in stream:
        if 'response' in chunk:
            full_response += chunk['response']
            print(chunk['response'], end='', flush=True)
    print("\n\n--- Trace for streaming request should be visible above ---")

except Exception as e:
    print(f"\nError interacting with Ollama: {e}")
    print("Please ensure the Ollama server is running (e.g., `ollama serve`) ")
    print("and the 'llama2' model is pulled (e.g., `ollama pull llama2`).")