OpenTelemetry Weaviate Instrumentation

0.58.0 · active · verified Fri Apr 10

The `opentelemetry-instrumentation-weaviate` library provides OpenTelemetry tracing capabilities for interactions with Weaviate vector databases. It automatically captures spans for Weaviate client operations, allowing observability into your LLM and RAG pipelines. It is part of the broader OpenLLMetry project, currently at version 0.58.0, and receives frequent updates, especially concerning OpenTelemetry GenAI semantic conventions.

Warnings

gotcha Frequent updates to OpenTelemetry GenAI semantic conventions (v0.5.0 and later) in `openllmetry` may alter span attribute names and structure. If your consuming system (e.g., an LLM observability platform) relies on specific attribute paths, monitor these changes carefully when upgrading.
Fix: Review the release notes for `openllmetry` and the `opentelemetry-instrumentation-weaviate` package for details on semantic convention changes. Update your data processing or alerting configurations as needed.
gotcha The instrumentation requires `weaviate-client >=3.18.0`. Using older versions of the Weaviate client library may result in incomplete or incorrect instrumentation, or compatibility errors.
Fix: Ensure `weaviate-client` is updated to version 3.18.0 or newer. You can specify this in your `requirements.txt` or `pyproject.toml`.
gotcha This instrumentation requires explicit invocation of `WeaviateInstrumentor().instrument()` after setting up a global OpenTelemetry `TracerProvider`. It is not automatically enabled by `opentelemetry-bootstrap` or similar generic auto-instrumentation tools without a custom plugin configuration.
Fix: Always include `WeaviateInstrumentor().instrument()` in your application startup code, following the setup of your OpenTelemetry `TracerProvider`.

Install

pip install opentelemetry-instrumentation-weaviate weaviate-client opentelemetry-sdk Install with Weaviate client and OTel SDK

Imports

WeaviateInstrumentor

from opentelemetry.instrumentation.weaviate import WeaviateInstrumentor

Quickstart

This quickstart demonstrates how to set up OpenTelemetry tracing with `opentelemetry-instrumentation-weaviate`. It initializes a console exporter, instruments the Weaviate client, and then attempts to connect to a Weaviate instance, showing where your traced operations would occur. Ensure you have a Weaviate instance running locally or a Weaviate Cloud Service (WCS) instance configured with your URL and API key.

import os
import weaviate
from opentelemetry import trace
from opentelemetry.sdk.resources import Resource
from opentelemetry.sdk.trace import TracerProvider
from opentelemetry.sdk.trace.export import ConsoleSpanExporter, SimpleSpanProcessor
from opentelemetry.instrumentation.weaviate import WeaviateInstrumentor

# 1. Setup OpenTelemetry TracerProvider and Exporter
resource = Resource.create({"service.name": "weaviate-app"})
provider = TracerProvider(resource=resource)
processor = SimpleSpanProcessor(ConsoleSpanExporter())
provider.add_span_processor(processor)
trace.set_tracer_provider(provider)

# 2. Instrument Weaviate
WeaviateInstrumentor().instrument()

# 3. Use Weaviate client (ensure a Weaviate instance is running or mocked)
# Replace with your actual Weaviate URL and API key if needed
weaviate_url = os.environ.get("WEAVIATE_URL", "http://localhost:8080")
weaviate_api_key = os.environ.get("WEAVIATE_API_KEY", "YOUR_API_KEY_HERE")

try:
    client = weaviate.Client(
        url=weaviate_url
        # For Weaviate Cloud Services (WCS), uncomment and provide API key:
        # auth_client_secret=weaviate.AuthApiKey(api_key=weaviate_api_key)
    )

    if client.is_connected():
        print("Successfully connected to Weaviate. Spans will be generated for operations.")
        # Example: Perform a simple query (uncomment and adapt to your schema)
        # collection = client.collections.get("YourCollectionName")
        # response = collection.query.fetch_objects(limit=1)
        # print(f"Query executed. Response: {response}")
    else:
        print(f"Could not connect to Weaviate at {weaviate_url}. "
              "Please ensure Weaviate is running and accessible.")
except Exception as e:
    print(f"Error connecting to Weaviate or during operation: {e}")
    print("Please ensure your Weaviate instance is running and accessible.")

# Optional: Flush spans to ensure they are exported before program exit
provider.force_flush()

view raw JSON →