OpenTelemetry Milvus Instrumentation

Warnings

• gotcha It is critical to initialize the OpenTelemetry instrumentation for Milvus (e.g., `MilvusInstrumentor().instrument()`) *before* importing the `pymilvus` library. Failure to do so may result in gRPC calls not being properly traced.
  Fix: Ensure `from opentelemetry.instrumentation.milvus import MilvusInstrumentor` and `MilvusInstrumentor().instrument()` are executed early in your application's lifecycle, prior to any `import pymilvus` statements.
• gotcha This instrumentation primarily traces client-side interactions. For end-to-end observability, ensure your Milvus server instance is also configured to emit OpenTelemetry traces (Milvus 2.3.0+ supports this) and that an OpenTelemetry Collector or Jaeger is running to receive all telemetry data.
  Fix: Refer to Milvus documentation for server-side OpenTelemetry configuration. Configure `OTEL_EXPORTER_OTLP_ENDPOINT` and `OTEL_SERVICE_NAME` environment variables (or equivalent SDK configuration) to point to your trace collector.
• breaking OpenTelemetry semantic conventions, especially in rapidly evolving areas like Generative AI (which OpenLLMetry targets), can change between versions. While the instrumentation API might remain stable, the names or structure of emitted span attributes could change, potentially breaking dashboards or alerts relying on older conventions.
  Fix: Stay informed about OpenTelemetry semantic convention updates, particularly for `gen_ai.*` attributes. Review generated traces after upgrading to ensure compatibility with your observability backend and custom tooling.
• gotcha High CPU usage or unexpected behavior can occur if OpenTelemetry auto-instrumentation is enabled globally (e.g., via system-wide environment variables) and impacts unintended processes. This is a general risk with broad auto-instrumentation.
  Fix: Apply instrumentation selectively within your application code or use environment variables like `OTEL_DOTNET_AUTO_EXCLUDE_PROCESSES` (or language-specific equivalents) to prevent unwanted instrumentation.

Install

• pip install opentelemetry-instrumentation-milvus Install library
• pip install pymilvus opentelemetry-sdk opentelemetry-exporter-otlp opentelemetry-instrumentation-grpc Required dependencies for full tracing setup

Imports

• MilvusInstrumentor

  from opentelemetry.instrumentation.milvus import MilvusInstrumentor

Quickstart

This quickstart demonstrates how to set up OpenTelemetry tracing for a Python application interacting with Milvus. It initializes the OpenTelemetry SDK with an OTLP exporter, instruments the Milvus client using `MilvusInstrumentor`, and then performs a basic Milvus operation. Ensure an OpenTelemetry Collector or Jaeger is running and accessible at the specified `OTEL_EXPORTER_OTLP_ENDPOINT` to receive traces. The `MilvusInstrumentor().instrument()` call should happen before importing `pymilvus`.

import os
from opentelemetry import trace
from opentelemetry.sdk.resources import Resource
from opentelemetry.sdk.trace import TracerProvider
from opentelemetry.sdk.trace.export import BatchSpanProcessor
from opentelemetry.exporter.otlp.proto.grpc.trace_exporter import OTLPSpanExporter
from opentelemetry.instrumentation.grpc import GrpcInstrumentorClient

# IMPORTANT: Instrument Milvus BEFORE importing pymilvus
from opentelemetry.instrumentation.milvus import MilvusInstrumentor
MilvusInstrumentor().instrument()

# Configure OpenTelemetry (replace with your actual collector endpoint)
os.environ['OTEL_EXPORTER_OTLP_ENDPOINT'] = os.environ.get('OTEL_EXPORTER_OTLP_ENDPOINT', 'http://localhost:4317')
os.environ['OTEL_SERVICE_NAME'] = os.environ.get('OTEL_SERVICE_NAME', 'milvus-client-app')

resource = Resource.create({
    "service.name": os.environ['OTEL_SERVICE_NAME'],
    "application": "milvus-otel-test"
})

trace.set_tracer_provider(
    TracerProvider(resource=resource)
)

otlp_exporter = OTLPSpanExporter()
span_processor = BatchSpanProcessor(otlp_exporter)
trace.get_tracer_provider().add_span_processor(span_processor)

# Also instrument gRPC client explicitly for broader gRPC tracing
grpc_client_instrumentor = GrpcInstrumentorClient()
grpc_client_instrumentor.instrument()

# Now import pymilvus
from pymilvus import Collection, connections

# Connect to Milvus (assuming a local Milvus instance)
connections.connect("default", host="localhost", port="19530")

# Perform a simple Milvus operation to generate a trace
collection_name = "hello_milvus_otel"
if collection_name in connections.list_collections():
    Collection(collection_name).drop()

print(f"Milvus client instrumented. Check your OTLP collector/Jaeger at {os.environ['OTEL_EXPORTER_OTLP_ENDPOINT']}")