OpenTelemetry Together AI Instrumentation

Warnings

• breaking Frequent updates to OpenTelemetry GenAI semantic conventions (e.g., in versions 0.53.0 to 0.58.0) may introduce breaking changes to span attribute names and structures. Ensure your observability backend and custom dashboards are compatible with the latest semantic conventions.
  Fix: Refer to the OpenTelemetry documentation on semantic convention stability for migration guidance. You may need to update your instrumentation library, adjust custom attributes, or use the `OTEL_SEMCONV_STABILITY_OPT_IN` environment variable (e.g., `OTEL_SEMCONV_STABILITY_OPT_IN=http/dup` for HTTP changes) during migration to emit both old and new conventions temporarily.
• gotcha By default, this instrumentation logs sensitive data such as prompts, completions, and embeddings to span attributes. This is for visibility but can pose a privacy risk or increase trace size significantly.
  Fix: To disable the logging of sensitive content, set the environment variable `IFTRACER_TRACE_CONTENT` to `false`. Example: `export IFTRACER_TRACE_CONTENT=false`.
• gotcha This library is an OpenTelemetry *instrumentation* and requires a full OpenTelemetry SDK setup (including a `TracerProvider`, `SpanProcessor`, and `SpanExporter`) to actually collect and export traces. Without proper SDK configuration, the instrumentation will run but produce no visible telemetry.
  Fix: Ensure you have `opentelemetry-sdk` installed and correctly configured your `TracerProvider`, added at least one `SpanProcessor` (e.g., `BatchSpanProcessor`), and an `SpanExporter` (e.g., `OTLPSpanExporter`, `ConsoleSpanExporter`). Refer to OpenTelemetry's official Python SDK documentation for a complete setup guide.

Install

• pip install opentelemetry-instrumentation-together Install latest version

Imports

• TogetherAiInstrumentor

  from opentelemetry.instrumentation.together import TogetherAiInstrumentor

  The primary class to instrument Together AI calls.

Quickstart

This quickstart demonstrates how to set up OpenTelemetry to trace calls made to Together AI. It initializes a basic OpenTelemetry `TracerProvider` with a `ConsoleSpanExporter` to print traces directly to the console. The `TogetherAiInstrumentor` is then initialized and called to automatically instrument the Together AI client library. A mocked Together AI `Complete.create` call is included to make the example runnable without requiring an actual API key, though in a real scenario, you would provide your `TOGETHER_API_KEY` and make genuine API requests.

import os
import together
from opentelemetry import trace
from opentelemetry.sdk.trace import TracerProvider
from opentelemetry.sdk.trace.export import ConsoleSpanExporter, BatchSpanProcessor
from opentelemetry.instrumentation.together import TogetherAiInstrumentor

# Configure OpenTelemetry SDK
provider = TracerProvider()
processor = BatchSpanProcessor(ConsoleSpanExporter())
provider.add_span_processor(processor)
trace.set_tracer_provider(provider)

# Instrument Together AI
TogetherAiInstrumentor().instrument()

# Set your Together AI API key (replace with your actual key or environment variable)
# For demonstration, we'll use a placeholder and mock the API call.
together.api_key = os.environ.get('TOGETHER_API_KEY', 'sk-xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx')

# --- Mock Together AI call for a runnable example ---
# In a real application, you would make an actual call to Together AI
# For testing, you can use a library like 'unittest.mock' or 'pytest-mock'
# to prevent actual API calls.
class MockCompletion:
    def __init__(self, choices=None):
        self.choices = choices or [MockChoice()]

class MockChoice:
    def __init__(self, text="Mocked Together AI completion.", logprobs=None, finish_reason="stop"):
        self.text = text
        self.logprobs = logprobs
        self.finish_reason = finish_reason

# Temporarily patch together.Complete.create for the example
original_create = together.Complete.create
def mock_create(*args, **kwargs):
    print("Mocking together.Complete.create...")
    # Simulate a delay for realism in tracing
    import time
    time.sleep(0.1)
    return MockCompletion()
together.Complete.create = mock_create

try:
    print("Making a (mocked) Together AI call...")
    # Example Together AI call (this will be traced)
    response = together.Complete.create(
        prompt="Tell me a short story about a brave knight.",
        model="togethercomputer/llama-2-7b-chat",
        max_tokens=50
    )
    print(f"Response: {response.choices[0].text}")
finally:
    # Restore original method after example
    together.Complete.create = original_create
    # Ensure traces are flushed for console exporter
    provider.shutdown()

print("Check console for OpenTelemetry traces.")