OpenTelemetry Psycopg2 Instrumentation

raw JSON →
0.61b0 verified Tue May 12 auth: no python install: verified

This library provides automatic instrumentation for the `psycopg2` PostgreSQL adapter, enabling OpenTelemetry tracing for database operations within Python applications. It is part of the broader `opentelemetry-python-contrib` project, which sees frequent updates for bug fixes, new features, and compatibility with various libraries. The current version is 0.61b0 and it requires Python >=3.9.

pip install opentelemetry-instrumentation-psycopg2 psycopg2-binary
error ModuleNotFoundError: No module named 'opentelemetry.instrumentation.psycopg2'
cause The `opentelemetry-instrumentation-psycopg2` package or one of its core OpenTelemetry dependencies is not installed, or the application is running in a Python environment where it is not available.
fix
Ensure the package is installed in the correct environment: pip install opentelemetry-instrumentation-psycopg2. For a complete setup, pip install opentelemetry-distro opentelemetry-exporter-otlp opentelemetry-instrumentation-psycopg2 can help ensure all necessary and compatible OpenTelemetry components are present.
error DependencyConflict: requested: "psycopg2 >= X.Y.Z" but found: "None"
cause This typically occurs when `opentelemetry-instrumentation-psycopg2` is installed, but it cannot find the `psycopg2` package, often because `psycopg2-binary` is installed instead, or there's a version mismatch that fails the dependency check.
fix
Pass skip_dep_check=True to the instrumentor to bypass the dependency check: Psycopg2Instrumentor().instrument(skip_dep_check=True). Ensure you have either psycopg2 or psycopg2-binary installed in a compatible version manually.
error AttributeError: 'psycopg2.extensions.connection' object has no attribute '_is_instrumented_by_opentelemetry'
cause This specific `AttributeError` happens when attempting to manually instrument a `psycopg2` connection using `Psycopg2Instrumentor().instrument_connection()` with certain versions of `opentelemetry-instrumentation-psycopg2` that contained a bug related to checking the instrumentation status of the connection object.
fix
Upgrade opentelemetry-instrumentation-psycopg2 to a newer version (e.g., 0.46b0 or later as issues were fixed in later versions) to get the patch for this bug. Alternatively, use Psycopg2Instrumentor().instrument() to automatically instrument all connections instead of individual ones.
error RecursionError: maximum recursion depth exceeded (when using psycopg2.pool.ThreadedConnectionPool)
cause When `opentelemetry-instrumentation-psycopg2` is used with `psycopg2.pool.ThreadedConnectionPool` on Python versions older than 3.9, a recursive tracing issue can occur, leading to a `RecursionError`.
fix
Upgrade the Python version to 3.9 or newer. This issue is resolved in Python 3.9+ due to changes in how ThreadedConnectionPool interacts with instrumentation.
breaking Including SQLCommenter in the `db.statement` span attribute became opt-in from OpenTelemetry Python Contrib version 1.29.0/0.50b0.
fix To include SQLCommenter in the `db.statement` attribute, explicitly set `enable_attribute_commenter=True` when calling `Psycopg2Instrumentor().instrument()`.
gotcha Enabling SQLCommenter (`enable_commenter=True`) can lead to high cardinality in database span attributes if not managed, potentially increasing monitoring costs and reducing performance in some backends.
fix Evaluate the need for SQLCommenter carefully. If enabled, consider using `commenter_options` to opt out of specific key-value pairs (`Psycopg2Instrumentor().instrument(enable_commenter=True, commenter_options={'db_driver': False})`) to control cardinality.
gotcha Recursive tracing issues can occur when using `psycopg2.pool.ThreadedConnectionPool` with Python versions earlier than 3.9.
fix Upgrade your Python environment to 3.9 or higher to mitigate this issue. If upgrading is not possible, avoid using `ThreadedConnectionPool` with this instrumentation.
gotcha From OpenTelemetry Python Contrib version 1.32.0/0.53b0, instrumentors perform lazy-import dependency checks. If the underlying `psycopg2` (or `psycopg2-binary`) package is missing, `instrument()` might raise an `ImportError`.
fix Always ensure `psycopg2` or `psycopg2-binary` is installed alongside `opentelemetry-instrumentation-psycopg2`. In rare cases, `skip_dep_check=True` can be passed to `instrument()` if you manage dependencies manually or encounter unexpected issues with multiple `psycopg` distributions.
gotcha Using `opentelemetry-instrumentation-psycopg2` alongside other APM or tracing libraries that also instrument `psycopg2` can lead to conflicts, duplicate traces, or prevent database queries from being collected.
fix Avoid using multiple instrumentation libraries for the same underlying component. If you must, ensure to test thoroughly and potentially disable one of the instrumentations.
gotcha The instrumentation requires a running and accessible PostgreSQL database server to connect to. Failures such as 'Connection refused' indicate an environmental issue with the database setup, not a problem with the instrumentation library itself.
fix Ensure a PostgreSQL database server is running and accessible at the specified host and port. For example, use Docker to run a PostgreSQL instance as shown in the test output's suggestion: `docker run --rm --name some-postgres -e POSTGRES_DB=testdb -e POSTGRES_USER=user -e POSTGRES_PASSWORD=password -p 5432:5432 -d postgres`.
gotcha The instrumentation requires a running PostgreSQL database instance to connect to. A 'Connection refused' error indicates that the database server is not accessible at the specified host and port.
fix Ensure a PostgreSQL database is running and accessible from the environment where the application is executed. For example, use a Docker container: `docker run --rm --name some-postgres -e POSTGRES_DB=testdb -e POSTGRES_USER=user -e POSTGRES_PASSWORD=password -p 5432:5432 -d postgres`.
pip install opentelemetry-distro opentelemetry-exporter-otlp
python os / libc variant status wheel install import disk
3.10 alpine (musl) opentelemetry-distro wheel - - 112.3M
3.10 alpine (musl) opentelemetry-distro - - - -
3.10 alpine (musl) opentelemetry-instrumentation-psycopg2 wheel - 0.31s 31.2M
3.10 alpine (musl) opentelemetry-instrumentation-psycopg2 - - 0.34s 31.0M
3.10 slim (glibc) opentelemetry-distro wheel 5.6s - 180M
3.10 slim (glibc) opentelemetry-distro - - - -
3.10 slim (glibc) opentelemetry-instrumentation-psycopg2 wheel 2.8s 0.22s 35M
3.10 slim (glibc) opentelemetry-instrumentation-psycopg2 - - 0.23s 34M
3.11 alpine (musl) opentelemetry-distro wheel - - 120.9M
3.11 alpine (musl) opentelemetry-distro - - - -
3.11 alpine (musl) opentelemetry-instrumentation-psycopg2 wheel - 0.46s 33.6M
3.11 alpine (musl) opentelemetry-instrumentation-psycopg2 - - 0.56s 33.3M
3.11 slim (glibc) opentelemetry-distro wheel 5.0s - 189M
3.11 slim (glibc) opentelemetry-distro - - - -
3.11 slim (glibc) opentelemetry-instrumentation-psycopg2 wheel 2.8s 0.42s 37M
3.11 slim (glibc) opentelemetry-instrumentation-psycopg2 - - 0.43s 36M
3.12 alpine (musl) opentelemetry-distro wheel - - 111.1M
3.12 alpine (musl) opentelemetry-distro - - - -
3.12 alpine (musl) opentelemetry-instrumentation-psycopg2 wheel - 0.65s 25.3M
3.12 alpine (musl) opentelemetry-instrumentation-psycopg2 - - 0.71s 25.1M
3.12 slim (glibc) opentelemetry-distro wheel 4.1s - 179M
3.12 slim (glibc) opentelemetry-distro - - - -
3.12 slim (glibc) opentelemetry-instrumentation-psycopg2 wheel 2.5s 0.63s 29M
3.12 slim (glibc) opentelemetry-instrumentation-psycopg2 - - 0.67s 28M
3.13 alpine (musl) opentelemetry-distro wheel - - 107.7M
3.13 alpine (musl) opentelemetry-distro - - - -
3.13 alpine (musl) opentelemetry-instrumentation-psycopg2 wheel - 0.29s 25.1M
3.13 alpine (musl) opentelemetry-instrumentation-psycopg2 - - 0.34s 24.7M
3.13 slim (glibc) opentelemetry-distro wheel 4.0s - 178M
3.13 slim (glibc) opentelemetry-distro - - - -
3.13 slim (glibc) opentelemetry-instrumentation-psycopg2 wheel 2.5s 0.30s 28M
3.13 slim (glibc) opentelemetry-instrumentation-psycopg2 - - 0.35s 28M
3.9 alpine (musl) opentelemetry-distro wheel - - 111.2M
3.9 alpine (musl) opentelemetry-distro - - - -
3.9 alpine (musl) opentelemetry-instrumentation-psycopg2 wheel - 0.26s 30.7M
3.9 alpine (musl) opentelemetry-instrumentation-psycopg2 - - 0.29s 30.5M
3.9 slim (glibc) opentelemetry-distro wheel 6.3s - 179M
3.9 slim (glibc) opentelemetry-distro - - - -
3.9 slim (glibc) opentelemetry-instrumentation-psycopg2 wheel 3.3s 0.25s 34M
3.9 slim (glibc) opentelemetry-instrumentation-psycopg2 - - 0.25s 34M

This quickstart demonstrates how to instrument `psycopg2` to automatically generate traces for database interactions. It configures a simple ConsoleSpanExporter to print traces to the console and performs basic PostgreSQL operations. Ensure you have a PostgreSQL instance running, for example, using Docker.

import psycopg2
from opentelemetry import trace
from opentelemetry.sdk.resources import Resource
from opentelemetry.sdk.trace import TracerProvider
from opentelemetry.sdk.trace.export import ConsoleSpanExporter, SimpleSpanProcessor
from opentelemetry.instrumentation.psycopg2 import Psycopg2Instrumentor
import os

# Configure OpenTelemetry SDK
resource = Resource.create({"service.name": "my-psycopg2-app"})
provider = TracerProvider(resource=resource)
processor = SimpleSpanProcessor(ConsoleSpanExporter())
provider.add_span_processor(processor)
trace.set_tracer_provider(provider)

# Instrument psycopg2
# Pass enable_commenter=True and enable_attribute_commenter=True for SQL Commenter support
# Psycopg2Instrumentor().instrument(enable_commenter=True, enable_attribute_commenter=True, skip_dep_check=True)
Psycopg2Instrumentor().instrument()

# Database connection details (use environment variables or sensible defaults for quickstart)
db_name = os.environ.get('POSTGRES_DB', 'testdb')
db_user = os.environ.get('POSTGRES_USER', 'user')
db_password = os.environ.get('POSTGRES_PASSWORD', 'password')
db_host = os.environ.get('POSTGRES_HOST', 'localhost')
db_port = os.environ.get('POSTGRES_PORT', '5432')

try:
    # Connect to PostgreSQL
    conn = psycopg2.connect(dbname=db_name, user=db_user, password=db_password, host=db_host, port=db_port)
    cursor = conn.cursor()

    # Execute some SQL queries
    with trace.get_tracer(__name__).start_as_current_span("db_operations"):
        cursor.execute("CREATE TABLE IF NOT EXISTS otel_test (id serial PRIMARY KEY, name VARCHAR(255))")
        cursor.execute("INSERT INTO otel_test (name) VALUES (%s)", ("OpenTelemetry",))
        cursor.execute("SELECT * FROM otel_test")
        result = cursor.fetchall()
        print(f"Fetched result: {result}")

    conn.commit()
    cursor.close()
    conn.close()
    print("Database operations completed and traces should be visible.")
except Exception as e:
    print(f"An error occurred: {e}")
    print("Please ensure a PostgreSQL database is running and accessible (e.g., via Docker):")
    print("docker run --rm --name some-postgres -e POSTGRES_DB=testdb -e POSTGRES_USER=user -e POSTGRES_PASSWORD=password -p 5432:5432 -d postgres")