Google Cloud Logging exporter for OpenTelemetry

Warnings

• breaking As of v1.11.0a0, the `CloudLoggingExporter` defaults to outputting structured JSON to stdout via `structured_json_file=True` if `structured_json_file` is not explicitly set to `False` or if `exporter.export` is called directly without it. This changes the underlying mechanism of log export from direct RPC calls and assumes presence of a Cloud Logging agent to scrape stdout. If you relied on direct RPC behavior or are not in a compatible GCP environment, explicitly set `structured_json_file=False` or revert to an older version.
  Fix: If direct RPC export is needed, instantiate `CloudLoggingExporter` with `structured_json_file=False`. For GCP environments, structured JSON to stdout is generally preferred, so ensure your deployment has a suitable Cloud Logging agent configured.
• breaking There are known compatibility constraints with newer OpenTelemetry SDK versions. `opentelemetry-exporter-gcp-logging` v1.11.0a0 has an upper bound set on `opentelemetry-sdk` due to potential breaking changes in the SDK's logging API. Exceeding the specified SDK version range can lead to runtime errors.
  Fix: Always check the project's `pyproject.toml` or `setup.py` (e.g., `pyproject.toml` often specifies `opentelemetry-sdk < 1.40.0, >= 1.20.0`) for the exact compatible `opentelemetry-sdk` version range and ensure your environment adheres to it. Upgrade `opentelemetry-exporter-gcp-logging` if a newer SDK version is required.
• gotcha Ensure the Google Cloud service account used by your application or host environment has the `roles/logging.logWriter` IAM role. Missing this permission is a common cause of logs failing to appear in Cloud Logging.
  Fix: Grant the `Logging Log Writer` role (`roles/logging.logWriter`) to the service account associated with your application's environment (e.g., Compute Engine default service account, GKE Workload Identity service account, or a service account key file).
• gotcha While manual resource attribute creation is shown for `Resource.create()`, consider using `opentelemetry-resourcedetector-gcp` for automatic detection of environment-specific resource attributes when running in Google Cloud environments (e.g., GCE, GKE, Cloud Run). This enriches your telemetry data automatically with valuable metadata.
  Fix: Install `opentelemetry-resourcedetector-gcp` and integrate it into your `LoggerProvider` setup to automatically populate resource attributes relevant to your GCP environment.

Install

• pip install opentelemetry-exporter-gcp-logging Install latest version

Imports

• CloudLoggingExporter

  from opentelemetry.exporter.cloud_logging import CloudLoggingExporter

Quickstart

This quickstart demonstrates how to configure the `CloudLoggingExporter` to send Python logs to Google Cloud Logging. It includes examples for both the traditional RPC-based export (using `BatchLogRecordProcessor`) and the recommended structured JSON export to stdout (using `SimpleLogRecordProcessor`), which is more efficient when running in GCP environments with a Cloud Logging agent.

import logging
from opentelemetry.sdk._logs import LoggerProvider, LoggingHandler
from opentelemetry.sdk._logs.export import BatchLogRecordProcessor, SimpleLogRecordProcessor
from opentelemetry.sdk.resources import Resource
from opentelemetry._logs import set_logger_provider
from opentelemetry.exporter.cloud_logging import CloudLoggingExporter

# Configure OpenTelemetry LoggerProvider
resource = Resource.create(
    {
        "service.name": "my-python-app",
        "service.instance.id": "instance-1",
    }
)

logger_provider = LoggerProvider(resource=resource)
set_logger_provider(logger_provider)

# --- Example 1: Exporting via RPC (default for older versions, or if structured_json_file is not set) ---
exporter_rpc = CloudLoggingExporter(default_log_name='my_app_logs_rpc')
logger_provider.add_log_record_processor(BatchLogRecordProcessor(exporter_rpc))

# --- Example 2: Exporting as structured JSON to stdout (recommended for GCP environments >= v1.11.0) ---
# This is more efficient for environments with Cloud Logging agent that scrapes stdout
exporter_json = CloudLoggingExporter(default_log_name='my_app_logs_json', structured_json_file=True)
logger_provider.add_log_record_processor(SimpleLogRecordProcessor(exporter_json)) # Simple processor is often sufficient for stdout export

# Attach OTLP handler to the root logger
handler = LoggingHandler(level=logging.INFO, logger_provider=logger_provider)
logging.getLogger().addHandler(handler)

# Create a namespaced logger for your application
# It's recommended not to use the root logger with OTLP handler directly for app logs
app_logger = logging.getLogger("my_app_module")
app_logger.info("This is an info log with OTel context!")
app_logger.warning("A warning occurred: %s", "something went wrong")

# Logs will be exported asynchronously by BatchLogRecordProcessor
# For SimpleLogRecordProcessor, logs are sent immediately

# It's good practice to shut down the logger provider when the application exits
# This ensures all buffered logs are flushed.
# In a real application, you'd typically do this in a graceful shutdown hook.
logger_provider.shutdown()
print("Logs sent to Cloud Logging (or stdout for structured JSON exporter).")