Loki Logger Handler

Common errors

• AttributeError: 'LoggerFormatter' object has no attribute 'message_in_json_format'

  cause This error occurred in version 1.1.1 when `message_in_json_format` was set to `False` due to a bug in attribute handling.
  fix Upgrade the `loki-logger-handler` library to version 1.1.2 or newer.

• Logs are not appearing in Grafana Loki, or I see HTTP 400 Bad Request errors in my application logs when sending to Loki.

  cause This usually indicates an incorrect Loki URL, network connectivity issues, or Loki rejecting the log entries due to validation errors (e.g., missing stream labels, line_too_long, or invalid JSON format).
  fix Double-check the `url` parameter in `LokiLoggerHandler` for correctness, including the `/loki/api/v1/push` endpoint. Verify network access to the Loki instance. Ensure that `labels` are provided and that log lines do not exceed Loki's maximum entry size (default 256KB).

• Duplicate log entries appear when running tests or reloading my application.

  cause This is a common Python `logging` module issue where handlers are added multiple times to the same logger instance without being removed.
  fix Ensure `logger.addHandler()` is called only once for each handler. In testing scenarios, call `logger.removeHandler(handler)` in teardown or use `logging.basicConfig` with `force=True` (Python 3.8+) or check `if not logger.handlers:` before adding handlers.

• KeyError: 'some_extra_field' when trying to access log.extra in a custom formatter or `label_keys` is not working.

  cause The `extra` dictionary was not passed to the logging call, or the specified key was not present in the `extra` dictionary provided.
  fix When logging, ensure you pass the `extra` dictionary with the keys you expect: `logger.info('My message', extra={'some_extra_field': 'value'})`. Verify that `label_keys` or `loki_metadata_keys` correctly list the fields you intend to extract.

Warnings

• breaking Older versions (pre-1.0.0, especially pre-0.1.3) had Python 2/3 compatibility issues related to file operations (`open` vs `io.open`) and dictionary merging syntax, which could lead to runtime errors on specific Python versions.
  Fix: Upgrade to version 1.0.0 or higher to ensure broader Python 2/3 compatibility. If sticking to older versions, test thoroughly across target Python environments.
• gotcha The `auth` attribute for basic authentication was added in version 1.1.1. If you are using an older version and require authenticated access to Loki, this feature will be unavailable.
  Fix: Upgrade to version 1.1.1 or later to utilize the `auth` parameter for basic authentication credentials.
• gotcha Enabling structured metadata (for Loki 3.0+) requires specific parameters (`enable_structured_loki_metadata`, `loki_metadata`, `loki_metadata_keys`) introduced in version 1.1.0. Using these features with older versions or an incompatible Loki server will not work as expected.
  Fix: Ensure you are using `loki-logger-handler` version 1.1.0 or newer and have a Loki 3.0+ instance if you intend to send structured metadata.
• gotcha An `AttributeError` could be raised in version 1.1.1 when `message_in_json_format` was set to `False` due to an incorrect attribute access. This was fixed in 1.1.2.
  Fix: Upgrade to version 1.1.2 or higher to resolve the `AttributeError` when `message_in_json_format` is `False`.
• gotcha Avoid using high-cardinality values (e.g., `user_id`, `request_id`, dynamic timestamps) directly as Loki labels. This can lead to performance degradation and increased storage costs in Loki itself.
  Fix: Move high-cardinality fields into the log message body or as structured metadata (if using Loki 3.0+ and the handler supports it) rather than labels. Use labels for broad categories like `application`, `environment`, `service`.

Install

• pip install loki-logger-handler Install stable version

Imports

• LokiLoggerHandler

  from loki_logger_handler.loki_logger_handler import LokiLoggerHandler

• LoggerFormatter

  from loki_logger_handler.loki_logger_handler import LoggerFormatter

  Standard Python logging formatter provided by the library.
• LoguruFormatter

  from loki_logger_handler.loki_logger_handler import LoguruFormatter

  Formatter specifically for Loguru users.

Quickstart

This quickstart demonstrates how to configure the `LokiLoggerHandler` with Python's standard `logging` module. It sets up a handler to send logs to a Loki instance, including static labels and dynamically extracting `extra` dictionary keys as labels. Ensure the `LOKI_URL` environment variable is set or replace the placeholder.

import logging
import os
from loki_logger_handler.loki_logger_handler import LokiLoggerHandler, LoggerFormatter

LOKI_URL = os.environ.get('LOKI_URL', 'http://localhost:3100/loki/api/v1/push')

logger = logging.getLogger(__name__)
logger.setLevel(logging.INFO)

# Configure Loki Handler
loki_handler = LokiLoggerHandler(
    url=LOKI_URL,
    labels={
        'application': 'my-python-app',
        'environment': 'dev'
    },
    # Optionally, specify keys from 'extra' dict to be used as labels
    label_keys={'user_id', 'request_id'},
    # For Loki 3.0+ and structured metadata
    # enable_structured_loki_metadata=True,
    # loki_metadata_keys={'metadata_field'}
)

# Optionally, use a specific formatter (LoggerFormatter is default for standard logging)
loki_handler.setFormatter(LoggerFormatter())

logger.addHandler(loki_handler)

# Example logging
logger.info('Application started successfully.')
logger.warning('Potential issue detected.', extra={'user_id': '12345'})

try:
    1 / 0
except ZeroDivisionError:
    logger.error('A critical error occurred!', exc_info=True, extra={'request_id': 'abc-123'})