logfmter

Common errors

• Logs are not appearing in logfmt format / Not seeing any logfmter output.

  cause `Logfmter` is not correctly applied to a `logging.Handler`, or the handler is not attached to the logger. When using `dictConfig`, `disable_existing_loggers` might be `True` by default, disabling other loggers.
  fix Ensure a `Logfmter` instance is set as the formatter for at least one `StreamHandler` or `FileHandler`, and that handler is added to the relevant logger (or root logger) using `basicConfig` or `dictConfig`. If using `dictConfig`, explicitly set `"disable_existing_loggers": false` if you need existing loggers to continue logging.

• Logs show \n characters literally instead of creating newlines.

  cause Starting from `logfmter` v0.0.4, newline characters are explicitly escaped in all logged values to ensure single-line output. This is the intended behavior.
  fix This is the designed behavior since v0.0.4. If multi-line log statements are desired (which goes against the logfmt philosophy), either downgrade `logfmter` to a version prior to 0.0.4 (not recommended, as it fixes a bug) or adjust logging to avoid embedding newlines.

• ModuleNotFoundError: No module named 'logfmter'

  cause The `logfmter` package is not installed in the current Python environment.
  fix Install the package using pip: `pip install logfmter`.

• AttributeError: 'Logfmter' object has no attribute 'formatTime'

  cause Attempting to customize date format using `formatter.datefmt` directly on the `Logfmter` instance instead of passing `datefmt` during initialization.
  fix Pass the `datefmt` parameter directly to the `Logfmter` constructor, e.g., `Logfmter(datefmt="%Y-%m-%d %H:%M:%S")`.

Warnings

• breaking Python 3.9 support was dropped in `logfmter` v0.0.12. Users on Python 3.9 or older must pin their dependency to `logfmter<0.0.12`.
  Fix: Upgrade Python to 3.10 or newer, or pin `logfmter` dependency to `<0.0.12`.
• breaking Starting from v0.0.4, newline characters (`\n`) in all logged values are explicitly escaped to ensure single-line output per log record. If your application previously relied on generating multi-line log statements by embedding newlines, this behavior will change, and `\n` will appear literally in the output.
  Fix: Adjust logging messages to conform to the single-line logfmt standard. Avoid embedding newline characters or adapt your log parsing to handle escaped newlines.
• breaking In v0.0.10, proper escaping of backslashes was introduced to fix issues with log consumption by third-party tools. This is a fix, but it might alter the literal string representation of backslashes in your log output if previous versions had different (and potentially incorrect) behavior.
  Fix: Review log parsing tools to ensure they correctly interpret properly escaped backslashes. No code change is typically needed unless relying on the prior incorrect escaping behavior.
• gotcha When using `logging.config.dictConfig` to configure logging, the `disable_existing_loggers` parameter is `True` by default. This can inadvertently disable loggers from third-party modules that were configured before `dictConfig` is called, leading to missing log output from those libraries.
  Fix: If you need existing loggers (e.g., from third-party libraries) to continue logging, explicitly set `"disable_existing_loggers": false` within your `dictConfig` dictionary.
• gotcha `logfmter` does not officially support configuration via `logging.config.fileConfig`. This is due to `fileConfig`'s limitation in supporting custom formatter initialization, which `logfmter` relies on.
  Fix: Use `logging.basicConfig` for simple setups or `logging.config.dictConfig` for more complex logging configurations.

Install

• pip install logfmter Install stable version

Imports

• Logfmter

  from logfmter import Logfmter

Quickstart

This quickstart demonstrates how to integrate `logfmter` with the standard Python `logging` module using `basicConfig`. It sets up a `StreamHandler` to output logfmt-formatted messages to the console, including `extra` dictionary parameters which `logfmter` automatically flattens into key-value pairs. It also shows logging a dictionary as the main message.

import logging
import os
from logfmter import Logfmter

# Configure a basic StreamHandler with Logfmter
handler = logging.StreamHandler()
handler.setFormatter(Logfmter())

# Basic configuration to use the Logfmter handler
logging.basicConfig(handlers=[handler], level=os.environ.get('LOG_LEVEL', 'INFO'))

# Example logs
logging.info("Application started", env=os.environ.get('ENV', 'development'))
logging.warning("User login failed", user_id=123, ip_address="192.168.1.1", extra={"attempts": 3})
logging.error("An unexpected error occurred", error_code="E1001")

# Logging a dictionary directly as a message (keys will be flattened)
logging.info({"event": "user_data_processed", "status": "success", "records": 100})