{"id":2447,"library":"concurrent-log-handler","title":"Concurrent Log Handler","description":"A robust drop-in replacement for Python's `RotatingFileHandler` and `TimedRotatingFileHandler` that supports safe concurrent writes from multiple processes and threads. It also includes features like gzip compression and better Windows compatibility. The library is actively maintained, with frequent minor releases addressing bugs and adding features, typically every few months.","status":"active","version":"0.9.29","language":"en","source_language":"en","source_url":"https://github.com/Preston-Landers/concurrent-log-handler","tags":["logging","concurrency","file rotation","multi-process","multi-threading"],"install":[{"cmd":"pip install concurrent-log-handler","lang":"bash","label":"Install stable version"}],"dependencies":[],"imports":[{"note":"The standard RotatingFileHandler does not provide robust concurrent access from multiple processes.","wrong":"from logging.handlers import RotatingFileHandler","symbol":"ConcurrentRotatingFileHandler","correct":"from concurrent_log_handler import ConcurrentRotatingFileHandler"},{"note":"The standard TimedRotatingFileHandler does not provide robust concurrent access from multiple processes.","wrong":"from logging.handlers import TimedRotatingFileHandler","symbol":"ConcurrentTimedRotatingFileHandler","correct":"from concurrent_log_handler import ConcurrentTimedRotatingFileHandler"}],"quickstart":{"code":"import logging\nimport os\nfrom concurrent_log_handler import ConcurrentRotatingFileHandler\n\nlog_file_path = os.path.join(os.getcwd(), 'app.log')\n\n# Configure logging\nlog = logging.getLogger('my_app')\nlog.setLevel(logging.INFO)\n\n# Use ConcurrentRotatingFileHandler\n# Rotate log after reaching 10MB, keep 5 old copies.\n# Set encoding to 'utf-8' to avoid UnicodeEncodeError on some systems.\nhandler = ConcurrentRotatingFileHandler(\n    log_file_path, 'a', 10 * 1024 * 1024, 5, encoding='utf-8'\n)\n\nformatter = logging.Formatter('%(asctime)s - %(name)s - %(levelname)s - %(message)s')\nhandler.setFormatter(formatter)\nlog.addHandler(handler)\n\nlog.info('This is an informational message.')\nlog.warning('This is a warning message.')\nlog.error('This is an error message.')\n\nprint(f\"Log messages written to: {log_file_path}\")","lang":"python","description":"This example demonstrates how to set up a basic logger using `ConcurrentRotatingFileHandler` to ensure log file rotation works safely across multiple processes, preventing data loss or corruption due to concurrent writes. It configures a log file to rotate after reaching 10MB, keeping 5 backup copies."},"warnings":[{"fix":"Upgrade your Python environment to 3.6 or newer if you are using an older Python 2.x interpreter.","message":"Python 2.x support was dropped. Versions 0.9.23 and higher require Python 3.6 or higher.","severity":"breaking","affected_versions":">=0.9.23"},{"fix":"As of 0.9.29, inherited handlers automatically reopen lock files for isolation. However, it is still strongly recommended to re-initialize your logging configuration *after* calling `os.fork()` in child processes to ensure full isolation and robust behavior.","message":"When using `fork()` to create new processes, log handlers inherited by child processes could experience race conditions, leading to corrupted logs or deadlocks.","severity":"gotcha","affected_versions":"<0.9.29"},{"fix":"Remove calls to `setup_logging_queues()`. The standard synchronous logging calls with this handler are generally sufficient and recommended.","message":"The `queue.py` `setup_logging_queues()` function has been deprecated and should no longer be used.","severity":"deprecated","affected_versions":">=0.9.27"},{"fix":"Upgrade to version 0.9.28 or later. The handler now performs catch-up rollovers on initialization if scheduled times were missed.","message":"Before 0.9.28, if an application shut down during a scheduled rollover time (e.g., for `ConcurrentTimedRotatingFileHandler`), the handler might miss the rollover entirely and skip to the next scheduled time without rotating accumulated logs.","severity":"gotcha","affected_versions":"<0.9.28"},{"fix":"No direct fix needed, but be aware of this internal behavior. If you experience unexpected issues with file handles, you can revert to the old behavior by explicitly setting `keep_file_open=False` in the handler constructor, though this may impact performance.","message":"Since 0.9.26, the handler keeps file handles open (`keep_file_open=True` by default) for performance. However, on Windows, the log file must still be closed after each write for rollovers to occur correctly due to filesystem API behavior.","severity":"gotcha","affected_versions":">=0.9.26"}],"env_vars":null,"last_verified":"2026-04-10T00:00:00.000Z","next_check":"2026-07-09T00:00:00.000Z"}