{"id":6349,"library":"drain3","title":"Drain3 Log Template Miner","description":"Drain3 is a Python library for mining log templates from raw log messages, designed for stream processing. It's based on the Drain algorithm and is suitable for real-time log analysis. The library is actively maintained with frequent patch releases, currently at version 0.9.11.","status":"active","version":"0.9.11","language":"en","source_language":"en","source_url":"https://github.com/IBM/Drain3","tags":["log-parsing","template-mining","nlp","clustering","observability"],"install":[{"cmd":"pip install drain3","lang":"bash","label":"Install stable version"}],"dependencies":[{"reason":"Required for loading configuration from YAML/INI files.","package":"PyYAML","optional":true},{"reason":"Required for Redis-based persistence backend.","package":"redis","optional":true},{"reason":"Required for HTTP-based persistence backend.","package":"requests","optional":true}],"imports":[{"note":"The main `Drain3` class was moved to the top-level package in v0.9.0. Older versions used `from drain3.drain import Drain3`.","wrong":"from drain3.drain import Drain3","symbol":"Drain3","correct":"from drain3 import Drain3"},{"note":"The `TemplateMinerConfig` class was moved and significantly refactored in v0.9.0. It is now imported from its own module.","wrong":"from drain3.drain import TemplateMinerConfig","symbol":"TemplateMinerConfig","correct":"from drain3.template_miner_config import TemplateMinerConfig"}],"quickstart":{"code":"from drain3 import Drain3\nfrom drain3.template_miner_config import TemplateMinerConfig\nimport os\n\n# Configure Drain3. For production, consider loading from a file or Redis.\n# Example for file persistence:\n# config = TemplateMinerConfig.load('drain3.ini')\n# Ensure 'persist_state_to_file' and 'state_file_path' are set in config.\n\nconfig = TemplateMinerConfig()\nconfig.load_default_config()\nconfig.drain_sim_th = 0.4\nconfig.depth = 4\n# If you want to use file persistence (ensure directory exists and is writable):\n# config.persistence_type = 'FILE'\n# config.file_persistence_path = os.path.join(os.getcwd(), 'drain3_state.bin')\n\ndrain = Drain3(config)\n\nlog_messages = [\n    \"081109 203619 143 INFO dfs.DataNode$PacketResponder: PacketResponder \"\n    \"0 for block blk_3886504917409280145 terminating\",\n    \"081109 203619 369 INFO dfs.DataNode$PacketResponder: PacketResponder \"\n    \"0 for block blk_-6755409170280820986 terminating\",\n    \"081109 203620 357 INFO dfs.DataNode$PacketResponder: PacketResponder \"\n    \"2 for block blk_814013142207908518 terminating\",\n    \"081109 203620 543 INFO dfs.DataNode$DataXceiver: Receiving block blk_-6755409170280820986 \"\n    \"src: /10.250.9.141:50106 dest: /10.250.9.141:50010\",\n]\n\nfor log_message in log_messages:\n    cluster_id = drain.add_log_message(log_message)\n    print(f\"Log: '{log_message}' -> Cluster ID: {cluster_id}\")\n\n# After processing, it's good practice to save the state if using persistence\n# if config.persistence_type == 'FILE':\n#     drain.save_state()\n\nprint(\"\\n--- Current Clusters ---\")\nfor cluster in drain.drain.clusters:\n    print(cluster)\n","lang":"python","description":"This quickstart demonstrates how to initialize Drain3 with a basic configuration, add log messages, and retrieve the identified log clusters. For production use, consider configuring persistence (file, Redis, or HTTP) via `TemplateMinerConfig` to save and load the model state."},"warnings":[{"fix":"Review the official documentation and update imports and configuration instantiation. For example, `config = TemplateMinerConfig()` instead of `config = TemplateMinerConfig.load('config.ini')`, and `from drain3 import Drain3`.","message":"Version 0.9.0 introduced significant breaking changes, including a major refactor of the API. The `LogViewer` module was removed, and `TemplateMinerConfig` was refactored from a static method `load()` to a class that can be instantiated and configured. Imports for `Drain3` and `TemplateMinerConfig` have also changed.","severity":"breaking","affected_versions":">=0.9.0"},{"fix":"Configure a persistence backend (FILE, REDIS, or HTTP) using `TemplateMinerConfig`. Call `drain.save_state()` periodically or upon application shutdown, and `drain.load_state()` upon application startup to ensure state is maintained. Example: `config.persistence_type = 'FILE'; config.file_persistence_path = 'path/to/state.bin'`.","message":"Drain3 does not automatically persist its state (the learned log templates) by default. If your application restarts without explicit state saving and loading, it will lose all learned templates and start learning from scratch, leading to reprocessing and inconsistent cluster IDs.","severity":"gotcha","affected_versions":"All"},{"fix":"Experiment with `drain_sim_th` and `depth` based on your log data characteristics. A lower `drain_sim_th` (e.g., 0.3-0.4) creates more general templates, while a higher value (e.g., 0.6-0.8) creates more specific ones. `depth` controls how many tokens are considered for tree traversal before a new node is created.","message":"The quality of log templates heavily depends on the `TemplateMinerConfig` parameters, especially `drain_sim_th` (similarity threshold) and `depth`. Incorrect settings can lead to overly generic or too specific templates, reducing the effectiveness of log parsing.","severity":"gotcha","affected_versions":"All"}],"env_vars":null,"last_verified":"2026-04-15T00:00:00.000Z","next_check":"2026-07-14T00:00:00.000Z"}