{"id":8484,"library":"pygtail","title":"Pygtail","description":"Pygtail is a Python library and command-line tool that reads log file lines that have not been previously read. It is a 'port' of logcheck's logtail2 and is capable of handling log files that have been rotated. The current stable version is 0.14.0. Releases are infrequent, often tied to bug fixes or minor enhancements.","status":"active","version":"0.14.0","language":"en","source_language":"en","source_url":"http://github.com/bgreenlee/pygtail","tags":["logging","log tailing","file monitoring","log rotation"],"install":[{"cmd":"pip install pygtail","lang":"bash","label":"Install with pip"}],"dependencies":[],"imports":[{"note":"The primary class 'Pygtail' needs to be imported directly, not accessed via the top-level package name.","wrong":"import pygtail","symbol":"Pygtail","correct":"from pygtail import Pygtail"}],"quickstart":{"code":"import sys\nimport os\nfrom pygtail import Pygtail\n\n# Create a dummy log file for demonstration\nlog_file_path = 'example.log'\nwith open(log_file_path, 'w') as f:\n    f.write('Line 1\\n')\n    f.write('Line 2\\n')\n\nprint(f\"Reading new lines from {log_file_path}:\")\nfor line in Pygtail(log_file_path):\n    sys.stdout.write(line)\n\n# Simulate new log entries\nwith open(log_file_path, 'a') as f:\n    f.write('Line 3 (new)\\n')\n    f.write('Line 4 (new)\\n')\n\nprint(\"\\nReading newly added lines:\")\nfor line in Pygtail(log_file_path):\n    sys.stdout.write(line)\n\n# Clean up dummy log and offset files\nif os.path.exists(log_file_path):\n    os.remove(log_file_path)\nif os.path.exists(log_file_path + '.offset'):\n    os.remove(log_file_path + '.offset')","lang":"python","description":"This quickstart demonstrates how to use Pygtail to read new lines from a log file. It first creates a dummy log file, reads its initial content, then appends new lines, and finally uses Pygtail again to read only the newly added lines. Pygtail automatically manages an offset file (by default `logfile.offset`) to keep track of already read content."},"warnings":[{"fix":"Initialize `Pygtail` with `copytruncate=True` (e.g., `Pygtail('logfile.log', copytruncate=True)`). This enables support for `copytruncate`-style rotation.","message":"When log files are rotated using a 'copytruncate' method (where the file is emptied and rewritten), Pygtail might not detect this by default, leading to missing new entries. This is because the inode remains the same, but the content is reset.","severity":"gotcha","affected_versions":"<=0.14.0"},{"fix":"Specify the correct encoding when initializing `Pygtail` using the `encoding` parameter (e.g., `Pygtail('logfile.log', encoding='utf8')`).","message":"On Windows, Pygtail may encounter `UnicodeDecodeError` when reading log files containing non-ASCII characters, especially if the file was generated on a Linux system or uses UTF-8 encoding.","severity":"gotcha","affected_versions":"<=0.14.0"},{"fix":"Manage the lifecycle of the offset file carefully. If you need to re-read a file from the end if the offset file is missing, use the `read_from_end=True` parameter. For temporary reading without persistent offsets, you can specify `offset_file='/dev/null'` (on Unix-like systems) or use `save_on_end=False` and manually manage offsets with `tail.with_offsets()` and `tail.write_offset_to_file()`.","message":"Pygtail automatically creates an offset file (e.g., `logfile.log.offset`) to track read progress. If this offset file is deleted or becomes corrupted, Pygtail will re-read the entire log file from the beginning, which can lead to duplicate processing of log entries.","severity":"gotcha","affected_versions":"<=0.14.0"}],"env_vars":null,"last_verified":"2026-04-16T00:00:00.000Z","next_check":"2026-07-15T00:00:00.000Z","problems":[{"fix":"Pass the `encoding` parameter to the `Pygtail` constructor: `tail = Pygtail('your.log', encoding='utf8')`. Adjust 'utf8' to the actual encoding of your log file if different.","cause":"Attempting to read a log file with an encoding (e.g., UTF-8) that differs from the system's default encoding, particularly common on Windows systems expecting CP1252.","error":"UnicodeDecodeError: 'charmap' codec can't decode byte 0x__ in position __: character maps to <undefined>"},{"fix":"Enable `copytruncate` support by initializing `Pygtail` with `copytruncate=True`: `tail = Pygtail('your.log', copytruncate=True)`. If log rotation involves compression, `pygtail` might also need custom log patterns using the `--log-pattern` command-line option, though this is not directly supported in the Python API for dynamic patterns.","cause":"This can occur when log rotation is performed by moving the original log file and creating a new empty one, rather than truncating it (e.g., `copytruncate`). Pygtail, by default, might not recognize this as a rotation if the inode remains the same but the file content is reset.","error":"Pygtail stops updating offset file and always returns 0 for stats."},{"fix":"Ensure the directory where the log file resides (or where you specify the offset file) has write permissions for the user running the Pygtail script. Alternatively, specify an explicit and writable `offset_file` path: `tail = Pygtail('some.log', offset_file='/var/log/my_app/some.log.offset')`.","cause":"The default offset file name is derived from the log file. If Pygtail cannot create or access this file due to permissions or an invalid path, it will raise this error.","error":"FileNotFoundError: [Errno 2] No such file or directory: 'some.log.offset'"}]}