{"id":1039,"library":"watchtower","title":"Watchtower (Python CloudWatch Logging)","description":"Watchtower is a log handler for Amazon Web Services (AWS) CloudWatch Logs. It acts as a lightweight adapter between the Python `logging` system and CloudWatch Logs, using the `boto3` AWS SDK to aggregate logs into batches and send them to AWS. It is currently at version 3.4.0 and sees regular, although not strictly scheduled, releases with bug fixes and new features.","status":"active","version":"3.4.0","language":"python","source_language":"en","source_url":"https://github.com/kislyuk/watchtower","tags":["logging","aws","cloudwatch"],"install":[{"cmd":"pip install watchtower","lang":"bash","label":"Install Watchtower"}],"dependencies":[{"reason":"Required for interacting with AWS CloudWatch Logs API.","package":"boto3","optional":false}],"imports":[{"symbol":"CloudWatchLogHandler","correct":"from watchtower import CloudWatchLogHandler"},{"symbol":"logging","correct":"import logging"}],"quickstart":{"code":"import logging\nimport os\nfrom watchtower import CloudWatchLogHandler\n\n# Configure basic logging\nlogging.basicConfig(level=logging.INFO)\nlogger = logging.getLogger(__name__)\n\n# Set AWS credentials via environment variables for boto3 (e.g., AWS_ACCESS_KEY_ID, AWS_SECRET_ACCESS_KEY, AWS_REGION)\n# Or configure AWS CLI (aws configure) for boto3 to pick up credentials automatically\n\n# Create a CloudWatchLogHandler instance\n# Recommended: specify log_group_name and region_name explicitly\n# boto3 will automatically pick up credentials from env vars or IAM roles.\nhandler = CloudWatchLogHandler(\n    log_group_name=os.environ.get('AWS_LOG_GROUP', 'my-python-app'),\n    region_name=os.environ.get('AWS_REGION', 'us-east-1')\n)\n\n# Add the handler to the logger\nlogger.addHandler(handler)\n\n# Log some messages\nlogger.info('Hello from Watchtower!')\nlogger.warning('This is a warning message.')\nlogger.error(dict(error_code=500, message='Something went wrong'))\n\n# For applications that might exit quickly, ensure logs are flushed\n# In typical web applications or long-running services, this is handled on shutdown.\nhandler.flush()","lang":"python","description":"This quickstart demonstrates how to integrate Watchtower with the Python `logging` module to send logs to AWS CloudWatch. It sets up a basic logger and a `CloudWatchLogHandler`, then sends a few example log messages. Ensure your AWS credentials and default region are configured either via environment variables (`AWS_ACCESS_KEY_ID`, `AWS_SECRET_ACCESS_KEY`, `AWS_REGION`) or through the AWS CLI (`aws configure`) for `boto3` to automatically pick them up."},"warnings":[{"fix":"Review existing code that logs complex objects and adjust downstream log parsing or processing logic to account for `repr()` string representations. If the old behavior is desired, custom JSON serialization should be implemented before passing data to the logger.","message":"Starting with v3.0.0, non-JSON-serializable objects (e.g., `datetime` objects, custom classes) in log messages are now represented by their `repr()` string instead of being converted to `isoformat()` or `null`. This change might increase log data volume or alter parsing expectations.","severity":"breaking","affected_versions":">=3.0.0"},{"fix":"Ensure your Python environment uses a version of `watchtower` that includes the fix. For your own code, replace `datetime.utcnow()` with `datetime.now(timezone.utc)`.","message":"The `datetime.utcnow()` method, previously used internally by Watchtower, has been replaced due to deprecation. While this primarily affects internal implementation, users with custom logging formats or direct manipulation of `datetime` objects might encounter related issues or are encouraged to update their own code to use timezone-aware `datetime.now(timezone.utc)` for consistency.","severity":"deprecated","affected_versions":">=3.1.0"},{"fix":"Be aware of the byte-length limit for log messages. If logs frequently contain long Unicode strings, consider pre-truncating or encoding them to ensure meaningful parts are preserved within the byte limit before they reach Watchtower.","message":"As of v3.0.1, Watchtower truncates log messages based on byte length (256 KB CloudWatch Logs limit) rather than Unicode character count. This can lead to unexpected truncation of messages containing multi-byte Unicode characters, potentially cutting off messages mid-character.","severity":"gotcha","affected_versions":">=3.0.1"},{"fix":"If customizing log stream names, ensure they comply with CloudWatch naming rules and the expected `strftime` format. For high-volume applications or those using process pools, ensure the log stream name is unique per source using template variables like `{machine_name}/{program_name}/{logger_name}/{process_id}`.","message":"CloudWatch log stream naming conventions changed in v3.4.0, specifically removing ':' from `program_name` to prevent issues. Also, `strftime` format strings are explicitly noted as being required for certain configurations. Incorrect stream naming can lead to logs not appearing where expected or creating many unintended log streams.","severity":"gotcha","affected_versions":">=3.4.0"},{"fix":"To reduce noise, set the logging level for `boto3`, `botocore`, and `urllib3` to `WARNING` or higher. For example: `logging.getLogger('boto3').setLevel(logging.WARNING)`, `logging.getLogger('botocore').setLevel(logging.WARNING)`, `logging.getLogger('urllib3').setLevel(logging.WARNING)`.","message":"Watchtower uses `boto3`, which in turn relies on `botocore` and `urllib3`. These dependencies can produce a significant amount of `DEBUG` level log messages, which can overwhelm application logs if not properly filtered.","severity":"gotcha","affected_versions":"*"},{"fix":"Attach an AWS managed IAM policy (e.g., `CloudWatchLogsFullAccess` for testing, or a more restrictive custom policy with `logs:CreateLogGroup`, `logs:CreateLogStream`, `logs:PutLogEvents`) to the IAM role or user credentials used by your application. Refer to `boto3` credentials documentation for how credentials are loaded.","message":"The process running Watchtower requires appropriate AWS Identity and Access Management (IAM) permissions to call the CloudWatch Logs API (e.g., `logs:CreateLogGroup`, `logs:CreateLogStream`, `logs:PutLogEvents`). Lack of these permissions is a common reason for logs not appearing in CloudWatch.","severity":"gotcha","affected_versions":"*"},{"fix":"Upgrade `watchtower` to version 3.0.0 or higher to use the `region_name` argument directly. If upgrading is not an option, configure the AWS region via a `boto3.session.Session` object and pass it using the `session` argument to `CloudWatchLogHandler` (e.g., `CloudWatchLogHandler(session=boto3.Session(region_name='your-region'))`), or ensure the AWS region is configured through environment variables or AWS config files which `boto3` will pick up automatically.","message":"The `CloudWatchLogHandler` constructor received an unexpected keyword argument 'region_name'. This argument was introduced in `watchtower` v3.0.0. Using it with older versions (prior to v3.0.0) will result in a `TypeError`.","severity":"breaking","affected_versions":"<3.0.0"},{"fix":"Remove `region_name` from the `CloudWatchLogHandler` constructor. Ensure the AWS region is set via `AWS_REGION` environment variable or provide a `boto3.Session` configured with the desired region using the `boto3_session` argument. For example, `handler = CloudWatchLogHandler(log_group_name='my-group', boto3_session=boto3.Session(region_name='us-east-1'))` or simply rely on `AWS_REGION` environment variable if it's set.","message":"Starting with Watchtower v3.0.0, the `CloudWatchLogHandler` no longer accepts `region_name` as a direct keyword argument in its constructor. The AWS region must now be configured through environment variables (e.g., `AWS_REGION`), `boto3` configuration, or by providing an initialized `boto3.Session` object via the `boto3_session` argument.","severity":"breaking","affected_versions":">=3.0.0"}],"env_vars":null,"last_verified":"2026-05-12T22:56:31.465Z","next_check":"2026-06-27T00:00:00.000Z","problems":[{"fix":"pip install watchtower","cause":"The 'watchtower' package is not installed in the current Python environment.","error":"ModuleNotFoundError: No module named 'watchtower'"},{"fix":"from watchtower import WatchtowerLogHandler","cause":"The class name for the CloudWatch log handler was changed from `CloudWatchLogHandler` to `WatchtowerLogHandler` in library version 0.7.0; this error occurs when using an outdated import statement.","error":"ImportError: cannot import name 'CloudWatchLogHandler' from 'watchtower'"},{"fix":"Configure AWS credentials using environment variables (e.g., AWS_ACCESS_KEY_ID, AWS_SECRET_ACCESS_KEY), the shared credentials file (~/.aws/credentials), or an attached IAM role for EC2/ECS/EKS/Lambda.","cause":"Boto3, used by Watchtower, cannot find AWS access keys or assume an IAM role, which are necessary to interact with AWS services.","error":"botocore.exceptions.NoCredentialsError: Unable to locate credentials"},{"fix":"Grant the IAM entity (user/role) permissions for `logs:CreateLogGroup`, `logs:CreateLogStream`, and `logs:PutLogEvents` on the relevant CloudWatch Logs resources.","cause":"The AWS IAM user or role assumed by Watchtower lacks the necessary permissions to write log events to the specified CloudWatch Logs group and stream.","error":"botocore.exceptions.ClientError: An error occurred (AccessDeniedException) when calling the PutLogEvents operation: User is not authorized to perform: logs:PutLogEvents on resource"}],"ecosystem":"pypi","meta_description":null,"install_score":100,"install_tag":"verified","quickstart_score":null,"quickstart_tag":null,"pypi_latest":"3.4.0","cli_name":"","install_checks":{"last_tested":"2026-05-12","tag":"verified","tag_description":"installs cleanly on critical runtimes, fast import, recently tested","installed_version":"3.4.0","pypi_latest":"3.4.0","is_stale":false,"results":[{"runtime":"python:3.10-alpine","python_version":"3.10","os_libc":"alpine (musl)","variant":"watchtower","exit_code":0,"wheel_type":"wheel","failure_reason":null,"import_side_effects":"clean","install_time_s":null,"import_time_s":0.74,"mem_mb":14.3,"disk_size":"50.7M"},{"runtime":"python:3.10-alpine","python_version":"3.10","os_libc":"alpine (musl)","variant":"watchtower","exit_code":0,"wheel_type":null,"failure_reason":null,"import_side_effects":null,"install_time_s":null,"import_time_s":0.69,"mem_mb":14.3,"disk_size":"50.5M"},{"runtime":"python:3.10-slim","python_version":"3.10","os_libc":"slim (glibc)","variant":"watchtower","exit_code":0,"wheel_type":"wheel","failure_reason":null,"import_side_effects":"clean","install_time_s":3.9,"import_time_s":0.5,"mem_mb":14.3,"disk_size":"51M"},{"runtime":"python:3.10-slim","python_version":"3.10","os_libc":"slim (glibc)","variant":"watchtower","exit_code":0,"wheel_type":null,"failure_reason":null,"import_side_effects":null,"install_time_s":null,"import_time_s":0.54,"mem_mb":14.3,"disk_size":"51M"},{"runtime":"python:3.11-alpine","python_version":"3.11","os_libc":"alpine (musl)","variant":"watchtower","exit_code":0,"wheel_type":"wheel","failure_reason":null,"import_side_effects":"clean","install_time_s":null,"import_time_s":0.85,"mem_mb":17,"disk_size":"53.6M"},{"runtime":"python:3.11-alpine","python_version":"3.11","os_libc":"alpine (musl)","variant":"watchtower","exit_code":0,"wheel_type":null,"failure_reason":null,"import_side_effects":null,"install_time_s":null,"import_time_s":0.95,"mem_mb":17.1,"disk_size":"53.4M"},{"runtime":"python:3.11-slim","python_version":"3.11","os_libc":"slim (glibc)","variant":"watchtower","exit_code":0,"wheel_type":"wheel","failure_reason":null,"import_side_effects":"clean","install_time_s":3.6,"import_time_s":0.75,"mem_mb":17,"disk_size":"54M"},{"runtime":"python:3.11-slim","python_version":"3.11","os_libc":"slim (glibc)","variant":"watchtower","exit_code":0,"wheel_type":null,"failure_reason":null,"import_side_effects":null,"install_time_s":null,"import_time_s":0.73,"mem_mb":17,"disk_size":"54M"},{"runtime":"python:3.12-alpine","python_version":"3.12","os_libc":"alpine (musl)","variant":"watchtower","exit_code":0,"wheel_type":"wheel","failure_reason":null,"import_side_effects":"clean","install_time_s":null,"import_time_s":0.8,"mem_mb":16,"disk_size":"45.2M"},{"runtime":"python:3.12-alpine","python_version":"3.12","os_libc":"alpine (musl)","variant":"watchtower","exit_code":0,"wheel_type":null,"failure_reason":null,"import_side_effects":null,"install_time_s":null,"import_time_s":0.81,"mem_mb":16,"disk_size":"45.1M"},{"runtime":"python:3.12-slim","python_version":"3.12","os_libc":"slim (glibc)","variant":"watchtower","exit_code":0,"wheel_type":"wheel","failure_reason":null,"import_side_effects":"clean","install_time_s":3.3,"import_time_s":0.78,"mem_mb":16,"disk_size":"46M"},{"runtime":"python:3.12-slim","python_version":"3.12","os_libc":"slim (glibc)","variant":"watchtower","exit_code":0,"wheel_type":null,"failure_reason":null,"import_side_effects":null,"install_time_s":null,"import_time_s":0.78,"mem_mb":16,"disk_size":"46M"},{"runtime":"python:3.13-alpine","python_version":"3.13","os_libc":"alpine (musl)","variant":"watchtower","exit_code":0,"wheel_type":"wheel","failure_reason":null,"import_side_effects":"clean","install_time_s":null,"import_time_s":0.77,"mem_mb":17,"disk_size":"45.0M"},{"runtime":"python:3.13-alpine","python_version":"3.13","os_libc":"alpine (musl)","variant":"watchtower","exit_code":0,"wheel_type":null,"failure_reason":null,"import_side_effects":null,"install_time_s":null,"import_time_s":0.79,"mem_mb":17,"disk_size":"44.7M"},{"runtime":"python:3.13-slim","python_version":"3.13","os_libc":"slim (glibc)","variant":"watchtower","exit_code":0,"wheel_type":"wheel","failure_reason":null,"import_side_effects":"clean","install_time_s":2.9,"import_time_s":0.8,"mem_mb":17,"disk_size":"45M"},{"runtime":"python:3.13-slim","python_version":"3.13","os_libc":"slim (glibc)","variant":"watchtower","exit_code":0,"wheel_type":null,"failure_reason":null,"import_side_effects":null,"install_time_s":null,"import_time_s":0.84,"mem_mb":17,"disk_size":"45M"},{"runtime":"python:3.9-alpine","python_version":"3.9","os_libc":"alpine (musl)","variant":"watchtower","exit_code":0,"wheel_type":"wheel","failure_reason":null,"import_side_effects":"clean","install_time_s":null,"import_time_s":0.53,"mem_mb":13.6,"disk_size":"50.1M"},{"runtime":"python:3.9-alpine","python_version":"3.9","os_libc":"alpine (musl)","variant":"watchtower","exit_code":0,"wheel_type":null,"failure_reason":null,"import_side_effects":null,"install_time_s":null,"import_time_s":0.57,"mem_mb":13.6,"disk_size":"50.0M"},{"runtime":"python:3.9-slim","python_version":"3.9","os_libc":"slim (glibc)","variant":"watchtower","exit_code":0,"wheel_type":"wheel","failure_reason":null,"import_side_effects":"clean","install_time_s":4.6,"import_time_s":0.52,"mem_mb":13.6,"disk_size":"51M"},{"runtime":"python:3.9-slim","python_version":"3.9","os_libc":"slim (glibc)","variant":"watchtower","exit_code":0,"wheel_type":null,"failure_reason":null,"import_side_effects":null,"install_time_s":null,"import_time_s":0.48,"mem_mb":13.6,"disk_size":"50M"}]},"quickstart_checks":{"last_tested":"2026-04-24","tag":null,"tag_description":null,"results":[{"runtime":"python:3.10-alpine","exit_code":1},{"runtime":"python:3.10-slim","exit_code":1},{"runtime":"python:3.11-alpine","exit_code":1},{"runtime":"python:3.11-slim","exit_code":1},{"runtime":"python:3.12-alpine","exit_code":1},{"runtime":"python:3.12-slim","exit_code":1},{"runtime":"python:3.13-alpine","exit_code":1},{"runtime":"python:3.13-slim","exit_code":1},{"runtime":"python:3.9-alpine","exit_code":1},{"runtime":"python:3.9-slim","exit_code":1}]}}