Aliyun Log Service Python SDK

0.9.45 verified Thu Apr 16 auth: no python

The Aliyun Log Service Python SDK provides a client library for interacting with Alibaba Cloud's Log Service. It allows users to collect, consume, query, and process logs. The current version is 0.9.45, and it generally follows a continuous release cadence with frequent bug fixes and feature additions.

pip install aliyun-log-python-sdk

Common errors

error ModuleNotFoundError: No module named 'aliyun.log.consumer' ↓

cause Attempting to import `ConsumerGroupWorker` directly from `aliyun.log.consumer`, which is not the correct path in recent versions.

fix

The ConsumerGroupWorker class is located at aliyun.log.consumer.consumer_group_worker. Use from aliyun.log.consumer.consumer_group_worker import ConsumerGroupWorker.

error Failed to put logs: RequestError: InvalidAccessKeyId, Message: The Access Key ID is invalid. ↓

cause The provided Alibaba Cloud Access Key ID is either incorrect, empty, or malformed.

fix

Verify your accesskey_id value. Ensure it is correctly copied, has no leading/trailing spaces, and is associated with your Alibaba Cloud account.

error RequestError: LogServiceError: SignatureDoesNotMatch, Message: The request signature we calculated does not match the signature you provided. ↓

cause This error typically indicates an incorrect Access Key Secret, an incorrect Endpoint, or unsynchronized system time, leading to a mismatch in the request signature calculation.

fix

Double-check your accesskey_secret, accesskey_id, and endpoint. Ensure the endpoint matches the region of your Log Service project. Also, verify your system's time is accurate and synchronized.

error AttributeError: 'LogClient' object has no attribute 'put_compress_logs' ↓

cause Attempting to call an outdated or non-existent method for putting compressed logs. In newer versions, compression is often handled internally based on available dependencies or through parameters of existing methods.

fix

For LZ4 compression, ensure lz4a is installed via pip install aliyun-log-python-sdk[lz4]. The put_logs method will then automatically utilize LZ4 if available. Avoid calling specific compression methods directly unless specified by the latest documentation.

Warnings

breaking The API signatures for the Shipper component were updated in version 0.6.45. Existing code using the Shipper API will likely break. ↓

fix Review the official documentation or release notes for v0.6.45 regarding Shipper API changes and update your code accordingly.

breaking The interface for `ConsumerProcessorBase` used in Consumer Group implementations was enhanced in version 0.6.40. Custom processors might require updates to align with the new interface. ↓

fix Consult the updated examples and documentation for `ConsumerProcessorBase` in the `tests/consumer_group_examples` directory of the SDK repository.

gotcha While the SDK generally supports both Python 2 and 3, specific dependency versions (e.g., protobuf, dateparser) were pinned in v0.7.0 and later to explicitly maintain Python 2 compatibility. Using different versions or older SDK releases with Python 2 might lead to unexpected compatibility issues. ↓

fix For Python 2 compatibility, ensure your environment adheres to the pinned dependency versions mentioned in SDK releases (e.g., `protobuf<=3.17.3, dateparser<=0.7.6`). Python 3 is generally more robust and recommended for new development.

gotcha To enable LZ4 compression for faster log transmission (supported from v0.6.42), the `lz4a` package must be explicitly installed. Without it, the SDK will automatically fall back to GZIP compression. ↓

fix Install the `lz4a` dependency: `pip install aliyun-log-python-sdk[lz4]` or `pip install lz4a`.

Install

pip install aliyun-log-python-sdk[lz4]

Imports

LogClient
```
from aliyun.log import LogClient
```
LogItem
```
from aliyun.log import LogItem
```

PutLogsRequest

from aliyun.log.put_logs_request import PutLogsRequest

ConsumerGroupWorker
wrong
```
from aliyun.log.consumer import ConsumerGroupWorker
```
correct
```
from aliyun.log.consumer.consumer_group_worker import ConsumerGroupWorker
```
The ConsumerGroupWorker class is nested deeper within the 'consumer' module in recent versions, requiring a more specific import path.

Quickstart

This quickstart demonstrates how to initialize the LogClient and put a single log item to an Aliyun Log Service logstore. Ensure you replace placeholder values with your actual project, logstore, endpoint, and credentials, preferably loaded from environment variables.

import os
from aliyun.log import LogClient, LogItem, PutLogsRequest
import time

# Replace with your actual Aliyun Log Service details
endpoint = os.environ.get('ALIYUN_LOG_ENDPOINT', 'https://cn-hangzhou.log.aliyuncs.com') # e.g., 'cn-hangzhou.log.aliyuncs.com'
accesskey_id = os.environ.get('ALIYUN_AK_ID', '')
accesskey_secret = os.environ.get('ALIYUN_AK_SECRET', '')
project_name = os.environ.get('ALIYUN_LOG_PROJECT', 'your-project-name')
logstore_name = os.environ.get('ALIYUN_LOG_LOGSTORE', 'your-logstore-name')

if not all([accesskey_id, accesskey_secret]):
    print("Please set ALIYUN_AK_ID and ALIYUN_AK_SECRET environment variables.")
else:
    client = LogClient(endpoint, accesskey_id, accesskey_secret)

    # Create a log item
    log_item = LogItem()
    log_item.set_time(int(time.time()))
    log_item.set_contents([('level', 'INFO'), ('message', 'Hello from Python SDK'), ('source', 'quickstart')])

    # Put logs into a list
    log_items = [log_item]

    # Create a PutLogsRequest
    request = PutLogsRequest(
        project_name, 
        logstore_name, 
        'your-topic', 
        'your-source-ip', # Or use client.get_source_ip() if needed
        log_items
    )

    try:
        response = client.put_logs(request)
        print(f"PutLogs successful. Request ID: {response.get_request_id()}")
    except Exception as e:
        print(f"Failed to put logs: {e}")