{"id":8446,"library":"prometheus-remote-writer","title":"Prometheus Remote Writer","description":"The `prometheus-remote-writer` is a Python library (version 1.1.3) designed to simplify the process of sending time-series data to Prometheus-compatible storage using the Prometheus Remote Write protocol. It offers a platform-independent, intuitive API, supporting custom HTTP headers, various authentication methods, and efficient batch data sending. The library is actively maintained with regular updates.","status":"active","version":"1.1.3","language":"en","source_language":"en","source_url":"https://github.com/xy-kong/prometheus-remote-writer","tags":["prometheus","monitoring","metrics","remote write","observability","time-series"],"install":[{"cmd":"pip install prometheus-remote-writer","lang":"bash","label":"Install stable version"}],"dependencies":[],"imports":[{"symbol":"RemoteWriter","correct":"from prometheus_remote_writer import RemoteWriter"}],"quickstart":{"code":"import os\nimport time\nfrom prometheus_remote_writer import RemoteWriter\n\n# Configure the remote write endpoint URL\nREMOTE_WRITE_URL = os.environ.get('PROMETHEUS_REMOTE_WRITE_URL', 'https://example.com/prometheus/api/v1/write')\n# Configure authentication (e.g., Bearer token)\nAUTH_TOKEN = os.environ.get('PROMETHEUS_REMOTE_WRITE_TOKEN', 'YOUR_STATIC_TOKEN_HERE')\n\n# Create a RemoteWriter instance\nwriter = RemoteWriter(\n    url=REMOTE_WRITE_URL,\n    headers={'Authorization': f'Bearer {AUTH_TOKEN}'}\n)\n\n# Prepare some sample data to send\n# Prometheus Remote Write expects timestamps in milliseconds\ncurrent_time_ms = int(time.time() * 1000)\n\ndata_points = [\n    {\n        'metric': {'__name__': 'my_app_cpu_usage_total', 'host': 'server1', 'region': 'us-east-1'},\n        'values': [9.5],\n        'timestamps': [current_time_ms]\n    },\n    {\n        'metric': {'__name__': 'my_app_memory_utilization_bytes', 'host': 'server1', 'region': 'us-east-1'},\n        'values': [1024 * 1024 * 500],\n        'timestamps': [current_time_ms]\n    }\n]\n\ntry:\n    # Send the data\n    writer.send(data_points)\n    print(f\"Successfully sent {len(data_points)} data points to {REMOTE_WRITE_URL}\")\nexcept Exception as e:\n    print(f\"Failed to send data: {e}\")","lang":"python","description":"This quickstart demonstrates how to initialize the `RemoteWriter` with a target URL and an authorization header, then construct and send a list of time-series data points. Timestamps should be in milliseconds. Ensure `PROMETHEUS_REMOTE_WRITE_URL` and `PROMETHEUS_REMOTE_WRITE_TOKEN` environment variables are set or replace the placeholder values."},"warnings":[{"fix":"Monitor official documentation for `prometheus-remote-writer` for v2.0 support. If interacting with a v2.0-only endpoint, ensure the library version supports it or configure it explicitly if options become available.","message":"The underlying Prometheus Remote Write protocol has a v2.0 specification that introduces breaking changes, specifically deprecating `prometheus.WriteRequest` in favor of `io.prometheus.write.v2.Request`. Future versions of this library or Prometheus itself might transition, requiring updates or explicit configuration for v2.0 endpoints.","severity":"breaking","affected_versions":"<=1.1.3"},{"fix":"Implement client-side rate limiting or backoff mechanisms, or ensure your remote storage backend has sufficient capacity and robust handling for spikes. Monitor your remote write queue metrics on the Prometheus server or client side to detect dropped samples.","message":"Prometheus, and by extension clients pushing to it via remote write, often drops data when a remote endpoint returns a `429 Too Many Requests` status code, rather than implementing retries. This can lead to silent data loss during periods of high load or transient outages at the remote storage.","severity":"gotcha","affected_versions":"All versions"},{"fix":"Monitor the health and memory usage of the application sending metrics. Ensure the remote storage is healthy and responsive. If possible, configure queue capacity and timeout parameters in your client or Prometheus configuration to prevent excessive buffering.","message":"Prometheus remote write queues can saturate, potentially leading to 'Out of Memory' (OOM) errors or dropped samples, especially if the remote endpoint is slow, unavailable, or the client's internal buffering is not adequately managed.","severity":"gotcha","affected_versions":"All versions"}],"env_vars":null,"last_verified":"2026-04-16T00:00:00.000Z","next_check":"2026-07-15T00:00:00.000Z","problems":[{"fix":"Verify the `url` and `headers` (especially authentication tokens) provided to the `RemoteWriter` instance. Check network connectivity from the client to the remote endpoint. Ensure the remote endpoint (e.g., Mimir, Thanos, Grafana Cloud) is operational and accessible.","cause":"This is a generic error often found in Prometheus logs (or client-side exceptions) indicating a failure to transmit data. Common reasons include incorrect remote write URL, invalid authentication credentials, network connectivity issues, or the remote endpoint being down or overloaded.","error":"Error sending samples to remote storage"},{"fix":"Reduce the volume or rate of metrics being sent. Review and adjust the ingestion limits on your remote storage solution. Consider aggregating metrics or applying `write_relabel_configs` on the Prometheus server (if applicable) to drop high-cardinality or less critical metrics.","cause":"The remote storage backend is rate-limiting the incoming requests from your client. This means the client is sending data faster than the remote storage can accept it or exceeding configured ingestion limits.","error":"429 Too Many Requests"},{"fix":"Double-check the `url` in `RemoteWriter` to ensure the hostname and port are correct. Verify that the remote write endpoint service is running and accessible on the specified port. Check any firewalls or network security groups that might be preventing the connection.","cause":"The client attempted to connect to the remote write endpoint, but the connection was actively refused. This typically indicates that the target server is not running, the port is incorrect, or a firewall is blocking the connection.","error":"connection refused"},{"fix":"Review the authentication configuration for your remote storage and ensure the exact credentials are being passed correctly in the `RemoteWriter`'s `headers` or `auth` parameters. Pay attention to case sensitivity and token expiration.","cause":"The credentials (e.g., API key, bearer token, username/password) provided in the `headers` or `auth` parameters are incorrect or unauthorized by the remote storage endpoint.","error":"authentication failed"}]}