Elasticsearch Curator
Elasticsearch Curator is a tool designed to manage Elasticsearch indices and snapshots programmatically via YAML configuration files. It simplifies tasks like deleting old indices, optimizing shards, or closing/opening indices. Currently at version 9.0.0, it follows an active release cadence, with major releases aligning with Elasticsearch versions and frequent patch releases for bug fixes and minor features.
Common errors
-
RuntimeError: Python 3.8 and 3.9 are no longer supported. Please upgrade to Python 3.10 or later.
cause Attempting to run Curator 9.x on an unsupported Python version.fixUpgrade your Python environment to 3.10, 3.11, 3.12, or 3.13. Alternatively, install Curator 8.x (`pip install 'elasticsearch-curator<9'`) if you must use an older Python. -
elasticsearch.exceptions.UnsupportedProductError: The client noticed that the server is not Elasticsearch and is not supported by this client version.
cause Curator 9.x is attempting to connect to an Elasticsearch 8.x cluster, which is incompatible.fixIf managing Elasticsearch 9.x, ensure your `client` configuration points to an ES 9.x cluster. If managing ES 8.x, downgrade Curator to 8.0.21 or an earlier 8.x version using `pip install 'elasticsearch-curator<9'`. -
curator.exceptions.ConfigurationError: Missing a required setting. Check your configuration file.
cause Your YAML configuration file for Curator (client or action) is malformed, missing required keys, or contains invalid values.fixReview your `curator.yml` (client config) and action YAML files against the official Curator documentation. Ensure proper YAML syntax and that all mandatory fields are present and correctly formatted. -
bash: curator: command not found
cause The `curator` executable is not in your system's PATH, or `elasticsearch-curator` was not installed correctly.fixEnsure `pip install elasticsearch-curator` completed successfully. If installed in a virtual environment, activate it. If using `pipx`, ensure `curator` is added to your path. You might also need to ensure your `~/.local/bin` is in your PATH.
Warnings
- breaking Curator 9.x requires Python 3.10 or later. Python 3.8 and 3.9 are no longer supported.
- breaking Curator 9.x exclusively supports Elasticsearch 9.x. It is not compatible with Elasticsearch 8.x or older versions.
- gotcha A critical bug in Curator 8.0.17 (and earlier 8.x versions) with the `age` filter could erroneously delete indices that did not match the `timestring` pattern, leading to unintended data loss.
- gotcha The `--ignore_empty_list` option was not respected in Curator 8.0.20 and earlier, potentially causing actions to fail if no indices matched the filters.
- gotcha In Curator 8.0.13, default values from command-line options could incorrectly override settings defined in configuration files due to an `es_client` bug.
- gotcha Curator is primarily a command-line interface (CLI) application. While a Python API exists, programmatic use as a library is not its main design goal and might require more complex setup than intended.
Install
-
pip install elasticsearch-curator
Imports
- curator
import curator
- cli
from curator import cli
Quickstart
import subprocess
import os
# Most common usage is via CLI with configuration files.
# Example: Create a dummy action file and run curator in dry-run.
# Create a dummy action configuration file
action_config_content = """
actions:
1:
action: delete_indices
description: "Delete indices older than 30 days matching 'logstash-'"
options:
ignore_empty_list: True
timeout_override:
continue_if_exception: False
disable_action: False
dry_run: True # Always set dry_run in example code for safety
filters:
- filtertype: pattern
kind: prefix
value: logstash-
- filtertype: age
source: name
direction: older
unit: days
unit_count: 30
"""
action_file_path = "curator_action.yml"
with open(action_file_path, "w") as f:
f.write(action_config_content)
try:
# Get Curator version to check installation
print("Checking Curator version:")
version_result = subprocess.run(
["curator", "--version"], capture_output=True, text=True, check=True
)
print(version_result.stdout.strip())
# Simulate running an action in dry-run mode
print("\nSimulating an index deletion action (dry-run):")
# Note: A client configuration is typically also needed in a separate file,
# or specified via CLI arguments like --host, --port etc.
# For simplicity, this example assumes default client config or
# that ES is running on localhost:9200 and no auth is needed.
# For a real run, create a 'curator.yml' client config file or pass
# client options directly.
dry_run_result = subprocess.run(
["curator", "--host", "localhost", "--port", "9200", action_file_path, "--dry-run"],
capture_output=True,
text=True,
check=True
)
print(dry_run_result.stdout)
if dry_run_result.stderr:
print("STDERR:", dry_run_result.stderr)
except FileNotFoundError:
print("Error: 'curator' command not found. Is elasticsearch-curator installed?")
except subprocess.CalledProcessError as e:
print(f"Curator command failed with exit code {e.returncode}")
print(f"STDOUT: {e.stdout}")
print(f"STDERR: {e.stderr}")
finally:
if os.path.exists(action_file_path):
os.remove(action_file_path)
print("\nThis demonstrates how to run a Curator action in dry-run mode.")
print("For actual index management, remove '--dry-run' and ensure Elasticsearch is running.")
print("Refer to official documentation for full client and action configuration.")