Elasticsearch Python Client (v8)

Warnings

• gotcha The PyPI package name is `elasticsearch8`, but the import statement remains `from elasticsearch import Elasticsearch`. Attempting `from elasticsearch8 import Elasticsearch` will result in a `ModuleNotFoundError`.
  Fix: Always use `from elasticsearch import Elasticsearch` after `pip install elasticsearch8`.
• breaking Migration from `elasticsearch-py` v7 to `elasticsearch8` (v8.x) introduces several breaking changes. Key changes include: `request_timeout` parameter renamed to `timeout`, removal of automatic `sniffing` for cluster discovery, and `verify_certs` defaulting to `True` for enhanced security.
  Fix: Review the official migration guide for `elasticsearch-py` v8.x. Update `request_timeout` to `timeout`, explicitly manage connection details instead of relying on sniffing, and ensure proper CA certificates are configured for production environments or set `verify_certs=False` (with caution) for development.
• gotcha For asynchronous operations, you must install the `[async]` extra (e.g., `pip install elasticsearch8[async]`) and import `AsyncElasticsearch` instead of `Elasticsearch`. The default client is synchronous.
  Fix: Use `from elasticsearch import AsyncElasticsearch` and ensure your application's event loop is managed correctly. `httpx` is used as the underlying async HTTP client.
• gotcha Certificate verification is enabled by default (`verify_certs=True`). If connecting to a self-signed or improperly configured HTTPS endpoint without a trusted CA certificate, you will encounter SSL errors.
  Fix: For production, ensure your system trusts the CA certificate used by Elasticsearch. For development, you can temporarily disable verification with `verify_certs=False` in the client constructor, but this is highly insecure for production use.
• gotcha When connecting to Elastic Cloud, prefer `cloud_id` and `api_key` for authentication. For self-managed instances, `hosts` and `basic_auth` (username/password) are common. Failing to provide correct authentication will lead to connection errors.
  Fix: Always provide appropriate authentication details. For Elastic Cloud, generate an API Key. For self-managed, ensure the provided username/password has necessary permissions. Avoid hardcoding credentials in code; use environment variables.
• deprecated The `doc_type` parameter is largely removed or ignored in Elasticsearch v8, as documents no longer have types. Using it in client calls may raise warnings or errors.
  Fix: Remove `doc_type` from all client methods (e.g., `index`, `get`, `update`, `delete`). Elasticsearch 8.x operates on a single type per index, simplifying interactions.

Install

• pip install elasticsearch8 Standard Installation
• pip install elasticsearch8[async] Asynchronous Client (HTTPX)

Imports

• Elasticsearch

  from elasticsearch import Elasticsearch

  Despite installing `elasticsearch8`, the package exposes its main client class via the `elasticsearch` namespace for consistency with previous major versions of the client.
• AsyncElasticsearch

  from elasticsearch import AsyncElasticsearch

  Requires `pip install elasticsearch8[async]` for asynchronous support.

Quickstart

This quickstart demonstrates how to connect to Elasticsearch (either Elastic Cloud or self-managed), index a simple document, and perform a basic search. It uses environment variables for secure credential management. Remember to replace placeholder credentials with your actual ones.

import os
from elasticsearch import Elasticsearch

# --- Configuration for connecting to Elasticsearch ---
# For Elastic Cloud (recommended)
CLOUD_ID = os.environ.get("ELASTIC_CLOUD_ID", "")
API_KEY = os.environ.get("ELASTIC_API_KEY", "") # Base64 encoded API Key ID and Key

# For self-managed Elasticsearch
ES_HOST = os.environ.get("ELASTICSEARCH_HOST", "http://localhost:9200")
ES_USERNAME = os.environ.get("ELASTICSEARCH_USERNAME", "elastic")
ES_PASSWORD = os.environ.get("ELASTICSEARCH_PASSWORD", "changeme")

client = None
if CLOUD_ID and API_KEY:
    print("Connecting to Elastic Cloud...")
    client = Elasticsearch(
        cloud_id=CLOUD_ID,
        api_key=API_KEY,
    )
elif ES_HOST:
    print(f"Connecting to self-managed Elasticsearch at {ES_HOST}...")
    client = Elasticsearch(
        hosts=[ES_HOST],
        basic_auth=(ES_USERNAME, ES_PASSWORD),
        verify_certs=False # CAUTION: Only for local dev/testing without proper CA config
    )
else:
    raise ValueError(
        "Elasticsearch connection details not provided. Set ELASTIC_CLOUD_ID/ELASTIC_API_KEY "
        "or ELASTICSEARCH_HOST/ELASTICSEARCH_USERNAME/ELASTICSEARCH_PASSWORD environment variables."
    )

# --- Verify Connection ---
if client.ping():
    print("Successfully connected to Elasticsearch!")
else:
    print("Could not connect to Elasticsearch. Please check your configuration.")
    exit(1)

# --- Index a Document ---
index_name = "my-documents"
doc_id = "1"
document = {
    "author": "John Doe",
    "text": "Elasticsearch is a powerful open-source search and analytics engine.",
    "timestamp": "2024-05-15T12:00:00Z"
}

print(f"Indexing document with ID '{doc_id}' into index '{index_name}'...")
response = client.index(index=index_name, id=doc_id, document=document)
print(f"Indexed document: Result - {response['result']}")

# --- Search for Documents ---
print(f"Searching for documents in index '{index_name}'...")
search_query = {"match": {"text": "search engine"}}
response = client.search(index=index_name, query=search_query)

print(f"Found {response['hits']['total']['value']} hits:")
for hit in response['hits']['hits']:
    print(f"  ID: {hit['_id']}, Source: {hit['_source']}")

# --- Clean Up (Optional) ---
# client.indices.delete(index=index_name, ignore=[400, 404])
# print(f"Index '{index_name}' deleted (if it existed).")

client.close()
print("Client connection closed.")