OpenSearch Protobufs Python Client

1.3.0 · active · verified Sat Mar 28

The `opensearch-protobufs` library provides Protocol Buffer definitions and generated Python code for interacting with OpenSearch's gRPC APIs. It serves as a downstream consumer of the `opensearch-api-specification`, packaging pre-generated code to simplify client-server communication for various OpenSearch projects. The library is actively maintained with frequent releases, offering a high-performance alternative to traditional REST APIs through binary serialization and gRPC.

Warnings

breaking The gRPC Search API is currently experimental, and its protobuf structure is subject to changes in future OpenSearch versions. This means that API structures and generated code might change in backward-incompatible ways between minor or major releases, requiring client-side code updates.
Fix: Always consult the `opensearch-protobufs` GitHub repository and OpenSearch documentation's gRPC section, especially the compatibility matrix (`COMPATIBILITY.md`), when upgrading OpenSearch or `opensearch-protobufs` versions. Pin your `opensearch-protobufs` version carefully.
gotcha When sending document data (e.g., for Bulk API operations), documents must be provided as Base64 encoded bytes within the gRPC request. Directly providing plain JSON strings or Python dictionaries will result in errors.
Fix: Ensure all document content intended for fields like `doc` or `source` is first serialized (e.g., to JSON string), then encoded to bytes, and finally Base64 encoded before being assigned to the protobuf field. For example: `base64.b64encode(json.dumps({'field': 'value'}).encode('utf-8'))`.
gotcha This library (`opensearch-protobufs`) only provides the *generated Python code* from `.proto` files. To actually make gRPC calls, you must explicitly install and use the `grpcio` library. If you need to generate protobuf code yourself from `.proto` definitions, `grpcio-tools` is also required.
Fix: Install `grpcio` alongside `opensearch-protobufs`: `pip install opensearch-protobufs grpcio`. If generating code, also install `grpcio-tools`: `pip install opensearch-protobufs grpcio grpcio-tools`.
gotcha Compatibility between `opensearch-protobufs` client versions and OpenSearch server versions is crucial. Using an incompatible client-server pair can lead to unexpected behavior or gRPC errors.
Fix: Refer to the `COMPATIBILITY.md` file in the `opensearch-protobufs` GitHub repository to determine the correct client version for your OpenSearch cluster version. Upgrade both components in sync as recommended.

Install

pip install opensearch-protobufs Install stable version

Imports

SearchRequest
```
from opensearch.protobufs.schemas import SearchRequest
```
Commonly used for creating search requests to the OpenSearch gRPC API.
BulkRequest
```
from opensearch.protobufs.schemas import BulkRequest
```
Used for performing multiple document operations (index, update, delete) in a single gRPC call.
SearchServiceStub
```
from opensearch.protobufs.services import SearchServiceStub
```
The gRPC service stub for interacting with the OpenSearch Search API.
DocumentServiceStub
```
from opensearch.protobufs.services import DocumentServiceStub
```
The gRPC service stub for interacting with the OpenSearch Document (Bulk) API.

Quickstart

This quickstart demonstrates how to create a simple `SearchRequest` using the generated protobuf classes, establish an insecure gRPC channel, and send the request to an OpenSearch gRPC endpoint. It then processes the `SearchResponse`. Remember to configure your `OPENSEARCH_GRPC_TARGET` and use secure channels in production environments. An OpenSearch instance with gRPC transport enabled (e.g., in `opensearch.yml` with `aux.transport.types: [transport-grpc]`) is required for this code to function against a live service.

import grpc
import os
import base64
from opensearch.protobufs.schemas import SearchRequest, Query, MatchQuery
from opensearch.protobufs.services import SearchServiceStub

# Configure your OpenSearch gRPC endpoint
# In a real application, you'd replace this with your actual OpenSearch gRPC host and port
OPENSEARCH_GRPC_TARGET = os.environ.get('OPENSEARCH_GRPC_TARGET', 'localhost:9200') # Example

def run_search_query():
    try:
        # Establish an insecure gRPC channel (use secure channels for production!)
        with grpc.insecure_channel(OPENSEARCH_GRPC_TARGET) as channel:
            stub = SearchServiceStub(channel)

            # Create a simple match query
            match_query = MatchQuery(field='title', query='OpenSearch')
            query = Query(match=match_query)

            # Create the search request
            search_request = SearchRequest(
                query=query,
                size=5
            )

            print(f"Sending SearchRequest to {OPENSEARCH_GRPC_TARGET}:")
            print(search_request)

            # Make the gRPC call
            response = stub.Search(search_request)

            print("\nSearch Response:")
            for hit in response.hits.hits:
                # _source is returned as bytes, decode if it's JSON-like
                source_data = hit.source.value.decode('utf-8') if hit.source.value else '{}'
                print(f"  ID: {hit.id}, Score: {hit.score}, Source: {source_data}")

    except grpc.RpcError as e:
        print(f"gRPC Error: {e.code()} - {e.details()}")
    except Exception as e:
        print(f"An unexpected error occurred: {e}")

if __name__ == '__main__':
    print("Note: This quickstart requires a running OpenSearch instance with gRPC enabled.")
    print(f"Attempting to connect to gRPC target: {OPENSEARCH_GRPC_TARGET}\n")
    run_search_query()

view raw JSON →