SMG gRPC Proto Definitions

0.4.6 · active · verified Wed Apr 15

smg-grpc-proto provides the Protocol Buffer definitions and generated Python gRPC stubs for the Shepherd Model Gateway (SMG). These definitions enable communication with SMG, a universal gateway supporting various LLM inference engines like SGLang, vLLM, and TRT-LLM. While the main SMG project undergoes rapid development (currently v1.4.x), this library provides the stable API surface for gRPC interaction. The current version is 0.4.6, with updates typically tied to major SMG feature releases.

Warnings

gotcha This package (`smg-grpc-proto`) provides only the Protocol Buffer definitions and generated Python gRPC client/server stubs. It does not include the full Shepherd Model Gateway client SDK or the gateway server itself. For a higher-level client, consider `smg-client-sdk`.
Fix: If you need a full client library with convenience methods, use `pip install smg-client-sdk`. If you are building a custom gRPC client/server, this package is appropriate.
breaking Significant changes to the SMG gateway's API (e.g., new services, modified message structures for features like Multimodal or Messages API) often lead to updates in `smg-grpc-proto`. Using an older `smg-grpc-proto` version with a newer SMG gateway (or vice versa) can result in gRPC errors like `UNIMPLEMENTED`, `INVALID_ARGUMENT`, or malformed responses.
Fix: Always align the `smg-grpc-proto` version with the expected version of the SMG gateway you are interacting with. Consult the SMG gateway's release notes for compatible proto versions. Upgrade `smg-grpc-proto` to the latest version to ensure compatibility with the most recent SMG gateway features and fixes: `pip install --upgrade smg-grpc-proto`.
gotcha The generated Python package for the proto definitions is `smg_proto`, not `smg_grpc_proto`. Direct imports should use `from smg_proto import ...`.
Fix: When importing, always refer to the `smg_proto` module, e.g., `from smg_proto import smg_pb2, smg_pb2_grpc`.

Install

pip install smg-grpc-proto Install latest version

Imports

smg_pb2
```
from smg_proto import smg_pb2
```
Generated protobuf messages are typically under the 'smg_proto' package, not the 'smg_grpc_proto' top-level package name.
smg_pb2_grpc
```
from smg_proto import smg_pb2_grpc
```
Generated gRPC service stubs are typically under the 'smg_proto' package, matching the message definitions.
CompletionRequest
```
from smg_proto.smg_pb2 import CompletionRequest
```
Specific message types are imported directly from the smg_pb2 module.
CompletionServiceStub
```
from smg_proto.smg_pb2_grpc import CompletionServiceStub
```
Specific service stubs are imported directly from the smg_pb2_grpc module.

Quickstart

This quickstart demonstrates how to use the generated gRPC stubs from `smg-grpc-proto` to make a `CompletionRequest` to a running Shepherd Model Gateway. It establishes an insecure channel, creates a stub, and sends a request. Remember to replace '<your_model_name>' with a model available on your SMG instance. An SMG gateway must be running and accessible at `SMG_GRPC_TARGET` (defaults to `localhost:8000`) for this example to work.

import grpc
from smg_proto import smg_pb2, smg_pb2_grpc
import os

# NOTE: This example requires a running SMG gateway to connect to.
# For a local gateway, use 'localhost:8000' (default).
SMG_GRPC_TARGET = os.environ.get('SMG_GRPC_TARGET', 'localhost:8000')

def run_completion_client():
    try:
        with grpc.insecure_channel(SMG_GRPC_TARGET) as channel:
            stub = smg_pb2_grpc.CompletionServiceStub(channel)
            request = smg_pb2.CompletionRequest(
                model='<your_model_name>',  # e.g., 'mistral-7b-instruct-v0.2'
                prompt='Hello, world! What is your name?',
                max_tokens=50,
                temperature=0.7
            )
            print(f"Sending completion request to {SMG_GRPC_TARGET}...")
            response = stub.Complete(request)
            print("Completion Response:")
            print(f"Text: {response.text}")
            if response.finish_reason:
                print(f"Finish Reason: {response.finish_reason}")
            print(f"Prompt Tokens: {response.prompt_tokens}")
            print(f"Completion Tokens: {response.completion_tokens}")
    except grpc.RpcError as e:
        print(f"gRPC Error: {e.code()} - {e.details()}")
    except Exception as e:
        print(f"An unexpected error occurred: {e}")

if __name__ == '__main__':
    run_completion_client()

view raw JSON →