Llama Stack Client Python Library

Warnings

• breaking The `agents` API was renamed to `responses` API in `v0.7.0-alpha.1`. Code using `client.agents` will no longer work.
  Fix: Update all calls from `client.agents.*` to `client.responses.*`.
• breaking Breaking changes were introduced to the `GET /chat/completions/{completion_id}` and `/files/{file_id}` endpoints in `v0.7.0-alpha.1` to eliminate conformance issues.
  Fix: Review and update code interacting with `client.chat.completions.retrieve()` and `client.files.retrieve()` methods.
• breaking Consistency improvements were made to post-training API endpoints in `v0.6.1-alpha.1`, which may involve API surface changes.
  Fix: Review documentation and update code using post-training APIs accordingly.
• gotcha The `llama-stack-client` library is currently in alpha (`--pre`) release status. This implies that breaking changes and API instability are frequent and expected across minor versions. It is recommended to pin exact versions.
  Fix: Pin your `llama-stack-client` version in `requirements.txt` or `pyproject.toml` (e.g., `llama-stack-client==0.7.2`) to avoid unexpected breakage.
• gotcha This library is a client for the Llama Stack API. It requires a separate Llama Stack server instance to be running and accessible. This client library does not include the server itself.
  Fix: Ensure a Llama Stack server is deployed and running (e.g., via Docker) before attempting to use this client.
• gotcha For authentication, the client primarily relies on the `LLAMA_STACK_CLIENT_API_KEY` environment variable. If this is not set, API calls may fail or use a dummy key.
  Fix: Set the `LLAMA_STACK_CLIENT_API_KEY` environment variable with your actual API key before initializing the client, or pass it explicitly during client instantiation.
• gotcha The Responses API, a central feature for server-side agentic orchestration, is still under active development. While usable, some parts of its OpenAI-compatible implementation may still be unimplemented.
  Fix: Refer to the official Llama Stack documentation for the most up-to-date information on the Responses API capabilities and known limitations.

Install

• pip install --pre llama-stack-client Install pre-release version

Imports

• LlamaStackClient

  from llama_stack_client import LlamaStackClient

Quickstart

This quickstart demonstrates how to initialize the LlamaStackClient and perform a basic model listing and inference request. It assumes a Llama Stack server is already running and accessible, for instance, locally via Docker.

import os
from llama_stack_client import LlamaStackClient

# Ensure a Llama Stack server is running, e.g., locally at http://localhost:8321.
# Authentication typically uses the LLAMA_STACK_CLIENT_API_KEY environment variable.
# Example: export LLAMA_STACK_CLIENT_API_KEY="your_api_key"
# You can also set the base URL via LLAMA_STACK_BASE_URL environment variable.
client = LlamaStackClient(
    base_url=os.environ.get("LLAMA_STACK_BASE_URL", "http://localhost:8321"),
    api_key=os.environ.get("LLAMA_STACK_CLIENT_API_KEY", "dummy_key_for_testing_if_not_set"),
)

try:
    # List available models
    models = client.models.list()
    print("Available models:", [model.id for model in models.data])

    # Perform simple inference using the Responses API
    if models.data:
        response = client.responses.create(
            model=models.data[0].id, # Use the first available model
            input="Write a haiku about coding.",
        )
        print("\nHaiku from Llama Stack:", response.output_text)
    else:
        print("\nNo models found on the Llama Stack server.")

except Exception as e:
    print(f"An error occurred: {e}")
    print("Please ensure your Llama Stack server is running and accessible (e.g., via Docker), and the API key is set correctly.")