Any-LLM SDK

1.13.0 · active · verified Fri Apr 17

The Any-LLM SDK provides a unified Python interface for interacting with various Large Language Model (LLM) providers through the Any-LLM gateway. It aims to abstract away provider-specific complexities, offering a consistent API for chat completions, embeddings, and more. The current version is 1.13.0, and it maintains a rapid release cadence with frequent updates and improvements.

Common errors

```
ModuleNotFoundError: No module named 'any_llm_sdk'
```
cause The `any-llm-sdk` package is not installed in your Python environment.

fix Run: `pip install any-llm-sdk`
```
any_llm_sdk.exceptions.AuthenticationError: Missing API key or invalid credentials.
```
cause The `ANY_LLM_GATEWAY_API_KEY` environment variable is not set or contains an invalid key, preventing authentication with the Any-LLM gateway.

fix Set the `ANY_LLM_GATEWAY_API_KEY` environment variable to your valid API key. Example: `export ANY_LLM_GATEWAY_API_KEY='your_api_key_here'`.
```
any_llm_sdk.exceptions.ConfigurationError: Missing gateway URL. Set ANY_LLM_GATEWAY_URL environment variable.
```
cause The `ANY_LLM_GATEWAY_URL` environment variable is not set, preventing the SDK from knowing which gateway to connect to.

fix Set the `ANY_LLM_GATEWAY_URL` environment variable to your Any-LLM Gateway endpoint. Example: `export ANY_LLM_GATEWAY_URL='https://gateway.any-llm.ai'`.
```
ImportError: cannot import name 'Gateway' from 'any_llm_sdk.client' (or similar subpath)
```
cause You are trying to import `Gateway` from an incorrect sub-module path.

fix The `Gateway` class is a top-level import. Use `from any_llm_sdk import Gateway`.

Warnings

breaking As of v1.13.0, the `anthropic` SDK is now a required dependency, similar to `openai`. Users upgrading or installing might encounter new dependency conflicts if they weren't explicitly managing `anthropic` before.
Fix: Ensure `anthropic` is compatible with your environment or explicitly install `pip install any-llm-sdk anthropic`.
breaking For Anthropic provider users, v1.9.0 introduced a breaking change regarding adaptive thinking. Older models now require manual specification of the 'thinking budget' parameter, which is no longer set by default.
Fix: Review Anthropic provider calls and explicitly add the 'thinking_budget' parameter if you were relying on the default behavior for older models.
gotcha The SDK relies on `ANY_LLM_GATEWAY_URL` and `ANY_LLM_GATEWAY_API_KEY` environment variables. If these are not set, the `Gateway` client will fail to initialize or authenticate, leading to `ConfigurationError` or `AuthenticationError`.
Fix: Always set `ANY_LLM_GATEWAY_URL` and `ANY_LLM_GATEWAY_API_KEY` in your environment before running applications that use the SDK.
gotcha While the SDK abstracts providers, specific models (e.g., 'gpt-4o', 'claude-3-opus-20240229') must be available via your configured Any-LLM Gateway. Using a non-existent or unsupported model will result in an API error.
Fix: Verify that the model name you are using (`model='...'`) is correctly configured and available through your Any-LLM Gateway instance.

Install

pip install any-llm-sdk Install core SDK

Imports

Gateway
```
from any_llm_sdk import Gateway
```
messages
wrong:
```
from any_llm_sdk.messages import Message
```
correct:
```
from any_llm_sdk.types import messages
```
Message types are re-exported via 'any_llm_sdk.types.messages' and often directly from 'anthropic.types'.

Quickstart

Initializes the Any-LLM Gateway client using environment variables for the gateway URL and API key, then sends a simple chat completion request to a specified model. The `messages` types from `any_llm_sdk.types` (re-exporting Anthropic types) are used for constructing the payload.

import os
from any_llm_sdk import Gateway
from any_llm_sdk.types import messages

# Ensure these environment variables are set:
# os.environ['ANY_LLM_GATEWAY_URL'] = 'YOUR_GATEWAY_URL'
# os.environ['ANY_LLM_GATEWAY_API_KEY'] = 'YOUR_API_KEY'

try:
    gateway = Gateway(
        base_url=os.environ.get('ANY_LLM_GATEWAY_URL', 'https://gateway.any-llm.ai'),
        api_key=os.environ.get('ANY_LLM_GATEWAY_API_KEY', '')
    )

    messages_payload = [
        messages.UserMessage(content='Hello, what is the capital of France?')
    ]

    chat_completion = gateway.chat.completions.create(
        model='gpt-4o',
        messages=messages_payload,
        max_tokens=100
    )
    print(chat_completion.choices[0].message.content)

except Exception as e:
    print(f"An error occurred: {e}")
    print("Please ensure ANY_LLM_GATEWAY_URL and ANY_LLM_GATEWAY_API_KEY environment variables are set.")

view raw JSON →