Any-LLM SDK

1.13.0 verified Fri Apr 17 auth: no python

The Any-LLM SDK provides a unified Python interface for interacting with various Large Language Model (LLM) providers through the Any-LLM gateway. It aims to abstract away provider-specific complexities, offering a consistent API for chat completions, embeddings, and more. The current version is 1.13.0, and it maintains a rapid release cadence with frequent updates and improvements.

pip install any-llm-sdk

Common errors

error ModuleNotFoundError: No module named 'any_llm_sdk' ↓

cause The `any-llm-sdk` package is not installed in your Python environment.

fix

Run: pip install any-llm-sdk

error any_llm_sdk.exceptions.AuthenticationError: Missing API key or invalid credentials. ↓

cause The `ANY_LLM_GATEWAY_API_KEY` environment variable is not set or contains an invalid key, preventing authentication with the Any-LLM gateway.

fix

Set the ANY_LLM_GATEWAY_API_KEY environment variable to your valid API key. Example: export ANY_LLM_GATEWAY_API_KEY='your_api_key_here'.

error any_llm_sdk.exceptions.ConfigurationError: Missing gateway URL. Set ANY_LLM_GATEWAY_URL environment variable. ↓

cause The `ANY_LLM_GATEWAY_URL` environment variable is not set, preventing the SDK from knowing which gateway to connect to.

fix

Set the ANY_LLM_GATEWAY_URL environment variable to your Any-LLM Gateway endpoint. Example: export ANY_LLM_GATEWAY_URL='https://gateway.any-llm.ai'.

error ImportError: cannot import name 'Gateway' from 'any_llm_sdk.client' (or similar subpath) ↓

cause You are trying to import `Gateway` from an incorrect sub-module path.

fix

The Gateway class is a top-level import. Use from any_llm_sdk import Gateway.

Warnings

breaking As of v1.13.0, the `anthropic` SDK is now a required dependency, similar to `openai`. Users upgrading or installing might encounter new dependency conflicts if they weren't explicitly managing `anthropic` before. ↓

fix Ensure `anthropic` is compatible with your environment or explicitly install `pip install any-llm-sdk anthropic`.

breaking For Anthropic provider users, v1.9.0 introduced a breaking change regarding adaptive thinking. Older models now require manual specification of the 'thinking budget' parameter, which is no longer set by default. ↓

fix Review Anthropic provider calls and explicitly add the 'thinking_budget' parameter if you were relying on the default behavior for older models.

gotcha The SDK relies on `ANY_LLM_GATEWAY_URL` and `ANY_LLM_GATEWAY_API_KEY` environment variables. If these are not set, the `Gateway` client will fail to initialize or authenticate, leading to `ConfigurationError` or `AuthenticationError`. ↓

fix Always set `ANY_LLM_GATEWAY_URL` and `ANY_LLM_GATEWAY_API_KEY` in your environment before running applications that use the SDK.

gotcha While the SDK abstracts providers, specific models (e.g., 'gpt-4o', 'claude-3-opus-20240229') must be available via your configured Any-LLM Gateway. Using a non-existent or unsupported model will result in an API error. ↓

fix Verify that the model name you are using (`model='...'`) is correctly configured and available through your Any-LLM Gateway instance.

Imports

Gateway
```
from any_llm_sdk import Gateway
```
messages
wrong
```
from any_llm_sdk.messages import Message
```
correct
```
from any_llm_sdk.types import messages
```
Message types are re-exported via 'any_llm_sdk.types.messages' and often directly from 'anthropic.types'.

Quickstart

Initializes the Any-LLM Gateway client using environment variables for the gateway URL and API key, then sends a simple chat completion request to a specified model. The `messages` types from `any_llm_sdk.types` (re-exporting Anthropic types) are used for constructing the payload.

import os
from any_llm_sdk import Gateway
from any_llm_sdk.types import messages

# Ensure these environment variables are set:
# os.environ['ANY_LLM_GATEWAY_URL'] = 'YOUR_GATEWAY_URL'
# os.environ['ANY_LLM_GATEWAY_API_KEY'] = 'YOUR_API_KEY'

try:
    gateway = Gateway(
        base_url=os.environ.get('ANY_LLM_GATEWAY_URL', 'https://gateway.any-llm.ai'),
        api_key=os.environ.get('ANY_LLM_GATEWAY_API_KEY', '')
    )

    messages_payload = [
        messages.UserMessage(content='Hello, what is the capital of France?')
    ]

    chat_completion = gateway.chat.completions.create(
        model='gpt-4o',
        messages=messages_payload,
        max_tokens=100
    )
    print(chat_completion.choices[0].message.content)

except Exception as e:
    print(f"An error occurred: {e}")
    print("Please ensure ANY_LLM_GATEWAY_URL and ANY_LLM_GATEWAY_API_KEY environment variables are set.")