LiteLLM

Warnings

• breaking LiteLLM releases multiple times per week. Minor and patch versions introduce behavioral changes — response format normalization, new provider routing logic, cost calculation updates — without necessarily bumping the major version. Unpinned installs in production can silently change behavior overnight.
  Fix: Pin to a specific version in production: pip install litellm==1.81.15. Use -stable tagged Docker images for the proxy.
• breaking A known OOM (Out of Memory) issue on Kubernetes was introduced in a September 2025 release and caused proxy startup failures. The issue was patched in a subsequent release but affected users who were on that specific release without pinning.
  Fix: Always pin litellm versions. Check the release notes at docs.litellm.ai/release_notes before upgrading in production proxy deployments.
• gotcha Model string format matters. 'gpt-4o' (no prefix) works but LiteLLM must infer the provider. 'openai/gpt-4o' (with prefix) is explicit and more reliable, especially when multiple providers are configured. Ambiguous model names can be silently routed to the wrong provider.
  Fix: Always use 'provider/model-name' format: 'anthropic/claude-sonnet-4-20250514', 'openai/gpt-4o', 'bedrock/amazon.titan-embed-text-v1'.
• gotcha pip install litellm pulls in a large dependency tree (openai, anthropic, httpx, pydantic, tiktoken, and more). Cold install in CI can take 2-4 minutes. Docker images are significantly faster for repeated deploys.
  Fix: Use the official BerriAI Docker image for proxy deployments. For SDK use in CI, cache pip dependencies between runs.
• gotcha Cost tracking (completion_cost(), token_counter()) depends on LiteLLM's internal pricing database. Prices for new or custom models may lag behind actual provider pricing by days to weeks. Do not rely on LiteLLM cost estimates for billing-critical applications without cross-referencing provider invoices.
  Fix: Treat LiteLLM cost estimates as approximations. For billing-critical use, validate against provider usage dashboards monthly.
• gotcha litellm[proxy] requires additional system dependencies (prisma CLI for DB migrations) that are not installed automatically. Running litellm --use_prisma_migrate without prisma installed raises a confusing error.
  Fix: Install litellm-proxy-extras separately: pip install litellm-proxy-extras. Or use the Docker image which has all dependencies pre-installed.

Install

• pip install litellm Core SDK only
• pip install 'litellm[proxy]' With proxy/gateway server dependencies
• pip install 'litellm[caching]' With Redis/in-memory caching support
• litellm --model gpt-4o Run as proxy server (after pip install litellm[proxy])

Imports

• completion

  from litellm import completion

  Primary function is completion(), not Completion(). Mirrors openai.chat.completions.create() interface.
• acompletion

  from litellm import acompletion

  Async variant. Use acompletion() in async contexts, not completion() with asyncio.run().

Quickstart

Model strings use 'provider/model-name' format (e.g. 'anthropic/claude-sonnet-4-20250514', 'openai/gpt-4o'). Without the provider prefix, LiteLLM infers the provider — but explicit prefixes are safer. API keys are read from environment variables named per-provider.

import os
from litellm import completion

# Set provider API keys as env vars
os.environ["OPENAI_API_KEY"] = "..."
os.environ["ANTHROPIC_API_KEY"] = "..."

# OpenAI
response = completion(
    model="openai/gpt-4o",
    messages=[{"role": "user", "content": "Hello!"}]
)
print(response.choices[0].message.content)

# Anthropic — same interface, different model string
response = completion(
    model="anthropic/claude-sonnet-4-20250514",
    messages=[{"role": "user", "content": "Hello!"}]
)

# Streaming
for chunk in completion(
    model="openai/gpt-4o",
    messages=[{"role": "user", "content": "Count to 5"}],
    stream=True
):
    print(chunk.choices[0].delta.content or "", end="")

# Async
import asyncio
from litellm import acompletion

async def main():
    response = await acompletion(
        model="anthropic/claude-sonnet-4-20250514",
        messages=[{"role": "user", "content": "Hello!"}]
    )
    print(response.choices[0].message.content)

asyncio.run(main())

# Cost tracking
from litellm import completion_cost
cost = completion_cost(completion_response=response)
print(f"Cost: ${cost}")