{"id":42,"library":"litellm","title":"LiteLLM","description":"Unified Python SDK and proxy gateway for calling 100+ LLM APIs in OpenAI-compatible format. Single interface for OpenAI, Anthropic, Bedrock, VertexAI, Groq, Mistral, Cohere, HuggingFace, vLLM, and more. Supports /chat/completions, /embeddings, /images, /audio, /rerank, streaming, async, cost tracking, fallbacks, and load balancing. Also ships as a deployable proxy server (AI Gateway) with budget controls, virtual keys, and logging. Releases multiple times per week — version numbers are high (v1.81+) due to frequent patch releases. NOT related to the 'litellm' PyPI stub that existed before BerriAI claimed the name.","status":"active","version":"1.81.15","language":"python","source_language":"en","source_url":"https://github.com/BerriAI/litellm","tags":["litellm","llm-gateway","openai-compatible","anthropic","bedrock","vertexai","multi-provider","proxy","cost-tracking","streaming"],"install":[{"cmd":"pip install litellm","lang":"bash","label":"Core SDK only"},{"cmd":"pip install 'litellm[proxy]'","lang":"bash","label":"With proxy/gateway server dependencies"},{"cmd":"pip install 'litellm[caching]'","lang":"bash","label":"With Redis/in-memory caching support"},{"cmd":"litellm --model gpt-4o","lang":"bash","label":"Run as proxy server (after pip install litellm[proxy])"}],"dependencies":[{"reason":"Required. LiteLLM builds on the OpenAI SDK for request formatting and response parsing.","package":"openai","optional":false},{"reason":"Required. Used for async HTTP requests.","package":"httpx","optional":false},{"reason":"Required. Used for request/response validation.","package":"pydantic","optional":false},{"reason":"Optional. Required for AWS Bedrock provider.","package":"boto3","optional":true},{"reason":"Optional. Required for Vertex AI provider.","package":"google-cloud-aiplatform","optional":true}],"imports":[{"note":"Primary function is completion(), not Completion(). Mirrors openai.chat.completions.create() interface.","wrong":"import litellm; litellm.Completion()","symbol":"completion","correct":"from litellm import completion"},{"note":"Async variant. Use acompletion() in async contexts, not completion() with asyncio.run().","symbol":"acompletion","correct":"from litellm import acompletion"}],"quickstart":{"code":"import os\nfrom litellm import completion\n\n# Set provider API keys as env vars\nos.environ[\"OPENAI_API_KEY\"] = \"...\"\nos.environ[\"ANTHROPIC_API_KEY\"] = \"...\"\n\n# OpenAI\nresponse = completion(\n    model=\"openai/gpt-4o\",\n    messages=[{\"role\": \"user\", \"content\": \"Hello!\"}]\n)\nprint(response.choices[0].message.content)\n\n# Anthropic — same interface, different model string\nresponse = completion(\n    model=\"anthropic/claude-sonnet-4-20250514\",\n    messages=[{\"role\": \"user\", \"content\": \"Hello!\"}]\n)\n\n# Streaming\nfor chunk in completion(\n    model=\"openai/gpt-4o\",\n    messages=[{\"role\": \"user\", \"content\": \"Count to 5\"}],\n    stream=True\n):\n    print(chunk.choices[0].delta.content or \"\", end=\"\")\n\n# Async\nimport asyncio\nfrom litellm import acompletion\n\nasync def main():\n    response = await acompletion(\n        model=\"anthropic/claude-sonnet-4-20250514\",\n        messages=[{\"role\": \"user\", \"content\": \"Hello!\"}]\n    )\n    print(response.choices[0].message.content)\n\nasyncio.run(main())\n\n# Cost tracking\nfrom litellm import completion_cost\ncost = completion_cost(completion_response=response)\nprint(f\"Cost: ${cost}\")","lang":"python","description":"Model strings use 'provider/model-name' format (e.g. 'anthropic/claude-sonnet-4-20250514', 'openai/gpt-4o'). Without the provider prefix, LiteLLM infers the provider — but explicit prefixes are safer. API keys are read from environment variables named per-provider."},"warnings":[{"fix":"Pin to a specific version in production: pip install litellm==1.81.15. Use -stable tagged Docker images for the proxy.","message":"LiteLLM releases multiple times per week. Minor and patch versions introduce behavioral changes — response format normalization, new provider routing logic, cost calculation updates — without necessarily bumping the major version. Unpinned installs in production can silently change behavior overnight.","severity":"breaking","affected_versions":"all"},{"fix":"Always pin litellm versions. Check the release notes at docs.litellm.ai/release_notes before upgrading in production proxy deployments.","message":"A known OOM (Out of Memory) issue on Kubernetes was introduced in a September 2025 release and caused proxy startup failures. The issue was patched in a subsequent release but affected users who were on that specific release without pinning.","severity":"breaking","affected_versions":"v1.x (Sep 2025 range)"},{"fix":"Always use 'provider/model-name' format: 'anthropic/claude-sonnet-4-20250514', 'openai/gpt-4o', 'bedrock/amazon.titan-embed-text-v1'.","message":"Model string format matters. 'gpt-4o' (no prefix) works but LiteLLM must infer the provider. 'openai/gpt-4o' (with prefix) is explicit and more reliable, especially when multiple providers are configured. Ambiguous model names can be silently routed to the wrong provider.","severity":"gotcha","affected_versions":"all"},{"fix":"Use the official BerriAI Docker image for proxy deployments. For SDK use in CI, cache pip dependencies between runs.","message":"pip install litellm pulls in a large dependency tree (openai, anthropic, httpx, pydantic, tiktoken, and more). Cold install in CI can take 2-4 minutes. Docker images are significantly faster for repeated deploys.","severity":"gotcha","affected_versions":"all"},{"fix":"Treat LiteLLM cost estimates as approximations. For billing-critical use, validate against provider usage dashboards monthly.","message":"Cost tracking (completion_cost(), token_counter()) depends on LiteLLM's internal pricing database. Prices for new or custom models may lag behind actual provider pricing by days to weeks. Do not rely on LiteLLM cost estimates for billing-critical applications without cross-referencing provider invoices.","severity":"gotcha","affected_versions":"all"},{"fix":"Install litellm-proxy-extras separately: pip install litellm-proxy-extras. Or use the Docker image which has all dependencies pre-installed.","message":"litellm[proxy] requires additional system dependencies (prisma CLI for DB migrations) that are not installed automatically. Running litellm --use_prisma_migrate without prisma installed raises a confusing error.","severity":"gotcha","affected_versions":"all"},{"fix":"Before installing `litellm[proxy]`, ensure the Rust `cargo` toolchain is available in your environment. For Alpine, install it using `apk add rust cargo`. For other systems, refer to Rust installation guides. Alternatively, use the official BerriAI Docker images which come with all necessary build dependencies pre-installed.","message":"Installing `litellm[proxy]` in minimal environments like Alpine can fail if the Rust `cargo` build toolchain is not pre-installed. A dependency, `pyroscope-io`, requires `cargo` to build its wheel, leading to an `[Errno 2] No such file or directory: 'cargo'` error.","severity":"breaking","affected_versions":"all"}],"env_vars":null,"last_verified":"2026-05-12T05:33:06.321Z","next_check":"2026-05-28T00:00:00.000Z","problems":[{"fix":"Install the library using pip: `pip install litellm` or `pip install 'litellm[proxy]'` if you need the proxy server components.","cause":"The `litellm` library has not been installed in your Python environment or there is an issue with your Python path.","error":"ModuleNotFoundError: No module named 'litellm'"},{"fix":"Set the API key as an environment variable (e.g., `export OPENAI_API_KEY=\"sk-...\"` for OpenAI) or pass it directly to the `litellm.completion` function: `litellm.completion(model='gpt-3.5-turbo', messages=messages, api_key='sk-YOUR_API_KEY')`.","cause":"The API key for the specified LLM provider (e.g., OpenAI, Anthropic, etc.) is missing or incorrectly configured, either as an environment variable or passed directly to the `completion` call.","error":"litellm.exceptions.AuthenticationError: AuthenticationError: OpenAIException - The api_key client option must be set either by passing api_key to the client or by setting the OPENAI_API_KEY environment variable"},{"fix":"Ensure your model string explicitly includes the provider prefix (e.g., 'openai/gpt-3.5-turbo', 'anthropic/claude-2', 'google/gemini-pro') or set the `litellm.set_model_alias` if using custom mappings.","cause":"LiteLLM cannot infer the target LLM provider from the `model` string you provided, or the provider is not explicitly set.","error":"litellm.BadRequestError: LLM Provider NOT provided. Pass in the LLM provider you are trying to call."},{"fix":"Ensure `messages` is a list of dictionaries, where each dictionary has at least a 'role' (e.g., 'user', 'assistant', 'system') and 'content' field. Example: `litellm.completion(model='gpt-4', messages=[{'role': 'user', 'content': 'Hello!'}])`.","cause":"The `messages` parameter, which holds the conversation history for chat completions, is either missing from your `litellm.completion` call or is not a correctly formatted list of message dictionaries.","error":"litellm.BadRequestError: 'messages' is a required parameter"},{"fix":"Ensure that after a message with `role: 'tool'`, the next message has `role: 'assistant'` to acknowledge or act upon the tool's output, before another `role: 'user'` message is sent. The typical flow is `user` -> `assistant` -> `tool` -> `assistant` -> `user`.","cause":"This error occurs when the sequence of 'role' in your `messages` list does not follow the expected conversational turn-taking, especially after a 'tool' message. The model expects an 'assistant' response after a 'tool' output before a new 'user' message.","error":"litellm.BadRequestError: OpenAIException - Unexpected role 'user' after role 'tool' ... Invalid request parameters"}],"ecosystem":"pypi","meta_description":null,"install_score":85,"install_tag":"verified","quickstart_score":80,"quickstart_tag":"verified","pypi_latest":null,"install_checks":{"last_tested":"2026-05-12","tag":"verified","tag_description":"installs cleanly on critical runtimes, fast import, recently tested","results":[{"runtime":"python:3.10-alpine","python_version":"3.10","os_libc":"alpine (musl)","variant":"caching","exit_code":0,"wheel_type":null,"failure_reason":null,"install_time_s":null,"import_time_s":7.8,"mem_mb":88.9,"disk_size":"211.3M"},{"runtime":"python:3.10-alpine","python_version":"3.10","os_libc":"alpine (musl)","variant":"proxy","exit_code":1,"wheel_type":null,"failure_reason":null,"install_time_s":null,"import_time_s":null,"mem_mb":null,"disk_size":null},{"runtime":"python:3.10-alpine","python_version":"3.10","os_libc":"alpine (musl)","variant":"default","exit_code":0,"wheel_type":null,"failure_reason":null,"install_time_s":null,"import_time_s":7.74,"mem_mb":88.9,"disk_size":"210.9M"},{"runtime":"python:3.10-slim","python_version":"3.10","os_libc":"slim (glibc)","variant":"caching","exit_code":0,"wheel_type":null,"failure_reason":null,"install_time_s":null,"import_time_s":5.63,"mem_mb":88.9,"disk_size":"194M"},{"runtime":"python:3.10-slim","python_version":"3.10","os_libc":"slim (glibc)","variant":"proxy","exit_code":0,"wheel_type":null,"failure_reason":null,"install_time_s":null,"import_time_s":6.8,"mem_mb":94,"disk_size":"474M"},{"runtime":"python:3.10-slim","python_version":"3.10","os_libc":"slim (glibc)","variant":"default","exit_code":0,"wheel_type":null,"failure_reason":null,"install_time_s":null,"import_time_s":5.63,"mem_mb":88.9,"disk_size":"194M"},{"runtime":"python:3.11-alpine","python_version":"3.11","os_libc":"alpine (musl)","variant":"caching","exit_code":0,"wheel_type":null,"failure_reason":null,"install_time_s":null,"import_time_s":10.07,"mem_mb":91.6,"disk_size":"228.0M"},{"runtime":"python:3.11-alpine","python_version":"3.11","os_libc":"alpine (musl)","variant":"proxy","exit_code":1,"wheel_type":null,"failure_reason":null,"install_time_s":null,"import_time_s":null,"mem_mb":null,"disk_size":null},{"runtime":"python:3.11-alpine","python_version":"3.11","os_libc":"alpine (musl)","variant":"default","exit_code":0,"wheel_type":null,"failure_reason":null,"install_time_s":null,"import_time_s":10.02,"mem_mb":91.6,"disk_size":"227.6M"},{"runtime":"python:3.11-slim","python_version":"3.11","os_libc":"slim (glibc)","variant":"caching","exit_code":0,"wheel_type":null,"failure_reason":null,"install_time_s":null,"import_time_s":8.45,"mem_mb":91.6,"disk_size":"211M"},{"runtime":"python:3.11-slim","python_version":"3.11","os_libc":"slim (glibc)","variant":"proxy","exit_code":0,"wheel_type":null,"failure_reason":null,"install_time_s":null,"import_time_s":10.33,"mem_mb":97,"disk_size":"499M"},{"runtime":"python:3.11-slim","python_version":"3.11","os_libc":"slim (glibc)","variant":"default","exit_code":0,"wheel_type":null,"failure_reason":null,"install_time_s":null,"import_time_s":8.46,"mem_mb":91.6,"disk_size":"211M"},{"runtime":"python:3.12-alpine","python_version":"3.12","os_libc":"alpine (musl)","variant":"caching","exit_code":0,"wheel_type":null,"failure_reason":null,"install_time_s":null,"import_time_s":8.13,"mem_mb":88.5,"disk_size":"216.5M"},{"runtime":"python:3.12-alpine","python_version":"3.12","os_libc":"alpine (musl)","variant":"proxy","exit_code":1,"wheel_type":null,"failure_reason":null,"install_time_s":null,"import_time_s":null,"mem_mb":null,"disk_size":null},{"runtime":"python:3.12-alpine","python_version":"3.12","os_libc":"alpine (musl)","variant":"default","exit_code":0,"wheel_type":null,"failure_reason":null,"install_time_s":null,"import_time_s":7.93,"mem_mb":88.5,"disk_size":"216.1M"},{"runtime":"python:3.12-slim","python_version":"3.12","os_libc":"slim (glibc)","variant":"caching","exit_code":0,"wheel_type":null,"failure_reason":null,"install_time_s":null,"import_time_s":8.39,"mem_mb":88.5,"disk_size":"200M"},{"runtime":"python:3.12-slim","python_version":"3.12","os_libc":"slim (glibc)","variant":"proxy","exit_code":0,"wheel_type":null,"failure_reason":null,"install_time_s":null,"import_time_s":9.78,"mem_mb":93.7,"disk_size":"489M"},{"runtime":"python:3.12-slim","python_version":"3.12","os_libc":"slim (glibc)","variant":"default","exit_code":0,"wheel_type":null,"failure_reason":null,"install_time_s":null,"import_time_s":8.04,"mem_mb":88.5,"disk_size":"199M"},{"runtime":"python:3.13-alpine","python_version":"3.13","os_libc":"alpine (musl)","variant":"caching","exit_code":0,"wheel_type":null,"failure_reason":null,"install_time_s":null,"import_time_s":7.58,"mem_mb":89.5,"disk_size":"216.3M"},{"runtime":"python:3.13-alpine","python_version":"3.13","os_libc":"alpine (musl)","variant":"proxy","exit_code":1,"wheel_type":null,"failure_reason":null,"install_time_s":null,"import_time_s":null,"mem_mb":null,"disk_size":null},{"runtime":"python:3.13-alpine","python_version":"3.13","os_libc":"alpine (musl)","variant":"default","exit_code":0,"wheel_type":null,"failure_reason":null,"install_time_s":null,"import_time_s":7.68,"mem_mb":89.5,"disk_size":"215.9M"},{"runtime":"python:3.13-slim","python_version":"3.13","os_libc":"slim (glibc)","variant":"caching","exit_code":0,"wheel_type":null,"failure_reason":null,"install_time_s":null,"import_time_s":7.74,"mem_mb":89.5,"disk_size":"199M"},{"runtime":"python:3.13-slim","python_version":"3.13","os_libc":"slim (glibc)","variant":"proxy","exit_code":0,"wheel_type":null,"failure_reason":null,"install_time_s":null,"import_time_s":9.54,"mem_mb":94.8,"disk_size":"488M"},{"runtime":"python:3.13-slim","python_version":"3.13","os_libc":"slim (glibc)","variant":"default","exit_code":0,"wheel_type":null,"failure_reason":null,"install_time_s":null,"import_time_s":7.8,"mem_mb":89.5,"disk_size":"199M"},{"runtime":"python:3.9-alpine","python_version":"3.9","os_libc":"alpine (musl)","variant":"caching","exit_code":0,"wheel_type":null,"failure_reason":null,"install_time_s":null,"import_time_s":7.21,"mem_mb":91.2,"disk_size":"210.6M"},{"runtime":"python:3.9-alpine","python_version":"3.9","os_libc":"alpine (musl)","variant":"proxy","exit_code":1,"wheel_type":null,"failure_reason":null,"install_time_s":null,"import_time_s":null,"mem_mb":null,"disk_size":null},{"runtime":"python:3.9-alpine","python_version":"3.9","os_libc":"alpine (musl)","variant":"default","exit_code":0,"wheel_type":null,"failure_reason":null,"install_time_s":null,"import_time_s":7.29,"mem_mb":91.2,"disk_size":"210.2M"},{"runtime":"python:3.9-slim","python_version":"3.9","os_libc":"slim (glibc)","variant":"caching","exit_code":0,"wheel_type":null,"failure_reason":null,"install_time_s":null,"import_time_s":6.4,"mem_mb":91.2,"disk_size":"194M"},{"runtime":"python:3.9-slim","python_version":"3.9","os_libc":"slim (glibc)","variant":"proxy","exit_code":0,"wheel_type":null,"failure_reason":null,"install_time_s":null,"import_time_s":7.79,"mem_mb":96.5,"disk_size":"295M"},{"runtime":"python:3.9-slim","python_version":"3.9","os_libc":"slim (glibc)","variant":"default","exit_code":0,"wheel_type":null,"failure_reason":null,"install_time_s":null,"import_time_s":6.44,"mem_mb":91.2,"disk_size":"193M"}]},"quickstart_checks":{"last_tested":"2026-05-12","tag":"verified","tag_description":"quickstart runs on critical runtimes, recently tested","results":[{"runtime":"python:3.10-alpine","exit_code":0},{"runtime":"python:3.10-slim","exit_code":0},{"runtime":"python:3.11-alpine","exit_code":0},{"runtime":"python:3.11-slim","exit_code":0},{"runtime":"python:3.12-alpine","exit_code":0},{"runtime":"python:3.12-slim","exit_code":0},{"runtime":"python:3.13-alpine","exit_code":0},{"runtime":"python:3.13-slim","exit_code":0},{"runtime":"python:3.9-alpine","exit_code":0},{"runtime":"python:3.9-slim","exit_code":0}]}}