LangChain LiteLLM Integration

Warnings

• breaking Critical supply chain attack on `litellm` (versions 1.82.7 and 1.82.8) in March 2026. These versions contained credential-stealing malware. `langchain-litellm` version 0.6.2 and above explicitly excludes these compromised `litellm` versions from its dependencies.
  Fix: Upgrade `langchain-litellm` to v0.6.2 or later (`pip install -U langchain-litellm`) to ensure malicious `litellm` versions are excluded. Always pin `litellm` and `langchain-litellm` versions in production environments (`litellm==x.y.z`, `langchain-litellm==a.b.c`) and review dependencies.
• gotcha When using Claude models, `tool_choice` might be automatically downgraded to `auto` if 'thinking' is enabled. This can alter expected tool-use behavior.
  Fix: Review the specific behavior with Claude models and tool calling if 'thinking' is enabled. Test tool invocation carefully after upgrading to v0.6.4.
• gotcha A common error is `litellm.BadRequestError: LLM Provider NOT provided`. This occurs when the underlying LLM provider for LiteLLM is not correctly specified or configured, often due to missing `model` parameters or incorrect API key environment variables.
  Fix: Ensure the `model` parameter is correctly formatted (e.g., `openai/gpt-3.5-turbo`, `anthropic/claude-3-opus-20240229`) and the corresponding API key (e.g., `OPENAI_API_KEY`, `ANTHROPIC_API_KEY`) is set in environment variables or passed explicitly.
• gotcha Integrating `langchain-litellm` with a LiteLLM Proxy often requires special handling for authentication headers (e.g., `Authorization: Bearer <token>`). LangChain's internal HTTP request mechanisms might not easily expose the ability to inject these custom headers, leading to integration difficulties.
  Fix: Consult LiteLLM's documentation on proxy usage and `langchain-litellm`'s specific configurations for connecting to a proxy. You might need to configure LiteLLM directly or wrap the `ChatLiteLLM` instance with custom HTTP client logic.
• gotcha Version `0.6.4` included a fix to `extract reasoning tokens and handle pydantic usage in metadata`. This could imply subtle changes or sensitivities related to Pydantic versions and how structured outputs or metadata are processed, which is a frequent source of issues in the LangChain ecosystem.
  Fix: Ensure your environment's Pydantic version is compatible. If encountering issues with structured outputs or metadata handling, specifically review changes in `v0.6.4` related to Pydantic and update your code accordingly.

Install

• pip install langchain-litellm Install stable version

Imports

• ChatLiteLLM

  from langchain_litellm import ChatLiteLLM

• ChatLiteLLMRouter

  from langchain_litellm import ChatLiteLLMRouter

• LiteLLMEmbeddings

  from langchain_litellm import LiteLLMEmbeddings

• LiteLLMEmbeddingsRouter

  from langchain_litellm import LiteLLMEmbeddingsRouter

• LiteLLMOCRLoader

  from langchain_litellm import LiteLLMOCRLoader

Quickstart

This quickstart demonstrates how to instantiate and use ChatLiteLLM for basic chat completions and LiteLLMEmbeddings for text embedding. Ensure the relevant API key (e.g., OPENAI_API_KEY) is set in your environment or passed directly to the constructor. The `model` parameter should specify the desired LLM provider and model in LiteLLM's unified format.

import os
from langchain_litellm import ChatLiteLLM
from langchain_core.messages import HumanMessage

# Set your API key for LiteLLM's underlying provider (e.g., OpenAI)
# For a real application, use a secure method to manage API keys.
os.environ["OPENAI_API_KEY"] = os.environ.get("OPENAI_API_KEY", "sk-your-openai-key")

# Instantiate ChatLiteLLM, specifying the model in LiteLLM's format
# (e.g., 'openai/gpt-3.5-turbo' for OpenAI)
chat_model = ChatLiteLLM(model="openai/gpt-3.5-turbo")

# Invoke the chat model
response = chat_model.invoke([HumanMessage(content="Hello, how are you?")])

print(response.content)

# Example for LiteLLMEmbeddings
from langchain_litellm import LiteLLMEmbeddings

# Note: API key can be passed explicitly if not in environment for embeddings
embeddings = LiteLLMEmbeddings(
    model="openai/text-embedding-3-small",
    api_key=os.environ.get("OPENAI_API_KEY", "sk-your-openai-key")
)

text = "This is a test document."
embedding = embeddings.embed_query(text)
print(f"Embedding length: {len(embedding)}")