LangServe

Common errors

• ModuleNotFoundError: No module named 'fastapi' OR ModuleNotFoundError: No module named 'uvicorn'

  cause `fastapi` and `uvicorn` are peer dependencies for running a server, but not strict requirements of the `langserve` package itself.
  fix Install them manually: `pip install fastapi uvicorn`

• AttributeError: module 'pydantic' has no attribute 'json_schema' OR TypeError: 'Type[Any]' object is not subscriptable

  cause This indicates a Pydantic version mismatch. LangServe v0.3.0+ requires Pydantic V2, but another dependency might be forcing Pydantic V1, or your code is using V1 syntax.
  fix Ensure all dependencies requiring Pydantic are compatible with V2. Upgrade your `pydantic` package to `pydantic>=2` and verify other `langchain` ecosystem packages are also updated to their latest versions.

• AttributeError: 'BaseChatModel' object has no attribute 'invoke' OR TypeError: 'RunnableSequence' object is not callable

  cause Incompatibility between `langserve` and `langchain-core` versions. The `langchain-core` API underwent significant changes (e.g., introduction of `invoke`, `batch`, `stream`).
  fix Upgrade both `langserve` and `langchain-core` to their latest compatible versions (`pip install --upgrade langserve langchain-core`). For `langserve==0.3.3`, ensure `langchain-core>=1.0.0`.

• ModuleNotFoundError: No module named 'langchain_openai'

  cause Language model integrations are often in separate packages (e.g., `langchain-openai`, `langchain-anthropic`) and are not part of `langserve` or `langchain-core`.
  fix Install the specific integration package you need: `pip install langchain-openai` for OpenAI models.

Warnings

• breaking LangServe v0.3.0 and later upgraded to Pydantic V2. This can cause `TypeError`, `AttributeError`, or `ValidationError` issues if other installed packages (or your own code) are still using Pydantic V1, or if you try to use V1 syntax.
  Fix: Ensure all dependencies requiring Pydantic are compatible with V2. Upgrade your `pydantic` package to `pydantic>=2` and verify other `langchain` ecosystem packages are also updated to their latest versions. Use Pydantic's migration guide if updating your own models.
• breaking LangServe v0.3.3 is explicitly compatible with `langchain-core` V1. Older `langserve` versions might not work correctly, leading to `AttributeError` or unexpected behavior when interacting with modern LangChain components.
  Fix: Upgrade both `langserve` and `langchain-core` to their latest compatible versions (`pip install --upgrade langserve langchain-core`). For `langserve==0.3.3`, ensure `langchain-core>=1.0.0`.
• gotcha While LangServe integrates with FastAPI, `fastapi` and `uvicorn` are not strict dependencies of the `langserve` package itself. Running a LangServe API server requires these to be installed separately.
  Fix: Install them manually using `pip install fastapi uvicorn`.
• deprecated Prior to v0.2.0, some internal module structures or import paths might have differed due to rapid LangChain ecosystem evolution. Directly importing internal components from older LangChain or LangServe versions might lead to `ModuleNotFoundError` or `ImportError` after upgrading.
  Fix: Always use documented public APIs, typically imported from `langserve`, `langchain_core`, or `langchain_openai`. Consult migration guides for major version bumps in the LangChain ecosystem.
• gotcha LangServe (especially in versions <= 0.2) might not fully support OpenAPI docs generation when Pydantic V2 is used, though API endpoints and the playground typically function as expected.
  Fix: Upgrade to LangServe >=0.3.0 for better Pydantic V2 support, or downgrade Pydantic to V1 (`pip install pydantic==1.*`) if OpenAPI docs are critical for older LangServe versions.

Install

• pip install langserve Basic Installation
• pip install "langserve[server]" "langserve[client]" langchain-openai Full Installation (Server, Client, OpenAI LLM)

Imports

• add_routes
  wrong:

  from langchain.callbacks.langserve import add_routes

  correct:

  from langserve import add_routes

  The `add_routes` function is specific to `langserve` for deploying runnables, not a general LangChain callback.
• FastAPI

  from fastapi import FastAPI

• ChatOpenAI

  from langchain_openai import ChatOpenAI

• ChatPromptTemplate

  from langchain_core.prompts import ChatPromptTemplate

• Runnable
  wrong:

  from langchain.schema.runnable import Runnable

  correct:

  from langchain_core.runnables import Runnable

  With the modularization of LangChain, core components like Runnable are now in `langchain_core`.

Quickstart

This quickstart sets up a basic LangServe API server with two endpoints: one for a raw OpenAI Chat model and another for a simple 'joke' chain (prompt + OpenAI Chat model). It uses FastAPI and uvicorn. Ensure `OPENAI_API_KEY` is set in your environment or replace the dummy key. You'll need `fastapi`, `uvicorn`, and `langchain-openai` installed.

import os
from fastapi import FastAPI
from langchain_core.prompts import ChatPromptTemplate
from langchain_openai import ChatOpenAI
from langserve import add_routes
import uvicorn

# Set your OpenAI API key from environment variable for local testing.
# In a production environment, use proper secrets management.
openai_api_key = os.environ.get('OPENAI_API_KEY', 'sk-xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx')

app = FastAPI(
    title="LangChain Server",
    version="1.0",
    description="A simple API server using LangChain's Runnable interfaces",
)

# A simple runnable (e.g., just an LLM)
add_routes(
    app,
    ChatOpenAI(api_key=openai_api_key),
    path="/openai",
)

# A more complex runnable (e.g., prompt + LLM)
prompt = ChatPromptTemplate.from_template("Tell me a joke about {topic}")
model = ChatOpenAI(api_key=openai_api_key)
chain = prompt | model

add_routes(
    app,
    chain,
    path="/joke",
)

if __name__ == "__main__":
    # To run this, save it as a Python file (e.g., server.py) and execute:
    # uvicorn server:app --reload
    uvicorn.run(app, host="0.0.0.0", port=8000)