LangChain Ollama Integration

1.1.0 · active · verified Fri Apr 10

langchain-ollama is an integration package that connects LangChain, a framework for developing applications powered by language models, with Ollama, a platform for running open-source large language models (LLMs) locally. This allows developers to build AI applications that leverage local LLMs for enhanced privacy, reduced costs, and offline capabilities. The library is actively maintained with frequent releases, currently at version 1.1.0, requiring Python versions >=3.10.0 and <4.0.0.

Warnings

breaking The primary import paths for `ChatOllama`, `OllamaLLM`, and `OllamaEmbeddings` have shifted from `langchain_community` to `langchain_ollama` (e.g., `from langchain_ollama import ChatOllama`). Older code relying on `langchain_community` imports may break or use deprecated versions.
Fix: Update your import statements to use `from langchain_ollama import ...` for the latest functionality and stability. For embeddings, use `from langchain_ollama.embeddings import OllamaEmbeddings`.
gotcha The `ollama` server application must be installed and running, and the desired LLM model (e.g., 'llama3') must be pulled using `ollama pull <model_name>` via the command line before `langchain-ollama` can connect to it. Failure to do so will result in connection or model not found errors.
Fix: Install Ollama from https://ollama.com and run `ollama pull <model_name>` for your chosen model(s). Ensure the Ollama server is active before running your Python application.
gotcha Ollama's native tool calling support is less mature compared to providers like OpenAI. When implementing agents that use tools, `langchain-ollama` often relies on JSON-based agent workarounds, which might have a lower success rate for complex tasks than native function calling.
Fix: For complex agentic workflows requiring robust tool use, consider testing thoroughly and potentially simplifying tool definitions or using structured output parsers for more reliable JSON parsing. For critical production systems with advanced tool calling, evaluate if alternative LLM providers offer more native support.
gotcha Connecting to an Ollama instance not running on the default `http://localhost:11434` requires explicitly setting the `base_url` parameter during instantiation (e.g., `ChatOllama(base_url='http://your-ollama-host:port')`). Incorrect configuration can lead to connection errors.
Fix: Pass the correct `base_url` parameter to `ChatOllama`, `OllamaLLM`, or `OllamaEmbeddings` constructors. It's recommended to use an environment variable for flexibility (e.g., `os.environ.get('OLLAMA_BASE_URL', 'http://localhost:11434')`).
deprecated The `OllamaEmbeddings` class in `langchain_community` was officially deprecated in favor of `langchain_ollama.embeddings.OllamaEmbeddings` around version `0.3.1` of `langchain-community`, with removal planned for `1.0.0` of `langchain-community`.
Fix: Migrate your `OllamaEmbeddings` imports to `from langchain_ollama.embeddings import OllamaEmbeddings` to use the current and supported version.

Install

pip install langchain-ollama Install Python package
curl -fsSL https://ollama.com/install.sh | sh ollama pull llama3 Install Ollama server and pull a model (Prerequisite)

Imports

ChatOllama
```
from langchain_ollama import ChatOllama
```
As of recent versions, ChatOllama moved from langchain_community to langchain_ollama for direct integration, though langchain_community might still expose older or aliased versions.
OllamaLLM
```
from langchain_ollama.llms import OllamaLLM
```
The canonical LLM import for text completion is now directly from langchain_ollama.llms.OllamaLLM, not the legacy Ollama from langchain_community.llms.
OllamaEmbeddings
```
from langchain_ollama.embeddings import OllamaEmbeddings
```
OllamaEmbeddings has moved to langchain_ollama.embeddings for direct integration in newer versions.

Quickstart

This quickstart demonstrates how to set up a `ChatOllama` instance to interact with a local LLM (e.g., Llama 3) via the Ollama server. It includes a basic chat prompt template and uses `StrOutputParser` for clean output. It also shows how to instantiate `OllamaEmbeddings` to generate vector embeddings. Before running, ensure the Ollama server is installed and running, and the specified model (e.g., 'llama3') has been pulled using `ollama pull <model_name>`.

import os
from langchain_ollama import ChatOllama
from langchain_core.prompts import ChatPromptTemplate
from langchain_core.output_parsers import StrOutputParser

# Ensure Ollama server is running and 'llama3' model is pulled (e.g., `ollama pull llama3`)
# You can check Ollama status at http://localhost:11434/

# Instantiate the ChatOllama model
# You can specify base_url if Ollama is not on default localhost:11434
llm = ChatOllama(model="llama3", base_url=os.environ.get("OLLAMA_BASE_URL", "http://localhost:11434"))

# Define a prompt template
prompt = ChatPromptTemplate.from_messages([
    ("system", "You are a helpful AI assistant. Answer the user's questions truthfully."),
    ("user", "{question}")
])

# Create a simple chain
chain = prompt | llm | StrOutputParser()

# Invoke the chain
response = chain.invoke({"question": "What is the capital of France?"})
print(response)

# Example for embeddings
from langchain_ollama.embeddings import OllamaEmbeddings
embeddings_model = OllamaEmbeddings(model="llama3", base_url=os.environ.get("OLLAMA_BASE_URL", "http://localhost:11434"))

query_vector = embeddings_model.embed_query("What is the largest city in France?")
print(f"Embedding vector length: {len(query_vector)}")

view raw JSON →