LangChain Hugging Face Integration

Warnings

• breaking The LangChain ecosystem transitioned from a monolithic 'langchain' package to modular 'langchain-*' integration packages. Components previously imported directly from `langchain` (e.g., `langchain.llms.HuggingFaceHub`) are now in `langchain-huggingface` (e.g., `langchain_huggingface.llms.HuggingFacePipeline`).
  Fix: Update all relevant import statements to target the new `langchain_huggingface` package structure (e.g., `from langchain_huggingface.llms import HuggingFacePipeline`).
• gotcha Running large Hugging Face models locally requires significant hardware resources (RAM/VRAM). Attempting to load or run large models on insufficient hardware will lead to slow inference, out-of-memory errors, or crashes.
  Fix: For local inference, choose smaller, quantized models, or ensure adequate GPU/CPU resources. For larger models, consider using Hugging Face Inference Endpoints or other cloud-based LLM APIs.
• gotcha Many Hugging Face models, especially when accessed via the Hugging Face Hub Inference API or for downloading private models, require an `HUGGINGFACEHUB_API_TOKEN`. Without it, you might encounter authentication errors or rate limits.
  Fix: Obtain an API token from Hugging Face and set it as an environment variable (e.g., `export HUGGINGFACEHUB_API_TOKEN='hf_...'`) or pass it explicitly to the model constructor via `model_kwargs`.
• gotcha Using `trust_remote_code=True` when loading models from Hugging Face can be a security risk as it executes arbitrary code from the model repository. This is sometimes required for custom architectures or tokenizers.
  Fix: Only enable `trust_remote_code=True` for models from trusted sources. Understand the implications before using it. If possible, prefer models that do not require custom code.
• gotcha Version conflicts can arise between `langchain-huggingface`, `langchain-core`, `transformers`, and `huggingface-hub`. These packages often have strict dependency ranges, and mismatched versions can cause unexpected behavior or import errors.
  Fix: Always install `langchain-huggingface` first to allow `pip` to resolve compatible dependencies. Use a virtual environment to isolate dependencies. If conflicts occur, check the `install_requires` in `langchain-huggingface`'s `pyproject.toml` or `setup.py` for exact version ranges.

Install

• pip install langchain-huggingface Install package

Imports

• ChatHuggingFace

  from langchain_huggingface.chat_models import ChatHuggingFace

  The original `langchain` package was monolithic; integrations are now separate.
• HuggingFacePipeline

  from langchain_huggingface.llms import HuggingFacePipeline

  Moved from the monolithic `langchain` package to the dedicated integration package.
• HuggingFaceEmbeddings

  from langchain_huggingface.embeddings import HuggingFaceEmbeddings

  Embeddings were also moved to the dedicated integration package.

Quickstart

This quickstart demonstrates how to use `HuggingFacePipeline` to load a model and perform text generation. It uses `google/flan-t5-small` as an example, which works well for `text2text-generation` tasks. Ensure `transformers` and a deep learning backend (like `torch`) are installed.

import os
from langchain_huggingface.llms import HuggingFacePipeline
from langchain_core.prompts import PromptTemplate
from langchain_core.output_parsers import StrOutputParser

# Set your Hugging Face API token if accessing models from the Hub
# os.environ["HUGGINGFACEHUB_API_TOKEN"] = os.environ.get("HUGGINGFACEHUB_API_TOKEN", "")

# Initialize the HuggingFacePipeline with a small, accessible model
# Ensure 'transformers' and a deep learning backend (e.g., 'torch') are installed.
llm = HuggingFacePipeline.from_model_id(
    model_id="google/flan-t5-small",
    task="text2text-generation",
    pipeline_kwargs={"max_new_tokens": 100},
    # Pass token explicitly if needed, e.g., for private models or inference endpoints
    # model_kwargs={"huggingfacehub_api_token": os.environ.get("HUGGINGFACEHUB_API_TOKEN", "")}
)

# Create a simple prompt template
template = "Question: {question}\nAnswer:"
prompt = PromptTemplate.from_template(template)

# Create a chain
chain = prompt | llm | StrOutputParser()

# Invoke the chain
question = "What is the capital of France?"
response = chain.invoke({"question": question})
print(response)