Groq Integration for LangChain

1.1.2 · active · verified Sat Apr 11

langchain-groq is an integration package that connects Groq's ultra-fast inference API with the LangChain framework. It allows developers to leverage Groq's LPU-powered language models, such as Llama 3 and Mixtral, within their LangChain applications for fast and efficient LLM inference. The library is actively maintained, with frequent updates often aligning with new features or fixes in the broader LangChain ecosystem.

Warnings

gotcha The `GROQ_API_KEY` environment variable must be set for authentication with the Groq API. Failure to do so will result in authentication errors.
Fix: Set `GROQ_API_KEY` in your environment or pass it explicitly to the `ChatGroq` constructor (e.g., `ChatGroq(api_key=...)`).
gotcha Using an unsupported or incorrect `model` name when initializing `ChatGroq` will lead to runtime errors. Additionally, certain features like `reasoning_format` or vision input require specific Groq models.
Fix: Refer to the official Groq documentation for a list of available models and their supported capabilities. Common models include 'llama-3.3-70b-versatile' and 'mixtral-8x7b-32768'.
breaking As a LangChain partner package, `langchain-groq` relies on `langchain-core`. The LangChain v1.0 update introduced significant breaking changes to `langchain-core`'s API, including changes to import paths, agent initialization, and the universal `Runnable` interface. Old patterns from `langchain` v0.x will not work.
Fix: Consult the official LangChain v1.0 upgrade guide. Update import statements (e.g., `from langchain_core.prompts import ChatPromptTemplate`), use the `invoke` method consistently, and adopt the `Runnable` interface for chains and agents.
gotcha If utilizing vision capabilities with supported Groq models, ensure that image URLs provided are publicly accessible, return the image directly without redirects, and do not exceed 20MB in size.
Fix: Verify image URL accessibility and size constraints according to Groq API specifications.

Install

pip install langchain-groq Install `langchain-groq`

Imports

ChatGroq
```
from langchain_groq import ChatGroq
```
Used to interact with Groq's chat models for fast inference.

Quickstart

This quickstart demonstrates how to initialize and use the `ChatGroq` model within LangChain. It shows both a single `invoke` call and streaming capabilities. Ensure your `GROQ_API_KEY` is set as an environment variable.

import os
from langchain_groq import ChatGroq
from langchain_core.messages import HumanMessage, SystemMessage

# Ensure your Groq API key is set as an environment variable
# os.environ["GROQ_API_KEY"] = "your_groq_api_key_here"
# Or pass it directly to the ChatGroq constructor (not recommended for production)

groq_api_key = os.environ.get("GROQ_API_KEY")

if not groq_api_key:
    print("Error: GROQ_API_KEY environment variable not set.")
else:
    llm = ChatGroq(
        model="llama-3.3-70b-versatile",
        temperature=0.0,
        max_retries=2,
        api_key=groq_api_key
    )

    messages = [
        SystemMessage(content="You are a helpful assistant that translates English to French."),
        HumanMessage(content="I love programming."),
    ]

    # Invoke the model
    response = llm.invoke(messages)
    print("\n--- Invoke Response ---")
    print(f"Content: {response.content}")

    # Stream the response
    print("\n--- Streaming Response ---")
    for chunk in llm.stream(messages):
        print(chunk.content, end="")
    print()

view raw JSON →