Groq Python SDK

0.18.0 · active · verified Sat Feb 28

Official Python SDK for GroqCloud API. OpenAI-compatible interface for ultra-low-latency LLM inference on Groq LPU hardware. Model IDs change frequently as models are deprecated and replaced with no versioned aliases.

Warnings

breaking groq.cloud.core (pre-release API) is fully removed. ChatCompletion class no longer exists. Any code from early 2024 tutorials is broken.
Fix: Replace entire client setup with: from groq import Groq; client = Groq(api_key=...)
breaking Models are deprecated and removed with no versioned aliases. gemma-7b-it and mixtral-8x7b-32768 removed. llama-guard-3-8b decommissioned. Hardcoded model IDs break silently.
Fix: Never hardcode model IDs in production. Query https://api.groq.com/openai/v1/models to get current active models. Check https://console.groq.com/docs/deprecations before each release.
breaking max_tokens is deprecated in favor of max_completion_tokens. Still works but may be removed.
Fix: Replace max_tokens= with max_completion_tokens= in all chat.completions.create() calls
breaking functions and function_call parameters are deprecated in favor of tools and tool_choice respectively.
Fix: Migrate function_call pattern to tools=[{type: 'function', function: {...}}] pattern
breaking exclude_domains and include_domains parameters deprecated for agentic tooling. Use search_settings parameter instead.
Fix: Move domain filtering into search_settings={include_domains: [...]} or search_settings={exclude_domains: [...]}
gotcha n parameter (number of completions) only supports n=1. Passing any other value returns a 400 error.
Fix: Do not use n > 1. Run multiple requests instead.
gotcha logprobs, presence_penalty, and frequency_penalty are listed in the API but not supported by any current models. Passing them does not error but has no effect.
Fix: Do not rely on these parameters for model behavior control
gotcha Preview models can be discontinued at short notice. Do not use in production.
Fix: Use only production-tier models. Check model status at console.groq.com/docs/models.
gotcha Rate limits are per-model and vary significantly. Free tier limits are very low. 429s happen frequently in dev without a paid plan.
Fix: Check current limits at console.groq.com/settings/limits. Implement exponential backoff on groq.RateLimitError.

Install

pip install groq pip
uv add groq uv

Imports

Groq
```
from groq import Groq
```
groq.cloud.core was the original pre-release SDK (v0.3.0, 2024). Completely removed. All current code uses from groq import Groq.
AsyncGroq
```
from groq import AsyncGroq
```
Drop-in async version of Groq client. Same interface, use await.
aiohttp backend
```
from groq import DefaultAioHttpClient
```
Pass as http_client=DefaultAioHttpClient() to AsyncGroq for better concurrency.

Quickstart

Minimal chat completion

import os
from groq import Groq

client = Groq(api_key=os.environ['GROQ_API_KEY'])

response = client.chat.completions.create(
    model='llama-3.3-70b-versatile',
    messages=[{'role': 'user', 'content': 'Hello'}]
)
print(response.choices[0].message.content)

view raw JSON →