Together AI Python SDK

raw JSON →
1.4.1 verified Tue May 12 auth: no python install: verified quickstart: verified

Official Python SDK for Together AI platform. OpenAI-compatible API for running 100+ open-source models. Has gone through two full rewrites: pre-v1 (2023) → v1.0 (April 2024) → v2.0 RC (2025, not yet GA on PyPI). Can also be used via OpenAI client with base_url override.

pip install together

Common errors

error ModuleNotFoundError: No module named 'together' ↓
cause The Python package `together-ai` has not been installed, or the import statement uses an incorrect module name.
fix
Install the package using pip: pip install together-ai. The correct import is import together.
error together.TogetherError: API key is missing or invalid. ↓
cause The Together AI API key is either not provided to the client constructor, not set as an environment variable (`TOGETHER_API_KEY`), or the provided key is invalid.
fix
Ensure the TOGETHER_API_KEY environment variable is set or pass the API key directly when instantiating the client: client = together.Together(api_key='YOUR_API_KEY').
error AttributeError: 'Together' object has no attribute 'Completion' ↓
cause The user is attempting to access static `Completion` methods (pre-v1 API) on an instantiated `together.Together` client object (v1.0+ API), which is incorrect.
fix
Instantiate the client and then use client.completions.create or client.chat.completions.create: client = together.Together(); response = client.completions.create(...).
error openai.AuthenticationError: No API key provided. You can set your API key in code ... or via the OPENAI_API_KEY environment variable. ↓
cause When using the `openai` client with Together AI's `base_url`, the `openai` library expects the API key to be explicitly passed or the `OPENAI_API_KEY` environment variable to be set, not `TOGETHER_API_KEY`.
fix
Pass the Together AI API key explicitly to the OpenAI client constructor: from openai import OpenAI; client = OpenAI(base_url='https://api.together.xyz/v1', api_key=os.environ.get('TOGETHER_API_KEY')).
error TypeError: together.completions.create() got an unexpected keyword argument 'messages' ↓
cause The user is trying to pass `messages` (for chat models) to the `together.completions.create` endpoint, which expects a `prompt`.
fix
Use client.chat.completions.create(model='...', messages=[{'role': 'user', 'content': 'Hello'}]) for chat models, and client.completions.create(model='...', prompt='Hello, world!') for completion models.

Warnings

breaking Pre-v1 SDK (before April 2024) is completely incompatible. Different imports, different client structure, different method names. All pre-v1 tutorials and code are broken. ↓
fix pip install --upgrade together and rewrite using from together import Together pattern
breaking v2.0 RC is available via pip install --pre but is NOT GA. Breaking changes still possible during RC period. Do not use in production. ↓
fix Use pip install together (no --pre flag) for stable v1. Monitor GitHub for GA announcement.
breaking v2.0 redesigns error handling entirely. TogetherException replaced with TogetherError hierarchy (APIStatusError, BadRequestError, AuthenticationError, RateLimitError). Error handling code from v1 will not catch v2 errors. ↓
fix Update exception handlers to catch specific v2 error types when migrating to v2.
breaking Files, Batches, Endpoints, Evals, and Code Interpreter APIs have updated method names and response shapes in v2. Not drop-in compatible. ↓
fix Check the Python SDK Migration Guide for API-by-API before/after examples before upgrading.
gotcha Model IDs use provider/model-name format (e.g. meta-llama/Llama-3.3-70B-Instruct-Turbo). LLMs frequently hallucinate incorrect Together-specific model ID formats. ↓
fix Always verify current model IDs at https://api.together.xyz/models — they change as models are added/removed.
gotcha Together API is OpenAI-compatible. You can use the OpenAI Python client with base_url='https://api.together.xyz/v1' and TOGETHER_API_KEY. This is not a hack — it's documented and supported. ↓
fix If already using OpenAI SDK, just swap base_url and api_key. No native Together SDK needed for basic chat/completions/embeddings.
gotcha The legacy together package (pre-v1) used TOGETHER_API_KEY but stored it differently. v1+ reads TOGETHER_API_KEY env var automatically via Together() with no argument. ↓
fix Set TOGETHER_API_KEY env var. Together() reads it automatically — no need to pass api_key= explicitly.

Install

uv add together
pip install --pre --upgrade together

Install compatibility verified last tested: 2026-05-12

python os / libc variant status wheel install import disk
3.10 alpine (musl) --pre - - 1.04s 134.8M
3.10 alpine (musl) together - - 1.02s 134.7M
3.10 slim (glibc) --pre - - 0.79s 205M
3.10 slim (glibc) together - - 0.81s 205M
3.11 alpine (musl) --pre - - 1.43s 145.6M
3.11 alpine (musl) together - - 1.45s 145.4M
3.11 slim (glibc) --pre - - 1.19s 216M
3.11 slim (glibc) together - - 1.19s 216M
3.12 alpine (musl) --pre - - 1.49s 135.9M
3.12 alpine (musl) together - - 1.49s 135.7M
3.12 slim (glibc) --pre - - 1.44s 206M
3.12 slim (glibc) together - - 1.48s 206M
3.13 alpine (musl) --pre - - 1.36s 132.5M
3.13 alpine (musl) together - - 1.38s 132.3M
3.13 slim (glibc) --pre - - 1.36s 205M
3.13 slim (glibc) together - - 1.38s 205M
3.9 alpine (musl) --pre - - 1.03s 130.8M
3.9 alpine (musl) together - - 1.01s 130.6M
3.9 slim (glibc) --pre - - 0.93s 202M
3.9 slim (glibc) together - - 0.92s 202M

Imports

Quickstart verified last tested: 2026-05-12

Minimal chat completion using native Together SDK

import os
from together import Together

client = Together(api_key=os.environ['TOGETHER_API_KEY'])

response = client.chat.completions.create(
    model='meta-llama/Llama-3.3-70B-Instruct-Turbo',
    messages=[{'role': 'user', 'content': 'Hello'}]
)
print(response.choices[0].message.content)
last verified Tue May 12
next check Sat Mar 28