Together AI Python SDK

1.4.1 · active · verified Sat Feb 28

Official Python SDK for Together AI platform. OpenAI-compatible API for running 100+ open-source models. Has gone through two full rewrites: pre-v1 (2023) → v1.0 (April 2024) → v2.0 RC (2025, not yet GA on PyPI). Can also be used via OpenAI client with base_url override.

Warnings

breaking Pre-v1 SDK (before April 2024) is completely incompatible. Different imports, different client structure, different method names. All pre-v1 tutorials and code are broken.
Fix: pip install --upgrade together and rewrite using from together import Together pattern
breaking v2.0 RC is available via pip install --pre but is NOT GA. Breaking changes still possible during RC period. Do not use in production.
Fix: Use pip install together (no --pre flag) for stable v1. Monitor GitHub for GA announcement.
breaking v2.0 redesigns error handling entirely. TogetherException replaced with TogetherError hierarchy (APIStatusError, BadRequestError, AuthenticationError, RateLimitError). Error handling code from v1 will not catch v2 errors.
Fix: Update exception handlers to catch specific v2 error types when migrating to v2.
breaking Files, Batches, Endpoints, Evals, and Code Interpreter APIs have updated method names and response shapes in v2. Not drop-in compatible.
Fix: Check the Python SDK Migration Guide for API-by-API before/after examples before upgrading.
gotcha Model IDs use provider/model-name format (e.g. meta-llama/Llama-3.3-70B-Instruct-Turbo). LLMs frequently hallucinate incorrect Together-specific model ID formats.
Fix: Always verify current model IDs at https://api.together.xyz/models — they change as models are added/removed.
gotcha Together API is OpenAI-compatible. You can use the OpenAI Python client with base_url='https://api.together.xyz/v1' and TOGETHER_API_KEY. This is not a hack — it's documented and supported.
Fix: If already using OpenAI SDK, just swap base_url and api_key. No native Together SDK needed for basic chat/completions/embeddings.
gotcha The legacy together package (pre-v1) used TOGETHER_API_KEY but stored it differently. v1+ reads TOGETHER_API_KEY env var automatically via Together() with no argument.
Fix: Set TOGETHER_API_KEY env var. Together() reads it automatically — no need to pass api_key= explicitly.

Install

pip install together pip (v1 stable)
uv add together uv
pip install --pre --upgrade together pip (v2 RC — not GA)

Imports

Together
```
from together import Together
```
Pre-v1 had a completely different import structure. v1+ uses top-level from together import Together.
AsyncTogether
```
from together import AsyncTogether
```
Async client. Same interface as Together, use with await.
OpenAI client (alternative)
```
openai.OpenAI(api_key=os.environ['TOGETHER_API_KEY'], base_url='https://api.together.xyz/v1')
```
Together is fully OpenAI-compatible. Using the OpenAI client with base_url override is a valid and common pattern — no native SDK needed.

Quickstart

Minimal chat completion using native Together SDK

import os
from together import Together

client = Together(api_key=os.environ['TOGETHER_API_KEY'])

response = client.chat.completions.create(
    model='meta-llama/Llama-3.3-70B-Instruct-Turbo',
    messages=[{'role': 'user', 'content': 'Hello'}]
)
print(response.choices[0].message.content)

view raw JSON →