Outlines

Warnings

• breaking Python 3.9 not supported. outlines requires Python >=3.10.
  Fix: Upgrade to Python 3.10+.
• breaking outlines.integrations module removed in 0.1.0. Any code importing from outlines.integrations raises ImportError.
  Fix: Use outlines.processors directly (RegexLogitsProcessor, JSONLogitsProcessor, CFGLogitsProcessor) for custom logits processor integration.
• breaking outlines.fsm.json_schema.build_regex_from_object removed. Old low-level FSM imports from outlines.fsm are gone — this module was split into outlines-core.
  Fix: Use high-level outlines.generate.json() or outlines.generate.regex() instead of constructing FSMs directly.
• breaking pip install outlines installs no inference backend. Importing any model backend without its extra raises ImportError at runtime.
  Fix: Install the required extra: pip install 'outlines[transformers]', 'outlines[vllm]', 'outlines[llamacpp]', etc.
• gotcha Two coexisting APIs in 1.x. Old style: outlines.models.transformers('name') + outlines.generate.json(model, Schema)(prompt). New style: outlines.from_transformers(hf_model, tokenizer) + model(prompt, Schema). Both work but mix-and-match fails — old model objects are not compatible with the new model(prompt, Schema) call signature.
  Fix: Pick one API style per codebase. New style is recommended. Old style is still valid for outlines.generate.* generators.
• gotcha OpenAI and other API backends do NOT support regex or CFG constraints. generate.regex / generate.cfg silently falls back to prompt-only steering or raises NotImplementedError.
  Fix: Use local backends (Transformers, vLLM, llama.cpp) for hard regex/CFG constraints. API backends only support JSON schema via the provider's native structured output API.
• gotcha FSM index compilation is expensive for complex JSON schemas. generate.json(model, Schema) compiles once and must be reused — calling it fresh per request tanks throughput.
  Fix: Create generator/model+schema pair once at startup and call it repeatedly. Do not re-instantiate per request.
• gotcha Optional[...] = None fields in Pydantic schemas can cause FSM compilation to hang for minutes or never complete on complex schemas.
  Fix: Avoid Optional fields with complex nested types in large schemas. Use Union[SpecificType, None] with simpler types, or restructure the schema to eliminate unbounded optional nesting.

Install

• pip install outlines Base (no backends)
• pip install 'outlines[transformers]' Transformers backend
• pip install 'outlines[vllm]' vLLM backend
• pip install 'outlines[llamacpp]' llama.cpp backend

Imports

• from_transformers

  import outlines; model = outlines.from_transformers(hf_model, hf_tokenizer)

  1.x preferred API takes pre-initialized HF model + tokenizer objects. The old outlines.models.transformers('model-name') string-based loader still works in 1.x but is the legacy pattern — tutorials and LLM-generated code overwhelmingly use the old style.
• from_openai

  import outlines; model = outlines.from_openai(openai_client, 'gpt-4o')

  1.x API takes an initialized OpenAI client object. Note: OpenAI backend does NOT support regex/CFG constraints — only JSON schema via OpenAI's structured outputs API.

Quickstart

1.x API: pass output type directly to model call. For local models, swap outlines.from_openai for outlines.from_transformers.

import outlines
from pydantic import BaseModel
from typing import Literal
import openai

class Customer(BaseModel):
    name: str
    urgency: Literal['high', 'medium', 'low']
    issue: str

client = openai.OpenAI()
model = outlines.from_openai(client, 'gpt-4o')

customer = model(
    'Alice needs help with login issues ASAP',
    Customer
)
print(customer)  # Always returns valid Customer object