{"id":26,"library":"guidance","title":"Guidance","description":"Microsoft-backed constrained generation framework for LLMs. Programs interleave control flow with generation via lm += gen(...) syntax. Supports regex, CFGs, JSON schema, and select() constraints using a Rust-based llguidance engine. Backends: Transformers, llama.cpp, OpenAI, Azure AI. Model objects are immutable — each += produces a copy.","status":"active","version":"0.3.1","language":"python","source_language":"en","source_url":"https://github.com/guidance-ai/guidance","tags":["guidance","constrained-generation","llm","structured-output","regex","json-schema","microsoft","llama-cpp","transformers"],"install":[{"cmd":"pip install guidance","lang":"bash","label":"Python (base)"},{"cmd":"pip install guidance[transformers]","lang":"bash","label":"Python (Transformers backend)"},{"cmd":"pip install guidance[llamacpp]","lang":"bash","label":"Python (llama.cpp backend)"}],"dependencies":[{"reason":"Required for LlamaCpp backend. Version-pinned — guidance specifies minimum >=0.3.12. ABI breaks between llama.cpp versions cause silent failures.","package":"llama-cpp-python","optional":true},{"reason":"Required for Transformers backend. Not bundled in base install.","package":"transformers","optional":true}],"imports":[{"note":"guidance.llms namespace removed entirely. guidance.llm (singular) also removed. Only guidance.models is valid.","wrong":"guidance.llms.OpenAI('gpt-4o')","symbol":"models.OpenAI","correct":"from guidance.models import OpenAI"},{"note":"Import directly from guidance.models. Requires transformers package installed separately.","wrong":"from guidance import models; models.Transformers()","symbol":"Transformers","correct":"from guidance.models import Transformers"}],"quickstart":{"code":"from guidance import system, user, assistant, gen\nfrom guidance.models import Transformers\n\nlm = Transformers('microsoft/Phi-4-mini-instruct')\n\nwith system():\n    lm += 'You are a helpful assistant'\nwith user():\n    lm += 'What is the capital of France?'\nwith assistant():\n    lm += gen(name='answer', max_tokens=20)\n\nprint(lm['answer'])","lang":"python","description":"Minimal constrained generation with Transformers backend. Model object is immutable — lm is a copy at each step."},"warnings":[{"fix":"Use from guidance.models import OpenAI (or Transformers, LlamaCpp, AzureOpenAI).","message":"guidance.llms namespace removed. All pre-0.1.x code using guidance.llms.OpenAI() or guidance.llm.OpenAI() raises AttributeError.","severity":"breaking","affected_versions":"< 0.1.0"},{"fix":"JSON schema benchmarking moved to external repo guidance-ai/guidance-bench.","message":"bench module removed in 0.2.x. from guidance import bench raises ImportError.","severity":"breaking","affected_versions":"< 0.2.0"},{"fix":"Annotate stateless functions as @guidance(stateless=True). Pre-0.2 cookbook examples omit this and produce incorrect behavior.","message":"@guidance decorator requires explicit stateless=True for pure grammar composition functions that do not call gen() internally.","severity":"breaking","affected_versions":"< 0.2.0"},{"fix":"pip install 'llama-cpp-python>=0.3.12' explicitly. Check guidance release notes for updated pin on each upgrade.","message":"llama-cpp-python version must match guidance pin (currently >=0.3.12). Mismatched versions cause AttributeError or silent failures on LlamaCpp model init.","severity":"breaking","affected_versions":"all"},{"fix":"Always assign: lm = lm + ... or use lm += ... inside a with block where lm is re-bound.","message":"Model objects are immutable. lm += ... does not mutate in place — it returns a new copy. Not storing the result silently discards generated output.","severity":"gotcha","affected_versions":"all"},{"fix":"Install the required backend separately: pip install transformers or pip install llama-cpp-python.","message":"pip install guidance does not install any inference backend. Importing Transformers or LlamaCpp without the backend raises ImportError at runtime.","severity":"gotcha","affected_versions":"all"},{"fix":"Use Transformers or LlamaCpp backend for hard constraints. OpenAI backend supports JSON schema via OpenAI structured outputs API only.","message":"OpenAI and Azure backends do not support token-level constrained decoding (regex/CFG). Constraints fall back to prompt-based steering only. Full constrained generation requires a local backend.","severity":"gotcha","affected_versions":"all"},{"fix":"Ensure system-level C/C++ compilers (e.g., `apt-get install build-essential cmake` on Debian/Ubuntu, `yum install gcc gcc-c++ make cmake` on RHEL/CentOS, or Xcode Command Line Tools on macOS) are installed before attempting to `pip install llama-cpp-python`.","message":"Installing llama-cpp-python requires system-level C/C++ compilers (like gcc or clang) and CMake to be available in the environment for building its native components. Missing these tools will result in a 'Failed building wheel' error during installation.","severity":"breaking","affected_versions":"all"},{"fix":"Use a glibc-based Python image (e.g., Debian/Ubuntu), or for Alpine, install 'glibc-compat', 'gcc', 'g++', and 'rustup' and ensure the Rust toolchain is configured to build for musl targets.","message":"guidance or its Rust-based dependencies (e.g., llguidance) may fail to build on Alpine Linux (musl libc) environments due to missing system libraries or toolchain incompatibilities, leading to errors like 'libgcc_s.so.1 not found' or 'symbol not found' during Rust compilation.","severity":"breaking","affected_versions":"all"}],"env_vars":null,"last_verified":"2026-05-12T04:45:59.465Z","next_check":"2026-05-28T00:00:00.000Z","problems":[{"fix":"Install the 'guidance' package using pip: `pip install guidance`.","cause":"The 'guidance' package is not installed in your Python environment, or the environment where your code is running does not have it installed.","error":"ModuleNotFoundError: No module named 'guidance'"},{"fix":"Update your code to use `guidance.models.OpenAI(...)` or set the default language model using `guidance.llm = guidance.models.OpenAI(...)`.","cause":"The API for accessing language models within the `guidance` library has changed. The `llms` attribute is deprecated or incorrect.","error":"AttributeError: module 'guidance' has no attribute 'llms'"},{"fix":"Ensure you are using the correct `guidance` API, such as defining a program with the `@guidance` decorator for functions, or initializing a model object from `guidance.models`.","cause":"You are attempting to call the `guidance` module directly as a function, but it is intended to be used by calling its specific functions, constructors (e.g., `guidance.models.OpenAI`), or as a decorator (`@guidance`).","error":"guidance is not callable"},{"fix":"Try using a different Hugging Face model known to be compatible with `guidance` (check the official `guidance` documentation for recommended models), or ensure your `transformers` and `guidance` library versions are up to date.","cause":"This error occurs when using `guidance.models.Transformers()` with a Hugging Face tokenizer that does not have the expected `byte_decoder` property, or there's an incompatibility in how `guidance` processes the tokenizer for constrained generation.","error":"AssertionError: The passed tokenizer does have a byte_decoder property and using a standard gpt2 byte_decoder fails!"}],"ecosystem":"pypi","meta_description":null,"install_score":45,"install_tag":"draft","quickstart_score":0,"quickstart_tag":"stale","pypi_latest":null,"install_checks":{"last_tested":"2026-05-12","tag":"draft","tag_description":"notable install failures or slow imports","results":[{"runtime":"python:3.10-alpine","python_version":"3.10","os_libc":"alpine (musl)","variant":"default","exit_code":1,"wheel_type":null,"failure_reason":null,"install_time_s":null,"import_time_s":null,"mem_mb":null,"disk_size":null},{"runtime":"python:3.10-alpine","python_version":"3.10","os_libc":"alpine (musl)","variant":"llamacpp","exit_code":1,"wheel_type":null,"failure_reason":null,"install_time_s":null,"import_time_s":null,"mem_mb":null,"disk_size":null},{"runtime":"python:3.10-alpine","python_version":"3.10","os_libc":"alpine (musl)","variant":"transformers","exit_code":1,"wheel_type":null,"failure_reason":null,"install_time_s":null,"import_time_s":null,"mem_mb":null,"disk_size":null},{"runtime":"python:3.10-slim","python_version":"3.10","os_libc":"slim (glibc)","variant":"default","exit_code":0,"wheel_type":null,"failure_reason":null,"install_time_s":null,"import_time_s":2.21,"mem_mb":39.2,"disk_size":"152M"},{"runtime":"python:3.10-slim","python_version":"3.10","os_libc":"slim (glibc)","variant":"llamacpp","exit_code":1,"wheel_type":null,"failure_reason":null,"install_time_s":null,"import_time_s":null,"mem_mb":null,"disk_size":null},{"runtime":"python:3.10-slim","python_version":"3.10","os_libc":"slim (glibc)","variant":"transformers","exit_code":0,"wheel_type":null,"failure_reason":null,"install_time_s":null,"import_time_s":4.45,"mem_mb":53.6,"disk_size":"276M"},{"runtime":"python:3.11-alpine","python_version":"3.11","os_libc":"alpine (musl)","variant":"default","exit_code":1,"wheel_type":null,"failure_reason":null,"install_time_s":null,"import_time_s":null,"mem_mb":null,"disk_size":null},{"runtime":"python:3.11-alpine","python_version":"3.11","os_libc":"alpine (musl)","variant":"llamacpp","exit_code":1,"wheel_type":null,"failure_reason":null,"install_time_s":null,"import_time_s":null,"mem_mb":null,"disk_size":null},{"runtime":"python:3.11-alpine","python_version":"3.11","os_libc":"alpine (musl)","variant":"transformers","exit_code":1,"wheel_type":null,"failure_reason":null,"install_time_s":null,"import_time_s":null,"mem_mb":null,"disk_size":null},{"runtime":"python:3.11-slim","python_version":"3.11","os_libc":"slim (glibc)","variant":"default","exit_code":0,"wheel_type":null,"failure_reason":null,"install_time_s":null,"import_time_s":3.13,"mem_mb":44.3,"disk_size":"162M"},{"runtime":"python:3.11-slim","python_version":"3.11","os_libc":"slim (glibc)","variant":"llamacpp","exit_code":1,"wheel_type":null,"failure_reason":null,"install_time_s":null,"import_time_s":null,"mem_mb":null,"disk_size":null},{"runtime":"python:3.11-slim","python_version":"3.11","os_libc":"slim (glibc)","variant":"transformers","exit_code":0,"wheel_type":null,"failure_reason":null,"install_time_s":null,"import_time_s":6.55,"mem_mb":59.6,"disk_size":"307M"},{"runtime":"python:3.12-alpine","python_version":"3.12","os_libc":"alpine (musl)","variant":"default","exit_code":1,"wheel_type":null,"failure_reason":null,"install_time_s":null,"import_time_s":null,"mem_mb":null,"disk_size":null},{"runtime":"python:3.12-alpine","python_version":"3.12","os_libc":"alpine (musl)","variant":"llamacpp","exit_code":1,"wheel_type":null,"failure_reason":null,"install_time_s":null,"import_time_s":null,"mem_mb":null,"disk_size":null},{"runtime":"python:3.12-alpine","python_version":"3.12","os_libc":"alpine (musl)","variant":"transformers","exit_code":1,"wheel_type":null,"failure_reason":null,"install_time_s":null,"import_time_s":null,"mem_mb":null,"disk_size":null},{"runtime":"python:3.12-slim","python_version":"3.12","os_libc":"slim (glibc)","variant":"default","exit_code":0,"wheel_type":null,"failure_reason":null,"install_time_s":null,"import_time_s":3.94,"mem_mb":43.4,"disk_size":"149M"},{"runtime":"python:3.12-slim","python_version":"3.12","os_libc":"slim (glibc)","variant":"llamacpp","exit_code":1,"wheel_type":null,"failure_reason":null,"install_time_s":null,"import_time_s":null,"mem_mb":null,"disk_size":null},{"runtime":"python:3.12-slim","python_version":"3.12","os_libc":"slim (glibc)","variant":"transformers","exit_code":0,"wheel_type":null,"failure_reason":null,"install_time_s":null,"import_time_s":7.51,"mem_mb":58.4,"disk_size":"291M"},{"runtime":"python:3.13-alpine","python_version":"3.13","os_libc":"alpine (musl)","variant":"default","exit_code":1,"wheel_type":null,"failure_reason":null,"install_time_s":null,"import_time_s":null,"mem_mb":null,"disk_size":null},{"runtime":"python:3.13-alpine","python_version":"3.13","os_libc":"alpine (musl)","variant":"llamacpp","exit_code":1,"wheel_type":null,"failure_reason":null,"install_time_s":null,"import_time_s":null,"mem_mb":null,"disk_size":null},{"runtime":"python:3.13-alpine","python_version":"3.13","os_libc":"alpine (musl)","variant":"transformers","exit_code":1,"wheel_type":null,"failure_reason":null,"install_time_s":null,"import_time_s":null,"mem_mb":null,"disk_size":null},{"runtime":"python:3.13-slim","python_version":"3.13","os_libc":"slim (glibc)","variant":"default","exit_code":0,"wheel_type":null,"failure_reason":null,"install_time_s":null,"import_time_s":3.73,"mem_mb":44.8,"disk_size":"148M"},{"runtime":"python:3.13-slim","python_version":"3.13","os_libc":"slim (glibc)","variant":"llamacpp","exit_code":1,"wheel_type":null,"failure_reason":null,"install_time_s":null,"import_time_s":null,"mem_mb":null,"disk_size":null},{"runtime":"python:3.13-slim","python_version":"3.13","os_libc":"slim (glibc)","variant":"transformers","exit_code":0,"wheel_type":null,"failure_reason":null,"install_time_s":null,"import_time_s":7.1,"mem_mb":59.2,"disk_size":"290M"},{"runtime":"python:3.9-alpine","python_version":"3.9","os_libc":"alpine (musl)","variant":"default","exit_code":1,"wheel_type":null,"failure_reason":null,"install_time_s":null,"import_time_s":null,"mem_mb":null,"disk_size":null},{"runtime":"python:3.9-alpine","python_version":"3.9","os_libc":"alpine (musl)","variant":"llamacpp","exit_code":1,"wheel_type":null,"failure_reason":null,"install_time_s":null,"import_time_s":null,"mem_mb":null,"disk_size":null},{"runtime":"python:3.9-alpine","python_version":"3.9","os_libc":"alpine (musl)","variant":"transformers","exit_code":1,"wheel_type":null,"failure_reason":null,"install_time_s":null,"import_time_s":null,"mem_mb":null,"disk_size":null},{"runtime":"python:3.9-slim","python_version":"3.9","os_libc":"slim (glibc)","variant":"default","exit_code":0,"wheel_type":null,"failure_reason":null,"install_time_s":null,"import_time_s":2.54,"mem_mb":39.5,"disk_size":"161M"},{"runtime":"python:3.9-slim","python_version":"3.9","os_libc":"slim (glibc)","variant":"llamacpp","exit_code":1,"wheel_type":null,"failure_reason":null,"install_time_s":null,"import_time_s":null,"mem_mb":null,"disk_size":null},{"runtime":"python:3.9-slim","python_version":"3.9","os_libc":"slim (glibc)","variant":"transformers","exit_code":0,"wheel_type":null,"failure_reason":null,"install_time_s":null,"import_time_s":5.35,"mem_mb":53.8,"disk_size":"286M"}]},"quickstart_checks":{"last_tested":"2026-05-11","tag":"stale","tag_description":"widespread failures or data too old to trust","results":[{"runtime":"python:3.10-alpine","exit_code":1},{"runtime":"python:3.10-slim","exit_code":1},{"runtime":"python:3.11-alpine","exit_code":1},{"runtime":"python:3.11-slim","exit_code":1},{"runtime":"python:3.12-alpine","exit_code":1},{"runtime":"python:3.12-slim","exit_code":1},{"runtime":"python:3.13-alpine","exit_code":1},{"runtime":"python:3.13-slim","exit_code":1},{"runtime":"python:3.9-alpine","exit_code":1},{"runtime":"python:3.9-slim","exit_code":1}]}}