Token Management: Handling Over-Sized Agent Requests

Reliability · updated Mon Feb 23

Preventing crashes when prompts or document retrievals exceed model limits.

Steps

  1. Count tokens before sending requests.
  2. Reserve context space for outputs.
  3. Summarize large documents recursively.
  4. Prune history with a sliding window.
  5. Warn users when quotas are near limits.

view raw JSON →