llmprobe
JSON →Probe LLM API endpoints and report health metrics including time to first token, latency, and throughput. Check single models or run full config-based health checks.
Tools · 4
- probe_all Probe all configured endpoints from probes.yml. Returns TTFT, latency, throughput, and health status for every model. Accepts an optional config parameter for a custom config path.
- probe_model Probe a single model without a config file. Requires provider, model, and api_key_env. Supports optional base_url for OpenAI-compatible endpoints and optional label for display.
- list_providers List all providers and models in the config file with their thresholds. Use this to discover available models before probing.
- get_config Return the full parsed configuration including defaults, providers, models, and thresholds.
Environment variables
OPENAI_API_KEYANTHROPIC_API_KEYAZURE_OPENAI_API_KEYAWS_ACCESS_KEY_IDAWS_SECRET_ACCESS_KEYGROQ_API_KEYDEEPSEEK_API_KEYTOGETHER_API_KEY
Links
★ 1 GitHub stars