LLMTest

LLM proxy that benchmarks AI models on real prompts and finds cheaper, faster alternatives across 340+ models.

Install

npx llmtest-mcp

status Show proxy status and activity summary
list_flows List all AI flows with cost and latency stats
get_suggestions Get pending model-switch recommendations
update_suggestion Accept or dismiss a suggestion
run_benchmark Benchmark a flow against challenger models
optimize_prompt Rewrite a flow's prompt and find a cheaper model that still works
seed_samples Add test prompts for pre-launch benchmarking
list_samples Show stored test samples per flow
list_new_models Show new and trending models
get_account Check credit balance and usage
get_autopilot_status Check whether autopilot is on and whether the account is eligible
enable_autopilot Turn on weekly auto-optimization with safety gates + drift-based auto-revert
disable_autopilot Turn off autopilot (existing optimizations stay active)
list_active_optimizations List auto-accepted optimizations still inside their 24h revert window
revert_optimization Roll an auto-accepted optimization back to the previous prompt