GenAI Perf Analyzer CLI
JSON →GenAI-Perf is a command-line interface (CLI) tool designed for measuring the throughput and latency of generative AI models (Large Language Models, Vision Language Models, Embedding Models, Ranking Models, and LoRA Adapters) served through an inference server. It generates load, measures key performance metrics such as output token throughput, time to first token, inter-token latency, and request throughput, and reports the results to the console, CSV, and JSON files. While currently at version 0.0.16 and under rapid development, it is being actively phased out in favor of NVIDIA's new `AIPerf` tool for generative AI benchmarking.
Traffic · last 30 days stale · no recent hits
total hits 61
actors 9 distinct systems
last hit 17d ago AhrefsBot
top countries 🇺🇸 United States · 🇸🇬 Singapore · 🇨🇳 China · VN · NZ
Resources
API endpoints
full doc /v1/registry/genai-perf
install /v1/registry/genai-perf/install
compatibility /v1/registry/genai-perf/compatibility