AgentEval CLI

0.8.14 verified Fri May 01 auth: no javascript

Linter, benchmarker, and CI gate for AI coding instruction files (CLAUDE.md, AGENTS.md, .cursorrules, etc.). Current stable version is v0.8.14, released with weekly cadence. Automatically detects dead references, token bloat, contradictions, stale instructions, broken links, and invalid skill metadata. Ships as a lightweight npm wrapper (~3 KB) that downloads a prebuilt native binary on first run; no Node runtime needed at execution time. Differentiators: zero-config startup, GitHub Action support, multi-format support (Claude, Copilot, Cursor, OpenAI), and built-in CI gating with regression detection.

Common errors

error npx: command not found: agenteval ↓

cause Typo: user typed 'agenteval' instead of 'agenteval-cli'.

fix

Use 'npx agenteval-cli' or install globally with 'npm install -g agenteval-cli'.

error Error: Failed to download binary - could not reach GitHub CDN ↓

cause Network restrictions or corporate proxy blocking the binary download.

fix

Set HTTP_PROXY/HTTPS_PROXY environment variables, or download binary manually from GitHub Releases and place it in PATH.

error Error: Unsupported platform ↓

cause Running on Windows or Linux ARM64 (aarch64), which are not supported.

fix

Use WSL on Windows, or use the experimental RISC-V build. Check GitHub Releases for platform availability.

error agenteval: command not found ↓

cause CLI not found after global install; PATH may not include npm global binaries.

fix

Ensure npm global bin directory is in PATH. Run 'npm bin -g' to locate it.

Warnings

gotcha Package name is 'agenteval-cli', not 'agenteval'. Running 'npx agenteval' will invoke a different package. ↓

fix Use 'npx agenteval-cli' or 'npm install -g agenteval-cli'.

breaking Prebuilt binary download requires internet access on first run; offline or air-gapped environments fail. ↓

fix Download the binary manually from GitHub Releases and place it in your PATH, or use the Homebrew/curl install methods.

deprecated In version 0.8.x, the 'agenteval harvest' command is experimental and may change without notice. ↓

fix Use 'agenteval harvest --dry-run' for preview, but expect potential breaking changes in future versions.

gotcha The 'agenteval ci' command's --min-score and --max-regression flags are parsed as decimals; integer values cause parse errors. ↓

fix Always pass decimal values, e.g., '--min-score 0.7' not '--min-score 1'.

gotcha The 'agenteval watch' command may cause file descriptor leaks on macOS if interrupted abruptly (v0.8.13 fix partially addresses this). ↓

fix Update to v0.8.13 or later, or manually kill the process with SIGTERM instead of closing terminal.

Install

npm install agenteval-cli

yarn add agenteval-cli

pnpm add agenteval-cli

Imports

agenteval CLI
wrong
```
npx agenteval lint
```
correct
```
npx agenteval-cli lint
```
The package name is 'agenteval-cli', not 'agenteval'. Use npx agenteval-cli or npm install -g agenteval-cli.
agenteval CLI (installed globally)
wrong
```
npm install -g agenteval
```
correct
```
npm install -g agenteval-cli
```
The package is published as 'agenteval-cli' on npm. Do not install 'agenteval' as it's a different package.
GitHub Action
wrong
```
uses: agenteval/agenteval@v0
```
correct
```
uses: lukasmetzler/agenteval@v0
```
The GitHub Action is hosted under the 'lukasmetzler/agenteval' repository, not a separate 'agenteval' org.

Quickstart

Initializes, lints, and CI-gates AI instruction files using the CLI with common flags.

npx agenteval-cli init  # create default agenteval.yaml
# Lint instruction files
npx agenteval-cli lint
# Lint with explanations
npx agenteval-cli lint --explain
# Run CI gate with score threshold
npx agenteval-cli ci --min-score 0.7 --max-regression 0.05
# Watch mode for continuous linting
npx agenteval-cli watch
# Auto-fix filler phrases
npx agenteval-cli lint --fix