MongoDB Assistant Evaluation Library

An evaluation library for the MongoDB Assistant API, version 0.0.8, designed to support dataset schemas and loaders, evaluation scorers, metrics, and a programmatic CLI helper for building evaluation pipelines. It integrates with Braintrust for running evals, OpenAI for tool call types, and Zod for schema validation. Optional peer dependencies enable LLM-as-judge scorers (via ai, autoevals), MongoDB dataset loading, and Slack reporting. It is actively developed by MongoDB, with TypeScript types included. Key differentiators include a focus on conversation evaluation with built-in scorers like Factuality and Faithfulness, and flexible dataset loaders from YAML, JSON, or MongoDB.