Benny Benchmarking Framework

3.7.1 · active · verified Sun Apr 19

Benny (version 3.7.1) is a straightforward benchmarking framework for JavaScript and TypeScript, designed to simplify the process of performance measurement in Node.js environments. It abstracts the complexity of the underlying `benchmark.js` library, offering a more intuitive API for defining and running benchmarks. Key features include native support for synchronous and asynchronous code, per-case setup and teardown functionalities, the ability to selectively run or skip test cases, and robust result handling with options to save output to JSON, CSV, or interactive HTML charts. It provides sensible defaults, but allows for detailed configuration when needed, and integrates well with IDEs through built-in type definitions. Benny targets Node.js version 12 and above and maintains a stable release cadence. Its primary differentiator is simplifying the API over raw `benchmark.js` while retaining its powerful measurement capabilities.

Common errors

```
TypeError: b.suite is not a function
```
cause The 'b' object was not correctly imported or required, or an attempt was made to destructure methods directly from the 'benny' package.

fix Use `const b = require('benny')` for CommonJS or `import b from 'benny'` for ESM, and then call methods like `b.suite()`, `b.add()`, etc.
```
Error: Cannot find module 'benny'
```
cause The 'benny' package has not been installed in the project.

fix Install the package using `npm install benny --save-dev` or `yarn add benny --dev`.
```
Benchmark results show very low operations per second (ops/s) or high error margins.
```
cause The benchmarked code includes I/O operations, network requests, or other factors external to the CPU-bound logic, or the benchmark runs are too short to stabilize.

fix Isolate the critical section of code for benchmarking. Use `b.beforeEach()` and `b.afterEach()` for setup/teardown that should not be timed. Adjust `minTime` or `minSamples` options within `b.add` or `b.configure` for longer, more stable runs if needed.

Warnings

gotcha Benchmarking results are highly sensitive to the execution environment (hardware, operating system, Node.js version, other running processes). Compare results only between identical environments and ensure isolation for accurate measurements.
Fix: Run benchmarks on dedicated machines or in controlled CI/CD environments. Consider running multiple times to observe consistency.
gotcha When benchmarking asynchronous code, ensure that `beforeEach`, `afterEach`, and the benchmark function itself correctly handle promises or callbacks. Improper handling can lead to inaccurate timings or incomplete execution. Benny is designed to handle async, but user implementation must be correct.
Fix: Return promises from async setup/teardown functions and benchmark cases. Ensure all async operations complete before the measurement for a given cycle is considered done.
gotcha Forgetting to include `b.cycle()` and `b.complete()` in your suite definition will prevent the benchmark from running or showing output. `b.cycle()` logs progress, and `b.complete()` finalizes the suite and prints the summary.
Fix: Always include `b.cycle()` and `b.complete()` after your `b.add()` calls within the `b.suite()` definition.
breaking While not explicitly noted as a breaking change in the provided README for version 3.x, older versions of Node.js (below 12) are not supported. Using Benny with an unsupported Node.js version may lead to unexpected errors or behavior.
Fix: Ensure your Node.js environment is version 12 or newer, as specified in the package's engine requirements.

Install

npm install benny npm
yarn add benny yarn
pnpm add benny pnpm

Imports

b
wrong:
```
import { suite, add, cycle } from 'benny'
```
correct:
```
import b from 'benny'
```
Benny exports a default object (`b`) containing its methods. Direct named imports for `suite`, `add`, etc., are incorrect as of v3.
b
wrong:
```
const { suite, add, cycle } = require('benny')
```
correct:
```
const b = require('benny')
```
In CommonJS, require the package directly to get the main 'b' object with all benchmarking utilities. Attempting to destructure individual methods will result in undefined values.
suite
wrong:
```
suite('My Suite', /* ... */)
```
correct:
```
b.suite('My Suite', /* ... */)
```
Methods like `suite`, `add`, `cycle`, `complete`, and `save` are properties of the default exported object `b` and must be accessed via `b.methodName`.

Quickstart

This quickstart demonstrates defining a benchmark suite with two cases, executing them, printing results to the console, and saving the results to a JSON file and an interactive HTML chart.

/* benchmark.js */
const b = require('benny')

b.suite(
  'Example Reduction',

  b.add('Reduce two elements', () => {
    ;[1, 2].reduce((a, b) => a + b)
  }),

  b.add('Reduce five elements', () => {
    ;[1, 2, 3, 4, 5].reduce((a, b) => a + b)
  }),

  b.cycle(),
  b.complete(),
  b.save({ file: 'reduce', version: '1.0.0' }),
  b.save({ file: 'reduce', format: 'chart.html' }),
)

view raw JSON →