Node.js Micro-benchmarking Suite

Common errors

• Dead code elimination detected. Result of benchmark might be inaccurate.

  cause The V8 engine has optimized away or partially eliminated the code being benchmarked because its return value or side effects are not used.
  fix Ensure the benchmarked function's result is used or has observable side effects. For example, assign the result to a variable that is then used in a conditional `if (result !== expected) throw new Error('Unexpected');` or return the result from the benchmark function. Enable `detectDeadCodeElimination` in suite options to get these warnings.

• TypeError: (0, _bench_node.Suite) is not a constructor

  cause This error typically indicates an incorrect import statement, often when attempting to use a CommonJS `require()` syntax with a module that is primarily designed for ES Modules (ESM) or when a named export is treated as a default export.
  fix If using ES Modules, ensure you're using `import { Suite } from 'bench-node';`. If in a CommonJS environment, use `const { Suite } = require('bench-node');` to correctly destructure the named export. Ensure your build configuration (e.g., TypeScript, Babel) handles module interop correctly.

• Benchmark results show extremely high ops/sec (e.g., millions or billions) for trivial operations, seeming unrealistic.

  cause The V8 JIT compiler has likely optimized away the benchmarked code entirely or partially because it detects no observable side effects or the return value isn't used, leading to misleadingly fast results.
  fix Modify your benchmark code to ensure that the operations performed have a visible effect or that their results are consumed. For instance, return the result, store it in an external variable (if not part of the timed operation), or perform a simple check like `if (result === undefined) throw new Error();`. Using the `detectDeadCodeElimination` option can help identify such cases.

Warnings

• gotcha Bench-node's primary mechanism uses V8 deoptimization to make microbenchmarks more stable and reproducible. However, this means the code under test is explicitly prevented from undergoing V8's real-world optimizations, leading to 'accurate but not realistic' performance numbers.
  Fix: Always interpret microbenchmark results with caution. For real-world performance assessments, consider profiling under production-like conditions or using macro-benchmarks that allow V8 to optimize code naturally.
• gotcha Dead Code Elimination (DCE) can significantly skew benchmark results if the V8 engine optimizes away portions of your benchmarked code because its output or side effects are not utilized. While `bench-node` provides an opt-in DCE detection plugin, enabling it disables the `V8NeverOptimizePlugin`.
  Fix: Ensure that the results of your benchmarked operations are always used, for example, by returning them, asserting against them, or logging them, to prevent V8 from eliminating the code. Refer to the 'Dead Code Elimination Detection' section in the documentation for specific patterns to avoid.
• breaking In `v0.13.0`, reporter modules were refactored to separate formatting logic from stdout printing. Direct use of reporters that previously printed to stdout might now require calling explicit `to<Format>()` methods (e.g., `toText()`, `toJson()`) for programmatic consumption.
  Fix: Review existing code that directly invokes reporter functions. If you were relying on implicit stdout printing, adapt your code to use the new `to<Format>()` methods or pass the reporter to the `Suite` options directly.
• gotcha When using the `timer` argument for manual timing within a benchmark function, the setup code executed before `timer.start()` can also be deoptimized by V8, potentially impacting the accuracy of the setup phase's performance.
  Fix: Isolate resource-intensive setup logic outside the timed block by performing it before `suite.add()`, or carefully consider if manual timing is truly necessary and understand its implications.
• gotcha Statistical significance testing (t-test mode) was introduced in v0.14.0. While useful for comparing benchmarks, it automatically sets `repeatSuite=30`, potentially increasing benchmark execution time significantly.
  Fix: Be aware of the increased execution time when enabling `ttest: true`. Adjust `repeatSuite` manually if fewer repetitions are acceptable for preliminary analysis, or ensure your CI/CD pipeline can accommodate longer runtimes for statistically robust comparisons.

Install

• npm install bench-node npm
• yarn add bench-node yarn
• pnpm add bench-node pnpm

Imports

• Suite
  wrong:

  const Suite = require('bench-node').Suite;

  correct:

  import { Suite } from 'bench-node';

  The primary entry point for creating benchmark suites. The `require` syntax is also widely supported for CJS environments, as shown in the package's documentation.
• textReport
  wrong:

  const textReport = require('bench-node').textReport;

  correct:

  import { textReport } from 'bench-node';

  One of the built-in reporters. Since v0.13.0, reporters also export `to<Format>` functions for programmatic use instead of direct stdout printing.
• chartReport
  wrong:

  const chartReport = require('bench-node').chartReport;

  correct:

  import { chartReport } from 'bench-node';

  A reporter that outputs benchmark results as a bar chart. As with `textReport`, it also exports `toChart` since v0.13.0.

Quickstart

This example demonstrates how to set up a `Suite` with a custom reporter, add multiple benchmarks, and run them. It compares different array population methods and explicitly uses the result to prevent dead code elimination.

import { Suite, chartReport } from 'bench-node';

const suite = new Suite({
  // Optionally configure the reporter
  reporter: chartReport,
  reporterOptions: {
    printHeader: true // Controls whether system info header is printed
  }
});

suite.add('Using Array.push', () => {
  const arr = [];
  for (let i = 0; i < 1000; i++) {
    arr.push(i);
  }
  return arr.length; // Ensure result is used to prevent DCE
});

suite.add('Using Array literal concatenation', () => {
  let arr = [];
  for (let i = 0; i < 1000; i++) {
    arr = [...arr, i];
  }
  return arr.length; // Ensure result is used to prevent DCE
});

suite.add('Using direct index assignment', () => {
  const arr = [];
  for (let i = 0; i < 1000; i++) {
    arr[i] = i;
  }
  return arr.length; // Ensure result is used to prevent DCE
});

suite.run();