Superbench Load Testing Framework

0.0.2 verified Thu Apr 23 auth: no javascript

Superbench is a lightweight load testing framework for Node.js, designed to handle distributed test execution and consolidate results. It enables users to define complex test scenarios directly in JavaScript, offering significant flexibility as it does not impose any specific API client dependencies, allowing it to test virtually any server. Currently at version 0.0.2, Superbench is in its early development stages, suggesting a relatively infrequent release cadence as features mature towards a stable 1.0 release. Key differentiators include its built-in support for master/slave clustering to distribute load efficiently and its focus on providing detailed performance metrics aggregated from various workers. This makes it suitable for scenarios requiring high concurrency across multiple client machines, offering a clear alternative to single-machine load testers.

Common errors

error SyntaxError: Cannot use import statement outside a module ↓

cause Your benchmark definition file is being interpreted as a CommonJS module, but it uses `import` statements.

fix

Ensure your benchmark file has a .mjs extension, or add "type": "module" to your project's package.json.

error superbench: command not found ↓

cause The Superbench CLI tool is not installed globally or is not in your system's PATH.

fix

Install Superbench globally using npm install -g superbench.

error Error: No results within measurementInterval ↓

cause This error occurs in Superbench versions <=0.0.1 when no successful test results are recorded within a specified measurement interval, which was a bug.

fix

Upgrade Superbench to version 0.0.2 or newer (npm update superbench) to fix this issue. Also, ensure your test scenario is correctly designed to produce results.

Warnings

gotcha Superbench is in early alpha (v0.0.2). The API and internal mechanisms are subject to change in future releases, potentially introducing breaking changes without major version bumps. ↓

fix Refer to the official GitHub repository for the latest documentation and release notes.

gotcha Benchmark definition files (`.js` or `.ts`) are expected to be treated as ECMAScript Modules (ESM). Using CommonJS `require()` syntax or running without `type: "module"` in your `package.json` for ESM files will result in errors. ↓

fix Ensure your benchmark file uses `.mjs` extension or your project's `package.json` includes `"type": "module"`. Always use `import` statements.

breaking Versions prior to 0.0.2 could encounter an error if no successful requests were recorded within a `measurementInterval`, leading to test instability. ↓

fix Upgrade Superbench to version `0.0.2` or later to resolve this measurement bug.

Install

npm install superbench

yarn add superbench

pnpm add superbench

Imports

defineBenchmark
wrong
```
const defineBenchmark = require("superbench");
```
correct
```
import defineBenchmark from "superbench";
```
Benchmark definition files are expected to be ESM modules (e.g., .mjs or 'type: module' in package.json).

Quickstart

Defines a distributed load test scenario using `superbench`, including parallel tasks and performance measurement for an arbitrary API.

import defineBenchmark from "superbench";

defineBenchmark({
    title: "someAPI",
    description: "check some API",
    concurrentRequestNum: 10,
    measurementInterval: 30,  // sec
    duration: 180 // sec
}, async benchmark => {
    // You need write a scenario code for parallelization that is run in a worker.
    const tasks = [];
    // superbench calculates request num of this worker.
    // You can get it from `benchmark.requestNum`.
    for (let i = 0; i < benchmark.requestNum; i++) {
        tasks.push(new Promise(async resolve => {
            // If you run loop processing, you need to check `benchmark.running`.
            while (benchmark.running) {
                const t1 = benchmark.test("healthcheck1");
                try {
                    // Replace 'yourAPIClient.callSomeAPI()' with your actual API call
                    // For demonstration, we'll simulate an async operation.
                    await new Promise(r => setTimeout(r, Math.random() * 50 + 100)); 
                    t1.success();
                } catch (e) {
                    t1.error(e);
                }
            }
            resolve();
        }));
    }
    await Promise.all(tasks);
});