Imhotap Test Runner

2.2.0 verified Thu Apr 23 auth: no javascript

Imhotap is a lightweight, 'meta-test-framework' designed to run test files as independent processes and aggregate their results. It operates on the principle that each test file should be executable and indicate success or failure via its exit code. The tool automatically detects and parses Test Anything Protocol (TAP) output from test files, but is agnostic to the specific underlying test framework used, supporting any TAP producer like `tape`, `node-tap`, or `pitesti`. Key differentiators include its parallel execution model (defaulting to CPU cores - 1 concurrency), support for custom runners (e.g., `ts-node` for TypeScript files), and flexible glob-based file selection. The current stable version is 2.2.0, with a focus on stability and compatibility rather than frequent breaking changes.

Common errors

error Error: Command failed: <reporter_command> ↓

cause The specified TAP reporter (via `-r`) was not found in the system's PATH or `node_modules/.bin`.

fix

Ensure the reporter package is installed globally or as a devDependency. If locally installed, ensure npx can find it or add it to your PATH.

error Error: Command failed: chmod EACCES ↓

cause A test file was attempted to be executed directly, but it lacks execute permissions.

fix

For shell scripts or other executable formats, run chmod +x <file> on the test file. Alternatively, specify a runner using -R (e.g., -R bash for a shell script).

error No files found matching glob pattern: 'test/**/*.spec.js' ↓

cause The provided glob pattern did not match any files in the project directory.

fix

Double-check the glob pattern for typos or incorrect paths. Ensure files actually exist at the specified location relative to where imhotap is run.

Warnings

gotcha Imhotap runs test files in parallel as separate processes. If your tests are not isolated and share resources (e.g., databases, global state, specific ports), you may encounter race conditions or unexpected failures. Adjust concurrency (`-c`) to 1 if tests require serial execution. ↓

fix Ensure test files are fully isolated and do not conflict when run simultaneously. If isolation is not feasible, use the `-c 1` option to run tests serially.

gotcha When using a custom runner (`-R/--runner`), ensure the runner is installed and accessible in the system's PATH, or within the project's `node_modules/.bin`. If not found, `imhotap` will attempt to use `npx`, but this might fail or be slower. ↓

fix Install the runner globally (`npm i -g <runner>`) or locally as a `devDependency` (`npm i -D <runner>`). For local installs, `npx` or a `package.json` script wrapping `imhotap` is recommended.

gotcha Imhotap expects JavaScript files (`.js`) to be executable by Node.js by default. For other file types (e.g., `.ts`, `.sh`, Python scripts), you must explicitly provide a `--runner` option, or ensure the files are marked as executable (e.g., `chmod +x file.sh`) and have a shebang. ↓

fix For non-JavaScript files, use `-R <runner_command>` (e.g., `-R tsnode`, `-R bash`). For executable scripts, ensure they have a shebang (`#!/usr/bin/env bash`) and execute permissions.

gotcha Imhotap expects TAP output from test files. If a test file's `stdout` or `stderr` does not contain valid TAP data, `imhotap` will display both streams inside a YAML block, which may be less parseable by TAP reporters. It prioritizes `stdout` over `stderr` for TAP detection by default. ↓

fix Ensure your test framework outputs valid TAP to `stdout`. If your framework outputs to `stderr` and you prefer that, use `--stream stderr`. Consult `testanything.org` for TAP specification.

Install

npm install imhotap

yarn add imhotap

pnpm add imhotap

Imports

cli
wrong
```
const cli = require('imhotap').cli;
```
correct
```
import { cli } from 'imhotap'; // Programmatic CLI execution
```
While primarily a CLI tool, Imhotap offers a programmatic interface for its CLI function, `cli`, allowing integration into custom scripts. Prefer ESM imports.
TapRunner
wrong
```
const TapRunner = require('imhotap').TapRunner;
```
correct
```
import { TapRunner } from 'imhotap'; // Programmatic runner class
```
The `TapRunner` class provides a more granular programmatic way to define and execute test runs without invoking the full CLI process. Use ESM imports.
Running Imhotap via CLI
wrong
```
node imhotap --files 'test/**/*.js'
```
correct
```
imhotap --files 'test/**/*.js'
```
Imhotap is intended to be run directly as a command-line tool after global installation or via `npx` / `package.json` scripts, not typically invoked via `node` directly.

Quickstart

This quickstart demonstrates how to set up two basic TAP-producing test files (one passing, one failing) and run them with `imhotap` using a glob pattern.

mkdir -p my-project/test
echo 'console.log("TAP version 13")' > my-project/test/basic.js
echo 'console.log("ok 1 - basic test")' >> my-project/test/basic.js

echo 'console.log("TAP version 13")' > my-project/test/failing.js
echo 'console.log("not ok 1 - failing test")' >> my-project/test/failing.js
echo 'process.exit(1)' >> my-project/test/failing.js

cd my-project
npx imhotap --files 'test/**/*.js'