reg-cli: Visual Regression Test Tool

Common errors

• Error: Node.js version too old. `reg-cli` requires Node.js v18 or higher.

  cause The installed Node.js version is below the minimum requirement (v18).
  fix Update your Node.js environment to version 18 or newer. For example, using `nvm install 18 && nvm use 18`.

• Error: No images found to compare.

  cause The specified 'actual' directory is empty or contains no images matching the expected comparison format.
  fix Ensure that your `actual` image directory contains the images you intend to compare. Verify the paths provided to `reg-cli` are correct.

• Command failed with exit code 2 (or similar non-zero code) after image comparison.

  cause By default, `reg-cli` exits with an error code if visual differences, new images, or deleted images are detected.
  fix If this is an expected outcome (e.g., in a CI pipeline where differences are allowed but reported), add the `--ignoreChange` flag to the `reg-cli` command. Use `--extendedErrors` in addition to `--ignoreChange` if you also want to ignore errors for added/deleted images.

• Error: Argument 'threshold' can't be used anymore. Use 'thresholdRate' or 'thresholdPixel'.

  cause You are using the deprecated `--threshold` option in a version where it has been removed or strictly aliased.
  fix Replace `--threshold` with `--thresholdRate` (e.g., `--thresholdRate 0.05`) or `--thresholdPixel` (e.g., `--thresholdPixel 10`) depending on your desired comparison metric.

Warnings

• breaking The `--threshold` option was deprecated and replaced by `--thresholdRate` and `--thresholdPixel` to provide more granular control over change detection.
  Fix: Migrate your commands to use `--thresholdRate` and/or `--thresholdPixel` instead of `--threshold`. `--threshold` is now an alias for `--thresholdRate`.
• breaking Node.js v18 or higher is now required. Older Node.js versions are no longer supported, which may cause installation or runtime errors.
  Fix: Upgrade your Node.js environment to version 18 or newer using a tool like `nvm` or by installing a recent Node.js distribution.
• gotcha By default, `reg-cli` will exit with a non-zero status code if image changes are detected (diff images are created).
  Fix: To prevent `reg-cli` from throwing an error and exiting non-zero when changes are detected, use the `--ignoreChange` (or `-I`) option. If you also want to ignore errors for added/deleted images, use `--extendedErrors` (or `-E`) in combination.
• gotcha reg-cli switched its underlying image difference library from `image-diff` to `argos-ci/image-difference` due to `image-diff` no longer being maintained. This change may subtly affect comparison results.
  Fix: Review your visual regression tests after upgrading to ensure the new comparison engine yields expected results. Adjust `--matchingThreshold`, `--thresholdRate`, or `--thresholdPixel` if necessary.
• gotcha Recent versions include security updates for dependencies like `serve-static` and `tar`. Running older versions might expose your project to known vulnerabilities.
  Fix: Regularly update `reg-cli` to its latest patch version to ensure all dependency vulnerabilities are addressed. Use `npm audit` to check for specific issues.

Install

• npm install reg-cli npm
• yarn add reg-cli yarn
• pnpm add reg-cli pnpm

Imports

• RegCli
  wrong:

  import { RegCli } from 'reg-cli';

  correct:

  import RegCli from 'reg-cli';

  The primary class for programmatic usage is exported as a CommonJS default. ESM imports work via interop.
• RegCliOptions
  wrong:

  import { RegCliOptions } from 'reg-cli';

  correct:

  import type { RegCliOptions } from 'reg-cli';

  Type definition for the RegCli class constructor options, primarily used in TypeScript projects.
• RegResult
  wrong:

  import { RegResult } from 'reg-cli';

  correct:

  import type { RegResult } from 'reg-cli';

  Type definition for the object returned by RegCli operations, useful for type-checking in TypeScript.

Quickstart

This quickstart demonstrates how to run `reg-cli` from a Node.js script to compare images, generate a diff, and output an HTML report. It sets up dummy directories and files, then executes the CLI command with common options.

import { spawn } from 'node:child_process';
import { mkdir, writeFile } from 'node:fs/promises';
import { join } from 'node:path';

const actualDir = './screenshots/actual';
const expectedDir = './screenshots/expected';
const diffDir = './screenshots/diff';
const reportPath = './reg-report/index.html';

async function setupDirs() {
  await mkdir(actualDir, { recursive: true });
  await mkdir(expectedDir, { recursive: true });
  await mkdir(diffDir, { recursive: true });
  // Simulate creating a dummy actual image
  await writeFile(join(actualDir, 'test.png'), Buffer.from('iVBORw0KGgoAAAANSUhEUgAAAAEAAAABCAQAAAC1HAwCAAAAC0lEQVR42mNkYAAAAAYAAjCB0C8AAAAASUVORK5CYII=', 'base64'));
  // Simulate creating a dummy expected image for comparison
  await writeFile(join(expectedDir, 'test.png'), Buffer.from('iVBORw0KGgoAAAANSUhEUgAAAAEAAAABCAQAAAC1HAwCAAAAC0lEQVR42mNkYAAAAAYAAjCB0C8AAAAASUVORK5CYII=', 'base64'));
}

async function runRegCli() {
  await setupDirs();

  const args = [
    actualDir,
    expectedDir,
    diffDir,
    '-R', reportPath,
    '--matchingThreshold', '0.01',
    '--thresholdRate', '0.001'
  ];

  const regCliProcess = spawn('npx', ['reg-cli', ...args], { stdio: 'inherit' });

  regCliProcess.on('close', (code) => {
    if (code === 0) {
      console.log('reg-cli comparison completed successfully. Check report at', reportPath);
    } else {
      console.error(`reg-cli exited with code ${code}. Visual differences might have been found.`);
    }
  });
}

runRegCli().catch(console.error);