Playwright TestRail Reporter

Common errors

• AggregateError: Multiple errors occurred...

  cause Malformed or missing environment variables for TestRail configuration, or non-numeric values for ID variables.
  fix Ensure `TESTRAIL_HOST` (including `https://`), `TESTRAIL_USERNAME`, `TESTRAIL_API_KEY`, `TESTRAIL_PROJECT_ID`, and `TESTRAIL_SUITE_ID` are correctly set and formatted in your `.env` file. Upgrade to `v1.3.3` or higher for automatic sanitization.

• playwright-testrail-reporter] ℹ️  No existing 'TESTRAIL_RUN_ID' provided by user. Creating a new TestR... [playwright-testrail-reporter] ❌ Error: Test results could not be added. Check TestRail configuration.

  cause TestRail API key, host, or user credentials are incorrect, leading to authentication or API errors when creating/updating runs.
  fix Verify `TESTRAIL_HOST`, `TESTRAIL_USERNAME`, and `TESTRAIL_API_KEY` are accurate and have the necessary permissions. Double-check for typos and ensure `TESTRAIL_HOST` includes `https://`.

• No test results appearing in TestRail after a Playwright test run completes.

  cause Test cases in Playwright are not tagged with TestRail case IDs (e.g., C12345), or the `playwright-testrail-reporter` is not correctly configured in `playwright.config.ts`.
  fix Add TestRail case IDs in the format `C12345` to your Playwright test names. Ensure `['playwright-testrail-reporter']` is present in the `reporter` array within your `playwright.config.ts`.

Warnings

• deprecated The environment variable `TESTRAIL_PASSWORD` is deprecated. Using it will result in a warning message in the console.
  Fix: Rename `TESTRAIL_PASSWORD` to `TESTRAIL_API_KEY` in your `.env` file and provide a TestRail API key instead of a regular user password.
• gotcha Prior to v1.3.3, incorrect or malformed environment variables (e.g., missing `https://` in `TESTRAIL_HOST`, or non-numeric IDs) could lead to runtime errors like `AggregateError` or 'TestRail Run ID: NaN'.
  Fix: Upgrade to `playwright-testrail-reporter@1.3.3` or newer, which includes smart environment variable validation and auto-sanitization. Always ensure `TESTRAIL_HOST` includes `https://` and IDs are numeric.
• gotcha It is critical to use your TestRail API key for `TESTRAIL_API_KEY`, not your regular login password. Using the password may lead to authentication failures or unexpected behavior.
  Fix: Generate an API key in your TestRail 'My Settings' and use that for the `TESTRAIL_API_KEY` environment variable.
• gotcha For Playwright to report skipped tests to TestRail, you must explicitly define `TESTRAIL_STATUS_SKIPPED` in your environment variables, as it does not have a default mapping.
  Fix: Add `TESTRAIL_STATUS_SKIPPED=6` (or your custom skipped status ID) to your `.env` file.

Install

• npm install playwright-testrail-reporter npm
• yarn add playwright-testrail-reporter yarn
• pnpm add playwright-testrail-reporter pnpm

Imports

• playwright-testrail-reporter
  wrong:

  import { TestRailReporter } from 'playwright-testrail-reporter';

  correct:

  ['playwright-testrail-reporter']

  The reporter is integrated into Playwright's configuration by providing its package name as a string in the `reporter` array within `playwright.config.ts`, not by importing a specific class or function.
• defineConfig
  wrong:

  const defineConfig = require('@playwright/test').defineConfig;

  correct:

  import { defineConfig } from '@playwright/test';

  Standard Playwright configuration helper. Playwright config files (`playwright.config.ts`) are typically run as ESM.

Quickstart

This quickstart demonstrates how to configure `playwright-testrail-reporter` in `playwright.config.ts` and tag tests with TestRail case IDs (e.g., C12345) for automatic reporting. It also highlights the necessary environment variables for TestRail integration.

import { defineConfig } from '@playwright/test';

// To ensure environment variables are loaded if using a .env file
// require('dotenv').config(); // Uncomment if you are using dotenv

export default defineConfig({
  testDir: './tests',
  fullyParallel: true,
  forbidOnly: !!process.env.CI,
  retries: process.env.CI ? 2 : 0,
  workers: process.env.CI ? 1 : undefined,
  reporter: [
    ['list'], // Keep the default list reporter
    ['playwright-testrail-reporter'] // Add TestRail reporter
  ],
  use: {
    baseURL: 'http://127.0.0.1:3000',
    trace: 'on-first-retry',
  },
});

// Example test file (tests/example.spec.ts)
// Make sure to set these environment variables in your .env file:
// TESTRAIL_HOST=https://yourcompany.testrail.io
// TESTRAIL_USERNAME=your.email@company.com
// TESTRAIL_API_KEY=your_api_key_here
// TESTRAIL_PROJECT_ID=123
// TESTRAIL_SUITE_ID=456
// TESTRAIL_RUN_NAME=Automated Playwright Run

import { test, expect } from '@playwright/test';

test('C12345: User can log in with valid credentials', async ({ page }) => {
  await page.goto('/');
  // Simulate login
  console.log('Test C12345 executed.');
  expect(true).toBe(true);
});

test('C67890 C98765: Dashboard loads correctly after login', async ({ page }) => {
  await page.goto('/dashboard');
  // Simulate dashboard checks
  console.log('Test C67890 and C98765 executed.');
  expect(page.url()).toContain('/dashboard');
});