Puppeteer: Headless Browser Automation

Common errors

• Error: Failed to launch the browser process! No browser found at...

  cause Puppeteer could not find a Chrome/Chromium executable at the expected path. This often happens if `puppeteer-core` is used without `executablePath`, or the `puppeteer` installation failed to download the browser.
  fix If using `puppeteer`, try reinstalling it to redownload the browser (`npm install puppeteer`). If using `puppeteer-core`, you must specify the `executablePath` in `puppeteer.launch()` to point to your browser binary.

• Error: browserContext.newPage: Target closed.

  cause You attempted to interact with a browser, page, or frame that has already been closed or navigated away from, invalidating the context.
  fix Ensure that `browser.close()` or `page.close()` is called only after all necessary operations on that context are complete. Check for asynchronous operations that might complete after a page or browser is closed.

• Error: Navigation timeout of 30000 ms exceeded

  cause A navigation action (`page.goto()`, `page.click()` followed by navigation, etc.) took longer than the default 30-second timeout.
  fix Increase the timeout for the specific navigation call (`await page.goto(url, { timeout: 60000 });`) or globally (`page.setDefaultNavigationTimeout(60000);`). Ensure the network and server are responsive.

• Error: function: Cannot find context with specified id

  cause This usually indicates that an `ElementHandle` or another browser-side object is being used after the page has reloaded, navigated, or the element itself no longer exists in the DOM.
  fix After any significant page navigation or DOM manipulation, re-select elements and obtain new `ElementHandle`s rather than reusing old ones. Ensure your element references are always fresh.

• Error: The 'engines' field in 'package.json' specifies that this package is compatible only with Node.js >=18.

  cause Your current Node.js version is older than the minimum required by Puppeteer.
  fix Upgrade your Node.js version to 18 or higher. You can use tools like `nvm install 18` and `nvm use 18`.

Warnings

• gotcha The default `puppeteer` package automatically downloads a compatible Chromium browser, which can significantly increase installation time and disk space. For smaller installations or to use an existing browser, consider `puppeteer-core`.
  Fix: Install `puppeteer-core` instead: `npm i puppeteer-core`. You will then need to provide the `executablePath` option to `puppeteer.launch()`.
• breaking Puppeteer requires Node.js version 18 or higher. Using an older Node.js version will result in installation or runtime errors.
  Fix: Upgrade your Node.js environment to version 18 or newer. Use a Node.js version manager like nvm (Node Version Manager) for easy switching.
• gotcha When running Puppeteer in environments like Docker containers or CI/CD pipelines, you may encounter errors related to sandboxing. Disabling the sandbox with `--no-sandbox` is often necessary but introduces security risks.
  Fix: Add `--no-sandbox` to the `args` array in `puppeteer.launch()`: `puppeteer.launch({ args: ['--no-sandbox', '--disable-setuid-sandbox'] })`. For production, investigate a more secure sandboxing configuration for your environment.
• gotcha Puppeteer frequently updates to support the latest Chrome/Firefox versions. Mismatches between your installed Puppeteer version and the browser it controls can lead to unexpected behavior or broken features due to DevTools Protocol changes.
  Fix: Regularly update your Puppeteer package (`npm update puppeteer`) to ensure compatibility with the browser it launches. If using `puppeteer-core` with an external browser, ensure that browser is also up-to-date.
• gotcha Puppeteer's custom selectors like `::-p-aria()`, `::-p-text()`, and `::-p-xpath()` offer powerful ways to locate elements but are not standard CSS or XPath. Relying heavily on them may make your tests less portable outside Puppeteer.
  Fix: While powerful, consider supplementing custom selectors with standard CSS selectors (`page.locator('div.my-class')`) or `page.$eval()` for robust and widely understood element selection, especially for stable UI components.

Install

• npm install puppeteer npm
• yarn add puppeteer yarn
• pnpm add puppeteer pnpm

Imports

• puppeteer
  wrong:

  const puppeteer = require('puppeteer');

  correct:

  import puppeteer from 'puppeteer';

  ESM is preferred, but CommonJS is also supported.

Quickstart

This example launches a headless Chrome browser, navigates to a URL, interacts with elements using Puppeteer's custom selectors, extracts text content, and then closes the browser.

import puppeteer from 'puppeteer';

(async () => {
  const browser = await puppeteer.launch();
  const page = await browser.newPage();

  await page.goto('https://developer.chrome.com/');

  await page.setViewport({width: 1080, height: 1024});

  await page.keyboard.press('/');

  await page.locator('::-p-aria(Search)').fill('automate beyond recorder');

  await page.locator('.devsite-result-item-link').click();

  const textSelector = await page
    .locator('::-p-text(Customize and automate)')
    .waitHandle();
  const fullTitle = await textSelector?.evaluate(el => el.textContent);

  console.log('The title of this blog post is "%s".', fullTitle);

  await browser.close();
})();