robotstxt-webpack-plugin

8.0.0 verified Sat Apr 25 auth: no javascript

A webpack plugin to generate a robots.txt file for your project. Version 8.0.0 is current, released August 2022. It requires webpack 5+ and Node.js >=14.15.0. Built on top of generate-robotstxt, it supports all options from that package. Simplified setup compared to manual creation, integrates with webpack's asset pipeline. Last update was 2022; no recent activity, but still functional for webpack 5 projects.

Common errors

error Error: Cannot find module 'robotstxt-webpack-plugin' ↓

cause Package not installed or not in node_modules.

fix

Run npm install --save-dev robotstxt-webpack-plugin

error TypeError: RobotstxtWebpackPlugin is not a constructor ↓

cause Using old class name RobotstxtWebpackPlugin which was renamed in v5.

fix

Use new class name RobotstxtPlugin.

error Module not found: Error: Can't resolve 'generate-robotstxt' ↓

cause Missing underlying dependency when using certain options.

fix

Ensure generate-robotstxt is installed: npm install generate-robotstxt

Warnings

breaking Minimum supported webpack version is 5 ↓

fix Upgrade webpack to version 5 or later.

breaking Minimum supported Node.js version is 14.15.0 ↓

fix Upgrade Node.js to version 14.15.0 or later.

breaking In version 5.0.0, the plugin was renamed from RobotstxtWebpackPlugin to RobotstxtPlugin ↓

fix Use the new class name RobotstxtPlugin instead of RobotstxtWebpackPlugin.

deprecated Old class name RobotstxtWebpackPlugin is no longer available ↓

fix Replace RobotstxtWebpackPlugin with RobotstxtPlugin.

gotcha CommonJS require does not work as expected in some bundler setups ↓

fix Use const RobotstxtPlugin = require('robotstxt-webpack-plugin'); (default export directly).

Install

npm install robotstxt-webpack-plugin

yarn add robotstxt-webpack-plugin

pnpm add robotstxt-webpack-plugin

Imports

RobotstxtPlugin
wrong
```
import RobotstxtPlugin from 'robotstxt-webpack-plugin';
```
correct
```
const RobotstxtPlugin = require('robotstxt-webpack-plugin');
```
Plugin is exported as CommonJS default. For ESM, use the require or use default export workaround.

RobotstxtPlugin

import RobotstxtPlugin from 'robotstxt-webpack-plugin';

Available as default export for ESM users.

default

wrong

const RobotstxtPlugin = require('robotstxt-webpack-plugin').default;

correct

const { default: RobotstxtPlugin } = require('robotstxt-webpack-plugin');

In CJS, the default export is the plugin itself, not under .default. But due to ESM interop, some setups may need this pattern.

Quickstart

Configures webpack to output a robots.txt disallowing all crawlers.

const RobotstxtPlugin = require('robotstxt-webpack-plugin');

module.exports = {
  plugins: [
    new RobotstxtPlugin({
      // Generates a robots.txt that disallows all
      userAgent: '*',
      disallow: '/',
    }),
  ],
};