readdir-glob: Streaming Recursive Directory Reader with Glob Filtering
readdir-glob provides a memory-efficient, recursive file system directory reader with a streaming API, designed as an alternative to `node-glob` for scenarios requiring constant memory usage regardless of the file system size or the number of matched files. It leverages the `minimatch` library for glob pattern filtering. The current stable version is `3.0.0`, which requires Node.js 18 or later. The library ships with TypeScript types since v2.0.0 and supports both CommonJS and ESM. Its primary differentiator is its low, constant memory footprint, making it suitable for processing very large file systems where other glob libraries might exhaust memory. Releases appear to be driven by Node.js version updates and dependency security patches, with major versions aligning with Node.js LTS cycles.
Common errors
-
TypeError: readdirGlob is not a function
cause Attempting to use `readdirGlob` as a named import in ESM when it's a default export (v3.0.0+), or incorrectly importing in CommonJS.fixFor ESM (v3.0.0+), use `import readdirGlob from 'readdir-glob';`. For CommonJS, use `const readdirGlob = require('readdir-glob');`. -
Error: The "path" argument must be of type string or an instance of Buffer or URL. Received undefined
cause The `root` argument for `readdirGlob` was not provided or was `undefined`.fixEnsure the first argument to `readdirGlob` is a valid string representing the directory path to start searching, e.g., `readdirGlob('.', { pattern: '**/*.js' });`. -
No files are being matched, even though they exist.
cause Incorrect glob pattern syntax, or `dot` option not enabled for matching dotfiles, or misconfigured `ignore`/`skip` patterns.fixVerify your `pattern` string. If matching files or directories starting with a `.` (dotfiles), ensure `options.dot` is set to `true`. Check `options.ignore` and `options.skip` patterns to confirm they aren't unintentionally excluding desired files. Note that `ignore` and `skip` patterns always operate in `dot:true` mode.
Warnings
- breaking Node.js version requirements have increased with major releases. v1.x required Node.js 10+, v2.x required Node.js 14+, and v3.x now requires Node.js 18 or later. Ensure your Node.js environment meets the minimum version for the `readdir-glob` version you are using.
- breaking Starting with v3.0.0, the package's ESM module uses a default export for the `readdirGlob` function. If you were using a named import (e.g., `import { readdirGlob } from 'readdir-glob'`) in ESM for previous versions (v2.x), you must change it to a default import (e.g., `import readdirGlob from 'readdir-glob'`).
- gotcha The `stat: true` and `follow: true` options can significantly reduce performance, as they require `readdir-glob` to perform `stat` calls on *all* matched results or follow symlinked directories, respectively. Use these options only when necessary.
- gotcha Understand the difference between `options.ignore` and `options.skip`. `ignore` patterns filter out matched files/folders but do not prevent their contents from being explored. `skip` patterns, however, prevent a matching folder and all its children from being returned or explored, significantly improving performance for large excluded branches.
- breaking The underlying `minimatch` dependency has seen multiple major version bumps (e.g., from 5.x to 9.x in v2.0.0, and 9.x to 10.x in v3.0.0). While `readdir-glob` aims to abstract this, direct `minimatch` behavior (like handling of `.` and `..` in patterns, or specific glob features) might subtly change. `minimatch` versions 9.0.6 and higher require Node.js 18+. `minimatch` versions 10.2.0 and below are vulnerable to ReDoS if patterns contain many consecutive `*` wildcards.
Install
-
npm install readdir-glob -
yarn add readdir-glob -
pnpm add readdir-glob
Imports
- readdirGlob
import { readdirGlob } from 'readdir-glob';import readdirGlob from 'readdir-glob';
- readdirGlob (CommonJS)
const readdirGlob = require('readdir-glob'); - ReaddirGlobOptions
import { ReaddirGlobOptions } from 'readdir-glob';import type { ReaddirGlobOptions, Match } from 'readdir-glob';
Quickstart
import readdirGlob from 'readdir-glob';
import path from 'path';
const rootDir = process.env.ROOT_DIR ?? '.'; // Specify a root directory or use current
const globber = readdirGlob(rootDir, { pattern: '**/*.js', ignore: 'node_modules/**', stat: true });
console.log(`Searching for .js files in '${rootDir}' (excluding node_modules)...`);
globber.on('match', match => {
// match.relative: relative path of the matched file
// match.absolute: absolute path of the matched file
// match.stat: stat of the matched file (only if stat:true option is used)
console.log(`Found: ${match.relative} (Absolute: ${match.absolute})`);
});
globber.on('error', err => {
console.error('Fatal error during globbing:', err);
});
globber.on('end', () => {
console.log('File search completed.');
});
// Example of pausing and resuming the search
setTimeout(() => {
if (!globber.paused) {
globber.pause();
console.log('Search paused for 2 seconds.');
setTimeout(() => {
globber.resume();
console.log('Search resumed.');
}, 2000);
}
}, 1000);