fetch-filecache-for-crawling

JSON →
library 5.1.1 ·javascript
verified Jun 7, 2026

Node.js module that wraps native fetch with a file-based HTTP cache for crawling purposes. Version 5.1.1 requires Node >= 18. It caches GET responses in a local folder (default `.cache`) and supports configurable cache expiration strategies (e.g., `once`, `never`, integer seconds). Unlike generic HTTP caches, it is designed for long-running crawls where cached files persist across application runs and conditional requests are sent. It is intentionally narrow in scope — not a general-purpose caching proxy but a specialized tool for crawling with opinionated defaults.