miniget - Small HTTP(S) GET Client

Common errors

• ReferenceError: require is not defined

  cause Attempting to use `require()` in an ES module (ESM) context, typically indicated by `"type": "module"` in `package.json` or using `.mjs` file extensions.
  fix If working in an ESM environment, use `import miniget from 'miniget';` instead of `const miniget = require('miniget');`.

• TypeError: miniget is not a function

  cause Incorrect import statement. This usually happens when `miniget` is treated as a named export (`import { miniget } from 'miniget';`) when it is a default export, or when attempting an incorrect CommonJS import in an ESM file.
  fix For ESM, use `import miniget from 'miniget';`. For CommonJS, use `const miniget = require('miniget');`.

• Stream error: Invalid URL

  cause The URL string provided to `miniget` is malformed, not a valid `URL` object, or lacks a proper protocol (e.g., `http://` or `https://`).
  fix Ensure the URL parameter is a correctly formatted string (e.g., `https://example.com/resource`) or a valid `URL` object instance.

• Error: The 'close' event is no longer emitted from the miniget stream.

  cause Your code is still attempting to listen for the `close` event on the `miniget` stream, which was removed as a breaking change in v4.0.0.
  fix Remove the `stream.on('close', ...)` listener from your code. If you need to detect stream completion, use `stream.on('end', ...)` or monitor the completion of downstream writable streams (e.g., `fileStream.on('finish', ...)`) if you are piping the data.

Warnings

• breaking The `close` event is no longer emitted from the miniget stream since v4.0.0. This change was implemented to prevent altering the stream's internal state and ensure proper finishing, aligning with Node.js stream best practices.
  Fix: Replace `stream.on('close', ...)` listeners with `stream.on('end', ...)` for the miniget stream or `fileStream.on('finish', ...)` if piping to a writable stream.
• breaking The `abort()` method was deprecated with a runtime warning in v3.0.0 and superseded by the `destroy()` method. The `destroy()` method was introduced to align more closely with Node.js's native HTTP request and stream destruction patterns.
  Fix: Update all calls from `stream.abort()` to `stream.destroy()` to use the current API.
• gotcha Prior to v4.2.1, `miniget` had a bug where it internally required the `url` module, which could cause issues when bundling with webpack for browser environments, leading to build failures or runtime errors.
  Fix: Upgrade to `miniget` v4.2.1 or a newer version to resolve webpack compatibility issues.
• gotcha `miniget` is built for Node.js environments and declares `"node": ">=12"` in its `engines` field. Using it in older Node.js versions (e.g., Node.js 10 or 8) may result in unexpected behavior or runtime errors due to reliance on newer JavaScript features or Node.js APIs.
  Fix: Ensure your Node.js runtime environment is version 12 or higher to guarantee compatibility and stability.

Install

• npm install miniget npm
• yarn add miniget yarn
• pnpm add miniget pnpm

Imports

• miniget
  wrong:

  import { miniget } from 'miniget';

  correct:

  import miniget from 'miniget';

  The primary export is a default function. For CommonJS, `const miniget = require('miniget');` works directly.
• MinigetError
  wrong:

  import MinigetError from 'miniget';

  correct:

  import { MinigetError } from 'miniget';

  A named export for the custom error class, available since v2.1.0, useful for catching specific miniget errors.
• Defaults

  import { Defaults } from 'miniget';

  A named export for the global default options object, available since v2.1.0, allowing programmatic modification of default request settings.

Quickstart

This quickstart demonstrates both promise-based text retrieval and streaming a response to a file using `miniget`, including error handling and event listeners for redirects and responses. It also shows how to configure global default options.

import miniget from 'miniget';
import { createWriteStream } from 'node:fs';

async function fetchAndStreamData(url: string, filePath: string) {
  try {
    // Fetch and concatenate content as text
    const textBody = await miniget(url).text();
    console.log(`Fetched text from ${url.substring(0, 30)}... : ${textBody.substring(0, 50)}...`);

    // Stream content to a file
    const stream = miniget(url, { maxRedirects: 5, maxRetries: 1 });
    stream.on('redirect', (newUrl) => console.log(`Redirected to: ${newUrl}`));
    stream.on('error', (err) => console.error(`Stream error: ${err.message}`));
    stream.on('response', (res) => console.log(`Received status: ${res.statusCode}`));

    const fileStream = createWriteStream(filePath);
    stream.pipe(fileStream);

    await new Promise<void>((resolve, reject) => {
      fileStream.on('finish', () => {
        console.log(`Streamed content to ${filePath}`);
        resolve();
      });
      fileStream.on('error', reject);
    });

  } catch (error: any) {
    console.error(`Failed to fetch or stream: ${error.message}`);
  }
}

// Example usage with a placeholder URL
// Replace with a valid URL for actual execution
const exampleUrl = 'https://jsonplaceholder.typicode.com/posts/1';
const outputFilePath = './output.json';
fetchAndStreamData(exampleUrl, outputFilePath);

// You can also modify global defaults
miniget.defaultOptions.headers = { 'User-Agent': 'miniget-example-app' };
console.log('Global User-Agent set.');