Web Stream Tools

Common errors

• Error: Cannot find module 'web-stream-tools'

  cause The package name has changed from 'web-stream-tools' to '@openpgp/web-stream-tools'.
  fix Change the dependency in `package.json` and all import paths in your code to `@openpgp/web-stream-tools`.

• TypeError: stream.transform is not a function

  cause Incorrect import statement or module not found, leading to 'stream' being undefined or an empty object.
  fix Ensure you are using `import * as stream from '@openpgp/web-stream-tools';` (for namespace import) or `import { transform } from '@openpgp/web-stream-tools';` (for named import).

• Error: The provided stream is not a WhatWG ReadableStream.

  cause You are passing a native Node.js `stream.Readable` object to a function that expects a WhatWG `ReadableStream`.
  fix If in Node.js v17+, convert your Node stream using `myNodeStream.toWeb()` before passing it to `web-stream-tools` functions. For older Node.js versions, you may need to manually wrap the Node stream or use a polyfill.

Warnings

• breaking The official package name for Web Stream Tools has been changed from `web-stream-tools` to `@openpgp/web-stream-tools`. Direct imports using the old package name will result in `Module not found` errors or `undefined` imports.
  Fix: Update your `package.json` dependencies and all import statements in your code from `web-stream-tools` to `@openpgp/web-stream-tools`.
• breaking As of v0.1.0, the library no longer supports native Node.js `Readable` or `Writable` streams directly as input or output for many functions. It now exclusively expects and operates on WhatWG `ReadableStream` and `WritableStream` instances (often referred to as Node's WebStreams in modern Node.js).
  Fix: For Node.js environments, ensure you are using Node.js v17+ which provides built-in utilities like `Readable.toWeb()` for converting native Node streams to Web Streams, or `Readable.fromWeb()` for the reverse. Alternatively, transform your data into `Uint8Array` or `String` before passing to functions like `transform` if stream-to-stream conversion is not feasible.
• gotcha Attempting to use CommonJS `require()` syntax with `@openpgp/web-stream-tools` might not work as expected or could lead to bundling issues. The library is primarily designed for ES Module (`import`) usage with Web Streams, especially given its browser-compatibility focus.
  Fix: Always prefer `import * as stream from '@openpgp/web-stream-tools';` or named imports in ES Modules. If CommonJS is strictly required, ensure your build tool (e.g., Webpack, Rollup) or Node.js configuration (e.g., `"type": "module"` in `package.json`) handles ESM to CJS interoperability correctly.

Install

• npm install web-stream-tools npm
• yarn add web-stream-tools yarn
• pnpm add web-stream-tools pnpm

Imports

• * as stream
  wrong:

  import stream from 'web-stream-tools';

  correct:

  import * as stream from '@openpgp/web-stream-tools';

  The library's official package name is `@openpgp/web-stream-tools`. Older references or custom build configurations might use `web-stream-tools` which will result in import errors or `undefined` imports. The primary export pattern is a namespace import (`* as stream`).
• transform
  wrong:

  import { transform } from 'web-stream-tools';

  correct:

  import { transform } from '@openpgp/web-stream-tools';

  Individual functions can be tree-shaken and imported directly. Ensure the correct package name `@openpgp/web-stream-tools` is used.
• readToEnd
  wrong:

  const { readToEnd } = require('@openpgp/web-stream-tools');

  correct:

  import { readToEnd } from '@openpgp/web-stream-tools';

  While Node.js may support CommonJS `require` for ESM packages via interoperability layers, the library is designed for modern ESM usage. Prefer `import` statements for better compatibility and tooling support.

Quickstart

This example demonstrates how to use the `stream.transform` function to encrypt data flowing through a WhatWG ReadableStream, including setup for an imaginary encryptor and reading the output.

import * as stream from '@openpgp/web-stream-tools';

// Imagine an encryptor API with process and finish methods
class Encryptor {
  process(chunk) {
    console.log('Processing chunk:', chunk.byteLength, 'bytes');
    // Simulate encryption, return a new Uint8Array
    return new Uint8Array(chunk.map(b => b ^ 0xFF));
  }
  finish() {
    console.log('Finishing encryption.');
    // Return any final encrypted bytes, or an empty Uint8Array
    return new Uint8Array([0xDE, 0xAD, 0xBE, 0xEF]);
  }
}

async function encryptDataStream() {
  const encryptor = new new Encryptor();
  const inputData = new TextEncoder().encode("This is a test string to be encrypted using web-stream-tools.");
  
  // Create a ReadableStream from the input data
  const input = new ReadableStream({
    start(controller) {
      controller.enqueue(inputData);
      controller.close();
    }
  });

  const encryptedStream = stream.transform(input, function process(chunk) {
    return encryptor.process(chunk);
  }, function finish() {
    return encryptor.finish();
  });

  // Read the encrypted stream to the end and collect all chunks
  const reader = encryptedStream.getReader();
  let allChunks = [];
  while (true) {
    const { done, value } = await reader.read();
    if (done) {
      break;
    }
    allChunks.push(value);
  }
  const finalEncryptedData = new Uint8Array(allChunks.reduce((acc, chunk) => acc.concat(Array.from(chunk)), []));
  console.log('Total encrypted data length:', finalEncryptedData.byteLength, 'bytes');
  console.log('Encryption complete.');
}

encryptDataStream().catch(console.error);