UTF-8 Text Encoding Polyfill

Common errors

• Error: Unknown encoding: 'iso-8859-1'

  cause The `text-encoding-utf-8` polyfill is strictly limited to UTF-8 and does not support other character encodings.
  fix Modify your code to only use 'utf-8' or 'UTF-8' as the encoding string. If other encodings are required, this polyfill is not suitable, and you should use a more comprehensive library or native APIs.

• ReferenceError: TextEncoder is not defined

  cause The `TextEncoder` (or `TextDecoder`) global object was not made available. This can happen if the polyfill script was not properly loaded, or if the import mechanism didn't expose the globals as expected.
  fix Ensure the polyfill script (`encoding.js`) is loaded in HTML, or for Node.js/bundlers, verify the import statement is correct (`import 'text-encoding-utf-8';` for global patching, or `import { TextEncoder } from 'text-encoding-utf-8';` for named imports).

Warnings

• gotcha This polyfill only supports UTF-8. Attempting to use any other encoding (e.g., 'iso-8859-1', 'gbk') will result in an error or unexpected behavior, as it will fall back to UTF-8 without explicit error.
  Fix: Ensure that only 'utf-8' or 'UTF-8' are passed as encoding arguments to TextEncoder and TextDecoder constructors. If other encodings are needed, use a comprehensive polyfill or check for native API support.
• deprecated The package is effectively unmaintained, with the last publish over seven years ago. Modern browsers (Chrome, Firefox, Edge, Safari) and Node.js environments have native, highly optimized implementations of TextEncoder and TextDecoder. Relying on this polyfill can introduce unnecessary bundle size and potential compatibility issues.
  Fix: For new projects, avoid this polyfill entirely. For existing projects, consider removing it and relying on native browser/Node.js APIs. Use feature detection (`typeof TextEncoder !== 'undefined'`) if targeting very old environments that lack native support.
• gotcha Using this polyfill in environments that already have native `TextEncoder` and `TextDecoder` can lead to conflicts, unexpected behavior, or simply redundant code execution.
  Fix: Implement robust feature detection to conditionally load or apply the polyfill only when `TextEncoder` or `TextDecoder` are genuinely missing. For example: `if (typeof TextEncoder === 'undefined') { require('text-encoding-utf-8'); }`.

Install

• npm install text-encoding-utf-8 npm
• yarn add text-encoding-utf-8 yarn
• pnpm add text-encoding-utf-8 pnpm

Imports

• TextEncoder
  wrong:

  const { TextEncoder, TextDecoder } = require('text-encoding-utf-8');

  correct:

  import { TextEncoder, TextDecoder } from 'text-encoding-utf-8';

  While CommonJS `require` might work, the package's age (v1.x, 7+ years old) predates widespread native ESM support in Node.js. For browser environments, `<script src="encoding.js"></script>` makes globals available.
• TextDecoder
  wrong:

  const TextDecoder = require('text-encoding-utf-8').TextDecoder;

  correct:

  import { TextEncoder, TextDecoder } from 'text-encoding-utf-8';

  A direct require of the package likely returns an object with both `TextEncoder` and `TextDecoder` properties, or it makes them global. Named imports are the most explicit way if using a bundler.
• Global Polyfill

  import 'text-encoding-utf-8';

  In older environments or when using the provided `encoding.js` script in HTML, importing the package as a side-effect (or loading the script) is intended to make `TextEncoder` and `TextDecoder` available globally.

Quickstart

Demonstrates basic UTF-8 encoding and decoding using TextEncoder and TextDecoder, including a conceptual example of streaming decode.

import { TextEncoder, TextDecoder } from 'text-encoding-utf-8';

// Basic UTF-8 Encoding
const encoder = new TextEncoder('utf-8');
const originalString = 'Hello, world! 👋';
const encodedBytes = encoder.encode(originalString);

console.log('Original String:', originalString);
console.log('Encoded Bytes (Uint8Array):', encodedBytes);

// Basic UTF-8 Decoding
const decoder = new TextDecoder('utf-8');
const decodedString = decoder.decode(encodedBytes);

console.log('Decoded String:', decodedString);

// Streaming Decode (example concept - next_chunk needs implementation)
let streamDecoder = new TextDecoder('utf-8');
let resultString = '';

// Imagine 'chunk1', 'chunk2', etc., are Uint8Arrays from a stream
const chunk1 = encoder.encode('This is the first part. ');
const chunk2 = encoder.encode('And this is the second part.');

resultString += streamDecoder.decode(chunk1, { stream: true });
resultString += streamDecoder.decode(chunk2, { stream: true });
resultString += streamDecoder.decode(); // Finish the stream

console.log('Stream Decoded String:', resultString);