Japanese Character Encoding Conversion

Common errors

• TypeError: Converting circular structure to JSON

  cause Attempting to `JSON.stringify` an array of character codes (especially Uint8Array) without proper serialization, or passing non-serializable objects to functions.
  fix Ensure that if you need to serialize character code arrays, you convert them to a JSON-compatible format first, such as a regular array of numbers (`Array.from(uint8Array)`), or directly to a string via `Encoding.codeToString` before stringifying.

• Cannot find module 'encoding-japanese'

  cause Incorrect import path or the package is not installed correctly, especially in environments with strict module resolution or when using specific build tools.
  fix Verify the package is installed (`npm install encoding-japanese`) and ensure your import statement is `import * as Encoding from 'encoding-japanese';` for ESM or `const Encoding = require('encoding-japanese');` for CommonJS. Check your bundler or TypeScript configuration if paths are being resolved incorrectly.

• Unrepresentable character error or incorrect character output after conversion.

  cause Occurs when a character from the 'from' encoding cannot be represented in the 'to' encoding, and the `fallback` option is not handled or is set to 'error'.
  fix Use the `fallback` option in `Encoding.convert` (e.g., `fallback: 'html-entity'`, `fallback: 'ignore'`, or `fallback: 'error'`) to explicitly define how unrepresentable characters should be handled. If 'error' is used, wrap the conversion in a `try...catch` block to gracefully handle the error.

Warnings

• breaking Version 2.0.0 introduced breaking changes by adding a `fallback` option to `Encoding.convert`. Previously, unrepresentable characters might have been handled differently or implicitly, but now explicit handling through `fallback` options ('html-entity', 'ignore', 'error') is available. While adding functionality, existing code relying on previous implicit behavior for unrepresentable characters might need adjustment.
  Fix: Review calls to `Encoding.convert` and explicitly define a `fallback` option (e.g., `fallback: 'html-entity'`, `fallback: 'ignore'`, or `fallback: 'error'`) to match desired behavior for unrepresentable characters.
• gotcha JavaScript strings are internally UTF-16. `encoding-japanese` primarily works with arrays of character codes (e.g., `Uint8Array`). Direct string arguments to `convert` or `detect` are not the primary use case and might lead to unexpected results if not correctly converted to/from character code arrays first using `stringToCode` and `codeToString`.
  Fix: Always convert JavaScript strings to character code arrays (e.g., `Encoding.stringToCode(myString)`) before passing them to `Encoding.convert` or `Encoding.detect` if you intend to specify a non-UTF-16 'from' encoding. Similarly, convert the resulting code arrays back to strings using `Encoding.codeToString`.
• gotcha When using `Encoding.convert`, the `type` option determines the return format. Omitting it or using `type: 'array'` returns a plain number array, while `type: 'Uint8Array'` returns a TypedArray. Mixing these types in subsequent operations without explicit conversion can lead to errors.
  Fix: Be explicit about the `type` option in `Encoding.convert` (e.g., `type: 'Uint8Array'`) if you expect a TypedArray. Ensure consistency in data types when chaining operations or explicitly convert between plain arrays and TypedArrays as needed.

Install

• npm install encoding-japanese npm
• yarn add encoding-japanese yarn
• pnpm add encoding-japanese pnpm

Imports

• Encoding
  wrong:

  import Encoding from 'encoding-japanese';

  correct:

  import * as Encoding from 'encoding-japanese';

  The library exports a namespace object, not a default export. Use `* as Encoding` for all methods.
• Encoding.detect
  wrong:

  import { detect } from 'encoding-japanese';

  correct:

  import { detect } from 'encoding-japanese/lib/encoding';

  While `* as Encoding` is common, individual methods can be imported directly from their `lib` paths for tree-shaking benefits, though the `lib/encoding` path is less intuitive.
• Encoding.convert
  wrong:

  const { convert } = require('encoding-japanese');

  correct:

  const Encoding = require('encoding-japanese');
  const convertedData = Encoding.convert(...);

  For CommonJS, `require('encoding-japanese')` returns the full `Encoding` object. Destructuring individual methods directly from the top-level package is not directly supported without deeper path imports.

Quickstart

Demonstrates converting Shift_JIS byte array to UTF-8 string, detecting encoding, and URL encoding, highlighting core library functionalities.

import * as Encoding from 'encoding-japanese';

// Example: Convert a Shift_JIS string to UTF-8
const shiftJisBytes = new Uint8Array([
  0x82, 0xA0, 0x82, 0xA2, 0x82, 0xA4, 0x82, 0xA6, 0x82, 0xA8 // 'あいうえお' in Shift_JIS
]);

const utf8Bytes = Encoding.convert(shiftJisBytes, {
  to: 'UTF8',
  from: 'SJIS',
  type: 'array'
});

const utf8String = Encoding.codeToString(utf8Bytes);
console.log('Original Shift_JIS (bytes):', shiftJisBytes);
console.log('Converted UTF-8 (bytes):', utf8Bytes);
console.log('Converted UTF-8 (string):', utf8String);

// Example: Detect encoding
const detectedEncoding = Encoding.detect(shiftJisBytes);
console.log('Detected encoding:', detectedEncoding);

// Example: URL encoding
const urlEncoded = Encoding.urlEncode('テスト&123');
console.log('URL Encoded:', urlEncoded);