Textlint Utility: AST Paragraph to String with SourceMap

3.3.4 · active · verified Tue Apr 21

textlint-util-to-string is a utility library within the textlint ecosystem designed to convert Text Abstract Syntax Tree (TxtAST) 'Paragraph' nodes into plain text strings. Its primary distinguishing feature is the integrated SourceMap functionality, which allows developers to accurately map positions in the generated plain text back to their corresponding original positions within the AST. This is crucial for tools like textlint and textstat that require precise error reporting and manipulation based on the original source. The current stable version is 3.3.4, and the package demonstrates an active release cadence with frequent patch and minor updates addressing bug fixes and introducing new features, such as the `replacer` option for text transformation during string conversion. This library serves as a foundational component for building advanced text processing and linting tools.

Common errors

```
TypeError: textlint_util_to_string_1.default is not a constructor
```
cause Attempting to import `StringSource` as a default export when it is now a named export (since v3.0.0). This often happens in CommonJS environments transpiled from ESM.

fix Update your import statement to `import { StringSource } from 'textlint-util-to-string'` for ESM or `const { StringSource } = require('textlint-util-to-string')` for CommonJS.
```
TypeError: StringSource is not a constructor
```
cause Attempting to use `StringSource` as a constructor directly from a default import (`import StringSource from '...'`) after the v3.0.0 breaking change.

fix Modify your import statement to `import { StringSource } from 'textlint-util-to-string'` to use the named export, and ensure your build tooling correctly handles ESM imports.

Warnings

breaking Version 3.0.0 introduced a breaking change by converting the primary export for `StringSource` from a default export to a named export. Code relying on the old import style will break.
Fix: Change your import statements from `import StringSource from 'textlint-util-to-string'` to `import { StringSource } from 'textlint-util-to-string'`.
gotcha When working with position mapping, be mindful of the terminology: `position` refers to `{ line, column }` where `line` is 1-based and `column` is 0-based, while `index` refers to a 0-based offset number. Ensure you use the correct utility method (`originalIndexFromIndex`, `originalPositionFromPosition`, etc.) based on the input type.
Fix: Refer to the API documentation carefully for `position` (line, column) vs. `index` (offset) definitions and their respective 0-based/1-based indexing conventions. Use the specific `original*From*` methods matching your input and desired output.

Install

npm install textlint-util-to-string npm
yarn add textlint-util-to-string yarn
pnpm add textlint-util-to-string pnpm

Imports

StringSource
wrong:
```
import StringSource from 'textlint-util-to-string'
```
correct:
```
import { StringSource } from 'textlint-util-to-string'
```
Since version 3.0.0, StringSource is a named export. Older versions used a default export.

Quickstart

This quickstart demonstrates how to instantiate `StringSource` with a `TxtParagraphNode`, convert it to plain text, use the `replacer` option to mask specific node types, and retrieve original AST positions from the generated string using source map functionality.

import { StringSource } from "textlint-util-to-string";
import { TxtParagraphNode, TxtCodeNode, TxtTextNode } from "@textlint/ast-node-types";

// Simulate a textlint paragraph node with nested content
const paragraphNode: TxtParagraphNode = {
  type: "Paragraph",
  loc: {
    start: { line: 1, column: 0 },
    end: { line: 1, column: 27 },
  },
  range: [0, 27],
  raw: "This is a `code` example.",
  children: [
    {
      type: "Str",
      value: "This is a ",
      loc: { start: { line: 1, column: 0 }, end: { line: 1, column: 10 } },
      range: [0, 10],
      raw: "This is a ",
    } as TxtTextNode,
    {
      type: "Code",
      value: "code",
      loc: { start: { line: 1, column: 11 }, end: { line: 1, column: 15 } },
      range: [11, 15],
      raw: "`code`",
    } as TxtCodeNode,
    {
      type: "Str",
      value: " example.",
      loc: { start: { line: 1, column: 17 }, end: { line: 1, column: 26 } },
      range: [17, 26],
      raw: " example.",
    } as TxtTextNode,
  ],
};

// 1. Basic conversion to plain text
const source = new StringSource(paragraphNode);
console.log("Plain text:", source.toString());
// Expected output: "Plain text: This is a code example."

// 2. Conversion with a replacer function to mask specific node types
const maskedSource = new StringSource(paragraphNode, {
  replacer: ({ node, maskValue }) => {
    if (node.type === "Code") {
      return maskValue("_"); // Masks "code" to "____"
    }
    return undefined; // No change for other node types
  },
});
console.log("Masked text:", maskedSource.toString());
// Expected output: "Masked text: This is a ____ example."

// 3. Demonstrate source map functionality (mapping back from generated string to original AST index)
const plainText = source.toString();
const codeStartIndexInPlainText = plainText.indexOf("code"); // "code" starts at index 10 in "This is a code example."
if (codeStartIndexInPlainText !== -1) {
    const originalIndex = source.originalIndexFromIndex(codeStartIndexInPlainText);
    console.log(`'code' starts at plain text index ${codeStartIndexInPlainText}, original AST index: ${originalIndex}`);
    // Expected: 'code' starts at plain text index 10, original AST index: 11
}

view raw JSON →