mdast to nlcst Transformer

7.0.1 · active · verified Sun Apr 19

mdast-util-to-nlcst is a utility in the unifiedjs ecosystem that transforms an mdast (markdown abstract syntax tree) into an nlcst (natural language concrete syntax tree). Currently at stable version 7.0.1, the package follows a semver release cadence with notable breaking changes often accompanying major version bumps. Its core function is to allow inspection of the natural language content within markdown documents by providing a structured NLCST representation. Unlike other utilities like `mdast-util-to-hast` which converts markdown to HTML, this package focuses specifically on language processing. It is frequently used in conjunction with natural language parsers like `parse-english` or `parse-latin` and is wrapped by the `remark-retext` plugin for a higher-level abstraction. A key limitation is that it does not provide functionality to apply changes from the NLCST back into the original mdast tree.

Common errors

```
ERR_REQUIRE_ESM: Must use import to load ES Module: .../node_modules/mdast-util-to-nlcst/index.js
```
cause Attempting to use `mdast-util-to-nlcst` with CommonJS `require()` syntax.

fix Change `const { toNlcst } = require('mdast-util-to-nlcst')` to `import { toNlcst } from 'mdast-util-to-nlcst'` and ensure your project is configured for ES modules (e.g., `"type": "module"` in `package.json` or using a bundler).
```
TypeError: ParseEnglish is not a constructor
```
cause Incorrectly importing the natural language parser (e.g., `parse-english`) as a default export instead of a named export.

fix Ensure the parser is imported using named imports: `import { ParseEnglish } from 'parse-english'` (not `import ParseEnglish from 'parse-english'`).
```
Your environment is not supported: `mdast-util-to-nlcst` requires Node.js `14.14` or later
```
cause Running `mdast-util-to-nlcst` (v6.0.0+) on an unsupported Node.js version.

fix Upgrade your Node.js environment to version 14.14 or higher. The unified collective generally maintains compatibility with active LTS Node.js versions.
```
TypeError: Cannot read properties of undefined (reading 'position') or similar errors related to VFile.
```
cause `toNlcst` was called with a `tree` lacking positional information or a `file` that is not a proper `VFile` instance.

fix Ensure your mdast tree is generated by a parser that includes positional data (like `mdast-util-from-markdown`). Always provide a valid `VFile` object as the `file` argument, which typically holds the original source content and its associated positional data.

Warnings

breaking Version 7.0.0 changed to use `export` maps, which affects how packages are resolved. It also requires updating `@types/mdast`, `@types/nlcst`, and `vfile` to compatible versions.
Fix: Ensure your project's `package.json` correctly resolves `mdast-util-to-nlcst` imports (e.g., in bundlers) and update all related `@types` and `vfile` dependencies to their latest compatible versions. Avoid using private APIs.
breaking Version 6.0.0 dropped support for Node.js 12. The package now requires Node.js 14.14+ or later. Additionally, the API for parser libraries like `parse-latin` and `parse-english` changed, requiring updates to those packages.
Fix: Upgrade your Node.js environment to version 14.14 or newer. Update your natural language parsers (e.g., `parse-english`, `parse-latin`) to their latest versions to match the new API.
breaking Version 5.0.0 transitioned the package to be ESM-only. CommonJS `require()` statements will no longer work and will throw an `ERR_REQUIRE_ESM` error.
Fix: Migrate your project to use ES modules (`import`/`export`) or use a bundler that transpiles ESM to CommonJS if strictly necessary.
gotcha The `toNlcst` function requires the input `tree` to have positional information (e.g., line, column, offset) and the `file` argument to be a `VFile` instance that corresponds to that tree. If this information is missing or inconsistent, the transformation may fail or produce incorrect NLCST.
Fix: Ensure that the mdast tree is generated from a source that provides positional data (e.g., `mdast-util-from-markdown` with a `VFile`) and always pass a valid `VFile` object as the second argument to `toNlcst`.
gotcha This utility is designed for transforming mdast to nlcst for analysis purposes. There is currently no official way to apply changes made to the nlcst tree back into the original mdast tree.
Fix: Understand that `mdast-util-to-nlcst` is a one-way transformation. If you need to modify markdown based on natural language analysis, you will need to implement a separate process to map nlcst changes back to mdast or work directly on the mdast tree using other utilities.

Install

npm install mdast-util-to-nlcst npm
yarn add mdast-util-to-nlcst yarn
pnpm add mdast-util-to-nlcst pnpm

Imports

toNlcst
wrong:
```
const toNlcst = require('mdast-util-to-nlcst')
```
correct:
```
import { toNlcst } from 'mdast-util-to-nlcst'
```
The package is ESM-only since v5.0.0. CommonJS `require` will result in an `ERR_REQUIRE_ESM` error.
Options
```
import type { Options } from 'mdast-util-to-nlcst'
```
TypeScript types are shipped with the package for configuring the `toNlcst` function.
ParseEnglish
wrong:
```
import ParseEnglish from 'parse-english'
```
correct:
```
import { ParseEnglish } from 'parse-english'
```
`parse-english` (and `parse-latin`) are named exports and must be imported correctly. This is a critical dependency for the `Parser` argument of `toNlcst`.

Quickstart

Demonstrates converting markdown content from a VFile into an mdast tree, then transforming it into an nlcst tree using `ParseEnglish`, and inspecting the resulting natural language AST.

import { fromMarkdown } from 'mdast-util-from-markdown'
import { toNlcst } from 'mdast-util-to-nlcst'
import { ParseEnglish } from 'parse-english'
import { read } from 'to-vfile'
import { inspect } from 'unist-util-inspect'
import { VFile } from 'vfile'

async function processMarkdown() {
  // Simulate reading a markdown file
  const markdownContent = 'Some *foo*sball.\n\nA second sentence.'
  const file = new VFile({
    path: 'example.md',
    value: markdownContent
  })

  // Convert markdown string to mdast tree
  const mdast = fromMarkdown(file.value.toString(), { 
    // Extensions might be needed for full markdown features, but not for basic text.
  })

  // Ensure the mdast tree has positional information, which `fromMarkdown` usually provides.
  // The `file` object is crucial here as `toNlcst` expects it.
  const nlcst = toNlcst(mdast, file, ParseEnglish)

  // Inspect the resulting nlcst tree
  console.log(inspect(nlcst))
}

processMarkdown().catch(console.error)

view raw JSON →