{"id":11318,"library":"micromark-util-subtokenize","title":"micromark Subtokenization Utility","description":"micromark-util-subtokenize is a low-level utility within the micromark ecosystem, designed to handle the internal process of tokenizing subtokens during markdown parsing. It provides the `subtokenize` function, which is crucial for extending micromark with custom parsing behaviors, particularly for nested content or specific markdown constructs. This package is currently at version 2.1.0 and is actively maintained as part of the unified collective's micromark project. Releases are typically coordinated with other micromark packages, following a frequent cadence in response to bug fixes, performance improvements, and new feature development within the core parser. Its primary differentiator is its specialized role in the micromark parsing pipeline, making it essential for advanced extension authors rather than direct end-user application.","status":"active","version":"2.1.0","language":"javascript","source_language":"en","source_url":"https://github.com/micromark/micromark#main","tags":["javascript","micromark","util","utility","tokenize","typescript"],"install":[{"cmd":"npm install micromark-util-subtokenize","lang":"bash","label":"npm"},{"cmd":"yarn add micromark-util-subtokenize","lang":"bash","label":"yarn"},{"cmd":"pnpm add micromark-util-subtokenize","lang":"bash","label":"pnpm"}],"dependencies":[],"imports":[{"note":"This package is ESM-only and does not provide a CommonJS export. There is no default export, so curly braces are required.","wrong":"const subtokenize = require('micromark-util-subtokenize')","symbol":"subtokenize","correct":"import { subtokenize } from 'micromark-util-subtokenize'"}],"quickstart":{"code":"import {subtokenize} from 'micromark-util-subtokenize'\nimport type {Event, Resolver} from 'micromark-util-types'\n\n/**\n * A resolver function type from micromark-util-types.\n * For custom micromark extensions, resolvers process a list of events.\n * In this example, `subtokenize` is called on the events to parse subcontent.\n *\n * @type {Resolver}\n */\nfunction resolveContent(events: Array<Event>): Array<Event> {\n  // `subtokenize` processes the event stream to tokenize any nested content.\n  // This makes sure that, for instance, definitions are parsed before\n  // their references in paragraphs.\n  subtokenize(events)\n  return events\n}","lang":"typescript","description":"This example demonstrates how `subtokenize` is typically used within a micromark resolver function to process nested token events."},"warnings":[{"fix":"Ensure your project is configured for ESM (e.g., `\"type\": \"module\"` in `package.json`) or use dynamic `import()` if you must consume it from CommonJS.","message":"This package is ESM-only. Attempting to `require()` it in a CommonJS environment will result in an error.","severity":"breaking","affected_versions":">=1.0.0"},{"fix":"Consult the `micromark` extension development guide before using this utility directly. Most users should interact with `micromark` through its main API or higher-level utilities.","message":"This package is an internal utility for `micromark` and is generally not intended for direct use by consumers unless developing custom `micromark` extensions. Incorrect usage outside its intended context can lead to unexpected parsing behavior or errors.","severity":"gotcha","affected_versions":">=1.0.0"},{"fix":"Upgrade your Node.js environment to version 16 or newer and ensure your `micromark` dependency is version 3 or higher for full compatibility.","message":"The package is compatible with Node.js 16+ and `micromark@3`. Using it with older Node.js versions or incompatible `micromark` major versions may lead to runtime issues or unexpected behavior.","severity":"breaking","affected_versions":"<=1.x"},{"fix":"Always use the latest patch release of `micromark-util-subtokenize` (currently 2.1.0) to ensure stability and benefit from bug fixes and performance improvements.","message":"Version 2.0.3 included a refactor to omit a spread operator that was causing crashes. Earlier versions might exhibit stability issues related to this, particularly in specific environments or with complex markdown structures.","severity":"gotcha","affected_versions":"<2.0.3"}],"env_vars":null,"last_verified":"2026-04-19T00:00:00.000Z","next_check":"2026-07-18T00:00:00.000Z","problems":[{"fix":"Switch to `import` statements and ensure your project or file is treated as an ES module. For example, add `\"type\": \"module\"` to your `package.json` or use `.mjs` file extensions.","cause":"Attempting to import an ESM-only package using CommonJS `require()`.","error":"Error [ERR_REQUIRE_ESM]: require() of ES Module .../node_modules/micromark-util-subtokenize/index.js from ... not supported."},{"fix":"Ensure you are using named imports: `import { subtokenize } from 'micromark-util-subtokenize'`. This package has no default export.","cause":"This typically occurs if `micromark-util-subtokenize` was imported incorrectly, e.g., using `import subtokenize from '...'`, or if it's an environment issue.","error":"TypeError: subtokenize is not a function"}],"ecosystem":"npm"}