micromark Utility to Resolve Subtokens

2.0.1 · active · verified Sun Apr 19

micromark-util-resolve-all is a core utility package within the micromark ecosystem, designed to facilitate the complex process of resolving subtokens that cannot be parsed in a straightforward left-to-right manner. This is crucial for Markdown constructs such as attention (e.g., strong and emphasis) and media (links and images), where opening and closing delimiters might be matched non-sequentially or need post-processing after initial tokenization. The package is currently at version 2.0.1 and is part of the micromark project, which typically sees frequent, granular updates across its many utility packages, often released in lock-step with micromark's major versions. As an internal utility, it's not typically intended for direct end-user consumption but rather for developers extending micromark's parsing capabilities. Its primary differentiator is its role in orchestrating the event manipulation necessary for correct CommonMark parsing of non-linear syntax structures.

Common errors

Warnings

Install

Imports

Quickstart

This quickstart demonstrates how to use the `resolveAll` function by providing mock micromark events, constructs, and context. It shows how `resolveAll` orchestrates custom resolvers within constructs to process and transform the event stream, crucial for parsing non-linear Markdown syntax elements.

import { resolveAll } from 'micromark-util-resolve-all';

// These types are simplified for demonstration. In a real micromark setup,
// these would be imported from micromark-util-types and other micromark core packages.
type MicromarkEvent = [string, { type: string; start: { offset: number }; end: { offset: number }; [key: string]: any }];
type MicromarkConstruct = {
  resolveAll?: (events: MicromarkEvent[], context: MicromarkTokenizeContext) => MicromarkEvent[];
  [key: string]: any;
};
type MicromarkTokenizeContext = {
  parser: {
    constructs: {
      insideSpan: {
        null: MicromarkConstruct[];
      };
      [key: string]: any;
    };
  };
  // Other context properties would be here in a real scenario
  defineSkip: () => {};
  events: MicromarkEvent[];
  previous: number;
  sliceStream: (start: {offset: number}, end: {offset: number}) => string;
};

// 1. Define some dummy events. In micromark, these represent tokens parsed from Markdown.
const inputEvents: MicromarkEvent[] = [
  ['enter', { type: 'paragraph', start: { offset: 0 }, end: { offset: 0 } }],
  ['enter', { type: 'text', start: { offset: 0 }, end: { offset: 5 }, value: 'Hello' }],
  ['exit', { type: 'text', start: { offset: 0 }, end: { offset: 5 } }],
  ['enter', { type: 'strongSequence', start: { offset: 6 }, end: { offset: 8 }, _open: true }], // e.g., '**'
  ['enter', { type: 'text', start: { offset: 8 }, end: { offset: 13 }, value: 'World' }],
  ['exit', { type: 'text', start: { offset: 8 }, end: { offset: 13 } }],
  ['exit', { type: 'strongSequence', start: { offset: 13 }, end: { offset: 15 }, _close: true }], // e.g., '**'
  ['exit', { type: 'paragraph', start: { offset: 15 }, end: { offset: 15 } }],
];

// 2. Define dummy constructs. `resolveAll` takes a list of constructs, 
// some of which may contain their own `resolveAll` logic for post-processing.
const dummyConstructs: MicromarkConstruct[] = [
  {
    name: 'emphasis',
    resolveAll: (events: MicromarkEvent[], context: MicromarkTokenizeContext): MicromarkEvent[] => {
      console.log('  -> Custom emphasis resolver called within resolveAll. Events count:', events.length);
      // In a real scenario, this resolver would match openers and closers (like `*` or `**`)
      // and transform `attentionSequence` events into `emphasis` or `strong` nodes.
      return events.map(event => {
        if (event[1].type === 'strongSequence') {
          return [event[0], { ...event[1], type: 'strongToken' }];
        }
        return event;
      });
    }
  },
  { name: 'link' } // A construct without a custom `resolveAll`
];

// 3. Define a dummy context. In a real micromark parser, this object holds state.
const dummyContext: MicromarkTokenizeContext = {
  parser: { constructs: { insideSpan: { null: dummyConstructs } } },
  defineSkip: () => {},
  events: inputEvents, // Often, context.events would be the input events
  previous: -1,
  sliceStream: () => ''
};

console.log('--- Before resolveAll ---');
inputEvents.forEach(e => console.log(`  ${e[0].padEnd(5)} ${e[1].type}`));

// 4. Call `resolveAll` with the constructs, events, and context.
// It iterates through the provided constructs and calls their `resolveAll` methods.
const outputEvents = resolveAll(
  dummyContext.parser.constructs.insideSpan.null, // Typically operates on constructs relevant to inline content
  inputEvents,
  dummyContext
);

console.log('\n--- After resolveAll ---');
outputEvents.forEach(e => console.log(`  ${e[0].padEnd(5)} ${e[1].type}`));

view raw JSON →