Streaming Markdown Parser

Common errors

• TypeError: getMarkdown is not a function

  cause This usually happens when attempting to use CommonJS `require()` syntax or an incorrect named import with a modern ESM-only package.
  fix Ensure you are using standard ESM named imports: `import { getMarkdown } from 'stream-markdown-parser'` and that your environment supports ESM (e.g., Node.js with type:module or a bundler like Webpack/Vite).

• Module not found: Can't resolve 'stream-markdown-parser'

  cause The package is not installed or the path is incorrect for your module resolver.
  fix First, ensure the package is installed: `npm install stream-markdown-parser` or `pnpm add stream-markdown-parser`. If the issue persists, check your bundler configuration (e.g., Vite alias, Webpack resolve paths) to ensure it can correctly locate `node_modules`.

Warnings

• gotcha The package version is currently in 0.0.x, indicating an early development stage. While highly functional, the API surface may still be subject to breaking changes without adhering strictly to semantic versioning (e.g., a minor version bump could introduce breaking changes).
  Fix: Regularly review the GitHub repository's changelog and release notes for updates, especially when upgrading to new versions. Consider pinning to exact versions in production environments.
• gotcha This package is part of a larger monorepository (markstream-vue) and extracts its core parsing logic. While framework-agnostic, its development roadmap and bug fixes might be influenced by the needs of the companion `markstream-vue` and `markstream-react` libraries.
  Fix: Refer to the main `markstream-vue` GitHub repository for the most comprehensive context on development, issues, and future plans, as this package's lifecycle is intertwined with it.
• gotcha While built on `markdown-it-ts` for API compatibility, `stream-markdown-parser` processes tokens into its own stream-friendly AST (`ParsedNode[]`) rather than directly producing HTML strings. If direct HTML rendering is required, you must explicitly call `md.render()` on the underlying `markdown-it-ts` instance.
  Fix: Understand that `parseMarkdownToStructure` returns an AST for progressive rendering, not HTML. If you need HTML, ensure your workflow explicitly uses the `render` method provided by the `markdown-it-ts` instance returned by `getMarkdown()`.

Install

• npm install stream-markdown-parser npm
• yarn add stream-markdown-parser yarn
• pnpm add stream-markdown-parser pnpm

Imports

• getMarkdown
  wrong:

  const getMarkdown = require('stream-markdown-parser').getMarkdown

  correct:

  import { getMarkdown } from 'stream-markdown-parser'

  The library primarily uses named ESM exports. CommonJS `require` is not officially supported and may lead to issues.
• parseMarkdownToStructure
  wrong:

  import parseMarkdownToStructure from 'stream-markdown-parser/dist/parseMarkdownToStructure'

  correct:

  import { parseMarkdownToStructure } from 'stream-markdown-parser'

  Ensure to use named imports from the main package entry point. Direct imports from internal paths are discouraged and unstable.
• setDefaultMathOptions
  wrong:

  import { setDefaultMathOptions } from 'stream-markdown-parser/math'

  correct:

  import { setDefaultMathOptions } from 'stream-markdown-parser'

  Configuration utilities like `setDefaultMathOptions` are also exposed as named exports from the primary package.

Quickstart

This quickstart demonstrates how to use stream-markdown-parser for both incremental (streaming) and static Markdown parsing, outputting a progressively built AST structure. It highlights reusing the parser instance for efficiency.

import { getMarkdown, parseMarkdownToStructure } from 'stream-markdown-parser';

// Create a markdown-it-ts instance with default plugins. Reusing this instance
// avoids redundant plugin registrations for performance.
const md = getMarkdown();

// Imagine receiving markdown content in chunks from a stream (e.g., SSE, AI response).
let bufferedMarkdown = '';

async function simulateStreamingParsing(chunks: string[]) {
  for (const chunk of chunks) {
    bufferedMarkdown += chunk;
    // Parse the accumulated buffer into an Abstract Syntax Tree (AST).
    // This AST is designed to be lightweight and suitable for progressive rendering.
    const nodes = parseMarkdownToStructure(bufferedMarkdown, md);
    console.log('--- Current AST ---');
    // In a real application, 'nodes' would be passed to a renderer (e.g., markstream-vue)
    // to update the UI progressively without re-parsing the entire content from scratch.
    console.log(JSON.stringify(nodes, null, 2).substring(0, 300) + '...');
    await new Promise(resolve => setTimeout(resolve, 50)); // Simulate delay
  }
  console.log('\n--- Final AST ---');
  const finalNodes = parseMarkdownToStructure(bufferedMarkdown, md);
  console.log(JSON.stringify(finalNodes, null, 2));
}

// Example usage: Simulate receiving markdown content in multiple parts.
const markdownChunks = [
  '# Hello ', 'World\n\nThis is a ', '**streaming** ', 'parser example.\n\n- Item 1\n- Item 2\n
```js
console.log("Hello");
```
'];
simulateStreamingParsing(markdownChunks);

// For basic, non-streaming parsing, you can directly render to HTML if needed:
const staticMarkdown = '## Static Content\n\nThis is _just_ a test.';
const staticNodes = parseMarkdownToStructure(staticMarkdown, md);
console.log('\n--- Static AST ---');
console.log(JSON.stringify(staticNodes, null, 2).substring(0, 300) + '...');
// If HTML output is still required, the underlying markdown-it-ts instance can render it.
// const htmlOutput = md.render?.(staticMarkdown); // Uncomment to see HTML output