Textparser

0.24.0 · active · verified Sat Apr 11

Textparser is a Python library designed for fast text parsing. It allows users to define token specifications using regular expressions and construct grammars to parse text into a structured parse tree. The project prioritizes parsing speed, as highlighted in its benchmarks. The current version is 0.24.0, released on April 16, 2022, with an infrequent release cadence.

Warnings

gotcha When defining token specifications with `(kind, name, re)`, ensure the `grammar` refers to the `name` instead of the `kind`. Using `kind` when `name` is provided will lead to parsing errors.
Fix: If `token_specs` contains `('TYPE', 'friendly_name', r'regex')`, then `grammar` patterns should reference `'friendly_name'` not `'TYPE'`.
gotcha The structure of parse trees returned by `textparser` can vary, and additional post-processing may be required to fit specific application needs. Its primary goal is speed, not necessarily a universally consistent parse tree format across different grammars.
Fix: Always inspect the output parse tree for your specific grammar and implement any necessary transformations or validations on the returned structure.
gotcha The library's last release was April 2022, indicating a slower development pace. While generally stable, users should be aware of potential future compatibility challenges with newer Python versions or lack of updates for new parsing paradigms.
Fix: Monitor the GitHub repository for updates. Test thoroughly with new Python versions or complex scenarios before deploying in production environments. Consider contributing if specific features or fixes are needed.

Install

pip install textparser Install stable version

Imports

Parser
```
from textparser import Parser
```
Primary class for defining a parser.
Sequence
```
from textparser import Sequence
```
Commonly used grammar element for sequential patterns.

Quickstart

This 'Hello World' example demonstrates how to define a custom parser by subclassing `textparser.Parser`. It specifies token types with regular expressions in `token_specs` and defines a simple grammar using `textparser.Sequence` in `grammar` to parse the string 'Hello, World!' into a parse tree.

import textparser
from textparser import Sequence

class MyParser(textparser.Parser):
    def token_specs(self):
        return [
            ('SKIP', r'[ \r\n\t]+'),
            ('WORD', r'\w+'),
            ('EMARK', '!', r'!'),
            ('COMMA', ',', r','),
            ('MISMATCH', r'.')
        ]

    def grammar(self):
        return Sequence('WORD', ',', 'WORD', '!')

tree = MyParser().parse('Hello, World!')
print('Tree:', tree)
# Expected output: Tree: ['Hello', ',', 'World', '!']

view raw JSON →