ABNF Parsers

2.4.1 · active · verified Sat Apr 11

ABNF is a Python package designed to generate parsers from Augmented Backus-Naur Form (ABNF) grammars, as specified in RFC 5234 and RFC 7405. While its primary use is to parse data defined in RFCs, it is capable of handling any valid ABNF grammar. The library originated from parsing HTTP headers in a web framework and has since been utilized in production environments. The current version is 2.4.1.

Warnings

breaking Version 2.0.0 introduced significant changes to backtracking implementation, which constituted a breaking change in behavior. Code relying on previous backtracking mechanisms may need adjustment.
Fix: Review grammar definitions and parsing logic for compatibility with the new backtracking implementation. Test thoroughly.
gotcha As of version 2.0.0, `Repetition` objects utilize an LRU cache (`ParseCache`) to store parse results, but the initial implementation does not set maximum sizes for these caches. This can lead to unbounded memory growth and long-term performance issues in applications with extensive or long-running parsing tasks.
Fix: Monitor memory usage and consider contributing feedback or implementing custom cache management if performance degradation is observed. Consult the library's GitHub for potential updates or configuration options related to cache sizing.
gotcha When loading grammars from text files using `Rule.from_file`, ABNF strictly expects CRLF (Carriage Return Line Feed) as a delimiter for rules. Many text editors silently convert CRLF to LF (Line Feed only), which can cause parsing failures or unexpected behavior when the grammar file is read by the library.
Fix: Ensure that grammar files provided to `Rule.from_file` explicitly use CRLF line endings. Verify line endings with a text editor that can display them (e.g., VS Code, Notepad++).
gotcha The library caches `Rule` objects (e.g., `Rule('rule-name')` will return the same object if the rule was previously created). While this behavior typically ensures consistency, the documentation advises against depending on it, suggesting that future versions or edge cases might alter this behavior.
Fix: Avoid relying on `Rule('rule-name')` always returning the identical Python object. Treat rule instantiation as potentially returning a new, but functionally equivalent, object.

Install

pip install abnf Install with pip

Imports

Rule
```
from abnf import Rule
```
rfc7232
```
from abnf.grammars import rfc7232
```
Example of importing a pre-defined RFC grammar.

Quickstart

Demonstrates how to create a custom ABNF rule and parse a string using a pre-defined grammar from the `abnf.grammars` module, such as RFC 7232.

from abnf import Rule
from abnf.grammars import rfc7232

# Example 1: Creating a custom rule and parsing (simplified)
# This demonstrates the structure, actual parsing needs a Rule instance.
custom_rule_grammar_str = 'double-quoted-string = DQUOTE *(%x20-21 / %x23-7E / %x22.22) DQUOTE'
custom_rule_instance = Rule.create(custom_rule_grammar_str)
print(f"Created custom rule: {custom_rule_instance.name}")

# Example 2: Parsing with a pre-defined RFC grammar
src = 'W/"moof"'
# Instantiate the ETag rule from rfc7232 grammar
etag_rule = rfc7232.Rule('ETag')

# Parse the source string
node, start = etag_rule.parse(src)

print(f"\nParsed ETag: {src}")
print(f"Resulting AST node: {node}")
print(f"Parsing started at index: {start}")

view raw JSON →