SPARK Parser Toolkit

Common errors

• ModuleNotFoundError: No module named 'spark'

  cause Attempting to import from `spark` instead of `spark_parser`.
  fix Change your import statements from `from spark import ...` to `from spark_parser import ...`.

• RecursionError: maximum recursion depth exceeded

  cause Prior to version 1.9.0, `spark-parser`'s internal `BuildTree` used recursion, which could hit Python's recursion limit with large or deeply nested parse trees. Complex grammars can also indirectly lead to this.
  fix Upgrade to `spark-parser` version 1.9.0 or higher. If the issue persists with a very complex grammar, review your grammar for excessively deep or recursive rules that could contribute to the problem.

• ValueError: Invalid character: '<char>'

  cause Your `GenericScanner` subclass encountered a character in the input string that it doesn't have a rule to tokenize, often due to missing whitespace handling, unhandled special characters, or malformed input.
  fix Review your `Scanner`'s `tokenize` method. Ensure it handles all possible characters in the input, including whitespace, numbers, symbols, and any other valid tokens. Add rules or skip unknown characters explicitly.

Warnings

• gotcha Starting with version 1.9.0, the internal `BuildTree` mechanism was rewritten from recursive to iterative. While this fixes `RecursionError` for large trees, it's a significant internal change that might subtly affect highly specialized code directly interacting with `BuildTree`'s structure or performance characteristics.
  Fix: Review any code that directly interacts with the internal `BuildTree` class. For most users, this change is a beneficial performance and stability improvement, fixing a common `RecursionError`.
• gotcha SPARK parser's compatibility with very old Python versions (e.g., <3.7) may be inconsistent or require specific `spark-parser` versions. Recent releases (1.9.0+) focus on a 'modern Python style' (e.g., type annotations, `pyproject.toml`).
  Fix: It is highly recommended to use Python 3.7.4 or newer with `spark-parser`. Consult the official GitHub releases for specific version compatibility notes if targeting older Python environments.
• gotcha Incorrectly defined grammar rules (e.g., ambiguity, infinite recursion, unreachable productions) can lead to unexpected parsing results, `SyntaxError` exceptions, or infinite loops.
  Fix: Thoroughly test your grammar with diverse inputs. `spark-parser` includes a debug mode; setting `parser._debug = 1` (or higher, up to 5) in your `GenericParser` subclass instance can provide detailed output on parsing steps and rule application, aiding in grammar debugging.

Install

• pip install spark-parser Install stable version

Imports

• GenericParser
  wrong:

  from spark import GenericParser

  correct:

  from spark_parser import GenericParser

  The package name is `spark-parser`, leading to the import path `spark_parser`. Many users mistakenly try to import from `spark` (without `_parser`), possibly confusing it with Apache Spark.
• GenericScanner

  from spark_parser import GenericScanner

Quickstart

This quickstart demonstrates how to define a simple arithmetic scanner and parser using `GenericScanner` and `GenericParser`. It tokenizes an input string and then parses it according to the defined grammar rules to calculate the result.

from spark_parser import GenericParser, GenericScanner

# 1. Define your scanner (lexer) by subclassing GenericScanner
class SimpleCalcScanner(GenericScanner):
    def tokenize(self, input_string):
        tokens = []
        i = 0
        while i < len(input_string):
            char = input_string[i]
            if char.isspace():
                i += 1
                continue
            if char.isdigit():
                num_str = ""
                while i < len(input_string) and input_string[i].isdigit():
                    num_str += input_string[i]
                    i += 1
                tokens.append(('NUMBER', int(num_str)))
            elif char in "+-*/()":
                tokens.append((char, char))
                i += 1
            else:
                raise ValueError(f"Invalid character: {char}")
        return tokens

# 2. Define your parser (grammar rules) by subclassing GenericParser
class SimpleCalcParser(GenericParser):
    def __init__(self, start_symbol='expr'):
        GenericParser.__init__(self, start_symbol)

    # Define grammar rules using docstrings for methods starting with 'p_'
    def p_expr_add(self, args):
        '''
        expr ::= expr + term
        '''
        return args[0] + args[2]

    def p_expr_term(self, args):
        '''
        expr ::= term
        '''
        return args[0]

    def p_term_num(self, args):
        '''
        term ::= NUMBER
        '''
        return args[0]

# 3. Instantiate scanner and parser, then tokenize and parse
scanner = SimpleCalcScanner()
parser = SimpleCalcParser()

text_to_parse = "10 + 5"
tokens = scanner.tokenize(text_to_parse)
result = parser.parse(tokens)

# print(f"Parsed result for '{text_to_parse}': {result}") # Expected: 15
assert result == 15, "Parsing failed!"