Tree-sitter Python Bindings

0.25.2 · active · verified Sat Mar 28

Tree-sitter provides Python bindings to the core Tree-sitter parsing library, enabling fast, incremental parsing and the generation of concrete syntax trees for various programming languages. It's actively maintained, with frequent updates (minor and patch releases typically every few weeks or months) to keep pace with the underlying C library. The current version is `0.25.2`.

Warnings

breaking The `Language` constructor no longer accepts a raw pointer (`int`) directly but expects a capsule object. If you were dynamically loading a `.so` file, you might need to use `ctypes.CDLL` and extract the language function or adjust your loading mechanism. Additionally, `Parser.parse()` no longer accepts a `keep_text` argument, and range arguments for `Query.captures()` and `Query.matches()` have been removed; use `query.set_byte_range()` or `query.set_point_range()` instead.
Fix: For custom language loading, ensure you're passing a proper `PyCapsule` or using a language binding that provides it. Remove `keep_text` from `parser.parse()` calls. For queries, pre-set the desired range using `query.set_byte_range()` or `query.set_point_range()` before calling `captures()` or `matches()`.
breaking Python 3.9 is no longer supported; the library now requires Python 3.10 or newer. The `Language(ptr: int)` constructor was deprecated, urging users to use language binding packages instead of raw pointers. The method `Node.child_containing_descendant` was also deprecated.
Fix: Upgrade your Python environment to 3.10 or later. Migrate away from `Language(ptr: int)` by using official language binding packages (e.g., `tree-sitter-python`) which provide a `language()` function. Update code using `Node.child_containing_descendant` to equivalent newer methods or tree traversal logic.
gotcha Incompatibilities can arise between the `tree-sitter` Python package and specific `tree-sitter-<language>` grammar packages if their underlying Tree-sitter ABI versions differ. This can lead to runtime errors when loading languages.
Fix: Ensure that the `tree-sitter` package and any `tree-sitter-<language>` packages you use are compatible by checking their release notes or by installing versions known to work together. If encountering issues, try aligning their versions, typically by upgrading/downgrading one or both packages.
gotcha High memory usage has been reported in certain scenarios, particularly when parsing very large files or using complex grammars, potentially indicating memory leaks in specific contexts or substantial resource requirements of the core library.
Fix: Monitor memory usage closely in production. For large inputs, consider processing files in chunks or optimizing your queries to reduce the scope. Report any persistent, unexplainable memory growth as a bug to the project maintainers, providing a minimal reproducible example.
gotcha While many languages have `tree-sitter-<language>` packages, for less common languages or custom grammars, you might need to manually compile the grammar into a shared library (`.so` or `.dll`) and then load it. The `tree-sitter-languages` package, which simplifies loading, is currently unmaintained and may have compatibility issues with newer `tree-sitter` versions.
Fix: For unsupported languages, follow the official Tree-sitter documentation for compiling custom grammars. Then, use `from tree_sitter import Language; MY_LANGUAGE = Language(f'path/to/my_language.so', 'my_language_name')` to load it, ensuring the ABI version matches. Avoid `tree-sitter-languages` in new projects.

Install

pip install tree-sitter Install core library
pip install tree-sitter-python Install Python grammar (example)

Imports

Language
```
from tree_sitter import Language
```
Parser
```
from tree_sitter import Parser
```
Query
```
from tree_sitter import Query
```
QueryCursor
```
from tree_sitter import QueryCursor
```
language
```
import tree_sitter_python as tspython; PY_LANGUAGE = Language(tspython.language())
```
It's best practice to import the language module with an alias and then access its `language()` function to avoid name collisions and for clarity. Direct import of `language` is also common but less explicit.

Quickstart

This quickstart demonstrates how to load a language grammar (Python in this case), create a parser, parse a code string into a syntax tree, and then use Tree-sitter's query system to find specific patterns within the tree, such as function names. It highlights the typical workflow of initializing a `Language` object from a pre-compiled grammar and then using a `Parser` to process source code. Ensure the relevant language package (e.g., `tree-sitter-python`) is installed alongside `tree-sitter`.

import os
from tree_sitter import Language, Parser

# NOTE: For this to work, you need 'tree-sitter-python' installed
# pip install tree-sitter-python
import tree_sitter_python as tspython

# Load the Python language grammar
PY_LANGUAGE = Language(tspython.language())

# Create a parser and set its language
parser = Parser()
parser.set_language(PY_LANGUAGE)

# Source code to parse
code = b""" 
def greet(name): # type: (str) -> None
    print(f"Hello, {name}!")

greet("World")
"""

# Parse the code
tree = parser.parse(code)

# Get the root node and print its type
root_node = tree.root_node
print(f"Root node type: {root_node.type}")

# Traverse the tree (example: find function definitions)
query_string = """ 
(function_definition
  name: (identifier) @function.name)
"""
query = PY_LANGUAGE.query(query_string)

captures = query.captures(root_node)
for node, name in captures:
    print(f"Found function: {node.text.decode('utf8')} (capture: {name})")

view raw JSON →