Tree-sitter Language Pack

1.5.0 · active · verified Thu Apr 09

tree-sitter-language-pack is a Python library providing pre-compiled Tree-sitter parsers for over 300 programming languages. It offers a unified `process()` API for efficient parsing, advanced code analysis, and intelligent code chunking. The library is actively maintained with frequent releases, typically introducing new language grammars and API enhancements.

Warnings

gotcha Prior to v1.4.2, installation documentation, especially for the CLI (`ts-pack-cli`), contained incorrect package names and references, leading to potential installation issues or confusion.
Fix: Ensure you are using the correct package name `tree-sitter-language-pack` for Python and refer to the latest documentation for CLI installation (`cargo install ts-pack-cli` or `brew install kreuzberg-dev/tap/ts-pack`).
gotcha Versions prior to v1.3.3 experienced issues with dynamic parser loading and `c_symbol` overrides for specific languages (e.g., C#, VB, Embedded Template, Nushell), potentially causing runtime errors.
Fix: Upgrade to v1.3.3 or newer to resolve issues with dynamic loading and `c_symbol` overrides for affected languages.
gotcha While direct access to `tree-sitter.Language` and `tree-sitter.Parser` objects via `get_language()` and `get_parser()` is available, the `process()` function with `ProcessConfig` is the recommended and unified API across all bindings for comprehensive code intelligence, including parsing, analysis, and chunking.
Fix: Prefer using the `process()` API with `ProcessConfig` for most code intelligence tasks to leverage the full capabilities and consistent interface of the library.
gotcha tree-sitter-language-pack aims to simplify Tree-sitter usage by bundling grammars. However, if integrating with other raw `tree-sitter` libraries or manually compiled grammars, be aware that the underlying `tree-sitter` ecosystem can have complex versioning and compilation requirements that might lead to conflicts or unexpected behavior.
Fix: Rely primarily on `tree-sitter-language-pack` for all your grammar needs to maintain consistency. If interoperability is required, carefully manage `tree-sitter` versions and compilation flags.

Install

pip install tree-sitter-language-pack Pip
uv add tree-sitter-language-pack Uv

Imports

process
```
from tree_sitter_language_pack import process, ProcessConfig
```
The `process` function with `ProcessConfig` is the recommended unified API for comprehensive parsing and analysis.
get_language
```
from tree_sitter_language_pack import get_language, get_parser
```
These functions provide direct access to `tree-sitter.Language` and `tree-sitter.Parser` objects, primarily for lower-level interaction if `process()` is not sufficient.

Quickstart

This quickstart demonstrates the core `process()` API to extract structured intelligence from source code, including functions, diagnostics, and comments. It also shows how to use AST-aware chunking for breaking down larger code blocks. The library automatically downloads necessary language parsers on demand.

from tree_sitter_language_pack import process, ProcessConfig
import os

source_code = """
def hello():
    # This is a comment
    print("Hello, World!")

class MyClass:
    def __init__(self):
        pass
"""

# Process source code for intelligence extraction (auto-downloads language if needed)
result = process(source_code, ProcessConfig(language="python"))

print(f"Functions found: {len(result.get('structure', []))}")
print(f"Diagnostics: {result.get('diagnostics', [])}")
print(f"Comments: {result.get('comments', [])}")

# Example with AST-aware chunking
chunked_result = process(
    source_code,
    ProcessConfig(language="python", chunk_max_size=50, comments=True)
)
print(f"Chunks found: {len(chunked_result.get('chunks', []))}")

view raw JSON →