ASTTokens

3.0.1 verified Tue May 12 auth: no python install: verified quickstart: verified

ASTTokens is a Python library that annotates Abstract Syntax Trees (ASTs) with the positions of tokens and text in the source code that generated them. This functionality is essential for tools that perform source code transformations, such as automated refactoring or syntax highlighting. The current version is 3.0.1, released on November 15, 2025. The library is actively maintained with a stable release cadence, ensuring ongoing support and updates.

pip install asttokens

Common errors

error ModuleNotFoundError: No module named 'asttokens' ↓

cause The 'asttokens' module is not installed in the Python environment.

fix

Install the module using pip: 'pip install asttokens'.

error ImportError: cannot import name 'formatargspec' from 'inspect' ↓

cause The 'formatargspec' function was removed from the 'inspect' module in Python 3.11, causing compatibility issues with 'asttokens'.

fix

Update 'asttokens' to a version compatible with Python 3.11 or later.

error AttributeError: module 'asttokens' has no attribute 'ASTTokens' ↓

cause The 'ASTTokens' class is not directly accessible from the 'asttokens' module due to incorrect import.

fix

Import the class correctly: 'from asttokens import ASTTokens'.

error ImportError: cannot import name 'ASTText' from 'asttokens' ↓

cause This error typically occurs when using an older version of the `asttokens` library where the `ASTText` class was not directly exposed in the top-level package, while attempting to use code or examples written for a newer API version.

fix

Upgrade asttokens to the latest version (e.g., pip install --upgrade asttokens) to ensure compatibility with recent API changes where ASTText is directly importable.

error AttributeError: 'NoneType' object has no attribute 'body' ↓

cause The `ASTTokens` object was initialized without setting the `parse=True` argument or providing an already parsed AST `tree`, causing its internal `tree` attribute to be `None`. Subsequent attempts to access attributes like `body` on the non-existent tree will fail.

fix

Initialize ASTTokens with parse=True to automatically parse the source code, or explicitly pass an ast.Module object to the tree argument if you've parsed it separately. Example: atok = asttokens.ASTTokens(source, parse=True).

Warnings

breaking ASTTokens requires Python 3.8 or higher. Ensure your environment meets this requirement to avoid compatibility issues. ↓

fix Upgrade to Python 3.8 or a later version.

gotcha ASTTokens may not support certain node types in Python 3.8 due to known issues. Refer to the documentation for a list of unsupported node types. ↓

fix Consider upgrading to a later Python version or consult the documentation for workarounds.

Install compatibility verified last tested: 2026-05-12

python os / libc status wheel install import disk

3.10 alpine (musl) - - 0.01s 17.9M

3.10 slim (glibc) - - 0.01s 18M

3.11 alpine (musl) - - 0.03s 19.8M

3.11 slim (glibc) - - 0.02s 20M

3.12 alpine (musl) - - 0.02s 11.7M

3.12 slim (glibc) - - 0.02s 12M

3.13 alpine (musl) - - 0.02s 11.3M

3.13 slim (glibc) - - 0.02s 12M

3.9 alpine (musl) - - 0.01s 17.4M

3.9 slim (glibc) - - 0.01s 18M

Imports

ASTTokens
```
from asttokens import ASTTokens
```
Direct import of ASTTokens class for annotation tasks
ASTText
```
from asttokens import ASTText
```
Import ASTText for text-based AST annotations
LineNumbers
```
from asttokens import LineNumbers
```
Import LineNumbers for line and column number conversions

Quickstart verified last tested: 2026-04-23

This example demonstrates how to parse and annotate source code using ASTTokens, and how to access the annotated AST and token positions.

import asttokens
import ast

# Sample source code
source = "Robot('blue').walk(steps=10*n)"

# Parse and annotate the source code
atok = asttokens.ASTTokens(source, parse=True)

# Access the annotated AST
tree = atok.tree

# Retrieve the first token of the first node
first_token = tree.body[0].first_token

# Print the start position of the first token
print(f"Start position: {first_token.startpos}")