Async Generators and Context Managers

1.10 · active · verified Thu Apr 09

The `async-generator` library provides a backport of asynchronous generators and asynchronous context managers to Python 3.5, which were introduced natively in Python 3.6 (PEP 525) and 3.7. It allows developers to write cleaner asynchronous code for stream processing and I/O-bound tasks using familiar generator syntax. Maintained by the Trio project, it is compatible with any async framework like asyncio or Trio. The current version is 1.10, and it is considered stable, with the last release in July 2018.

Warnings

gotcha Async generators (especially when partially consumed or exited via `break`) do not guarantee `finally` block execution or proper resource cleanup without explicit closing. The garbage collector cannot await cleanup tasks.
Fix: Always wrap async generator iteration in an `aclosing` context manager from this library (or `contextlib.async_closing` in 3.10+). For example: `async with aclosing(my_agen()) as agen: async for item in agen: ...`
breaking For Python 3.5, you *must* use `await yield_(value)` within an `@async_generator` decorated function. Direct `yield value` (native async generator syntax) is a syntax error in Python 3.5 and will not work.
Fix: Ensure all `yield` statements in `@async_generator` decorated functions are replaced with `await yield_()` when targeting Python 3.5. Python 3.6+ supports native `yield` in `async def` functions.
gotcha A native async generator in Python 3.6+ that *only* contains `await yield_from_(...)` calls might not be correctly recognized as an async generator by the Python compiler, leading to unexpected behavior. It needs at least one actual `yield` or `await yield_()` expression.
Fix: Ensure your async generator functions contain at least one explicit `yield` or `await yield_()` statement, even if their primary purpose is to delegate with `await yield_from_()` to another async iterator.
gotcha While `async-generator` supports returning non-`None` values from an `@async_generator` decorated function (similar to `return 'value'` in regular generators), native async generators introduced in Python 3.6 (PEP 525) do not support non-empty `return` statements and will raise a `SyntaxError`. This can cause confusion if mixing patterns.
Fix: When using native async generators (Python 3.6+), ensure they only return `None` implicitly or explicitly. If a value needs to be communicated, yield it as the last item. If using `async-generator`'s decorated functions, be aware this behavior differs from native.

Install

pip install async-generator Install stable version

Imports

async_generator
```
from async_generator import async_generator
```
The primary decorator for async generator functions is named `async_generator`.
yield_
```
from async_generator import yield_
```
In Python 3.5, inside an `@async_generator` decorated function, you must use `await yield_(value)` instead of a bare `yield value`.
asynccontextmanager
```
from async_generator import asynccontextmanager
```
This provides a backported `asynccontextmanager` decorator for Python 3.5, similar to `contextlib.asynccontextmanager` in Python 3.7+.
aclosing
```
from async_generator import aclosing
```
Used as an async context manager to ensure proper cleanup of async generators, similar to `contextlib.closing`.

Quickstart

This example demonstrates creating an asynchronous generator with `@async_generator` and `await yield_`, an asynchronous context manager with `@asynccontextmanager`, and ensuring proper cleanup of a partially consumed async generator using `aclosing`.

import asyncio
from async_generator import async_generator, yield_, asynccontextmanager, aclosing

@async_generator
async def my_async_generator(limit):
    for i in range(limit):
        await asyncio.sleep(0.01)
        await yield_(i)

@asynccontextmanager
@async_generator
async def managed_resource():
    print("\n[SETUP] Acquiring resource...")
    resource = []
    try:
        await yield_(resource)
    finally:
        print("[TEARDOWN] Releasing resource.")
        resource.clear()

async def main():
    print("--- Using async generator ---")
    async for item in my_async_generator(3):
        print(f"Consumed: {item}")

    print("\n--- Using async context manager ---")
    async with managed_resource() as res:
        res.append("data")
        print(f"Resource in context: {res}")
    print("Resource out of context.")

    print("\n--- Generator cleanup with aclosing ---")
    gen = my_async_generator(5)
    async with aclosing(gen):
        # Consume partially
        await gen.__anext__() # Consume first item (0)
        print("Partially consumed async generator, 'aclosing' will ensure cleanup.")
    print("Generator closed via aclosing.")

if __name__ == '__main__':
    asyncio.run(main())

view raw JSON →