aioitertools: itertools and builtins for AsyncIO

0.13.0 · active · verified Sat Mar 28

aioitertools provides asynchronous versions of standard library `itertools` and `builtins` for AsyncIO and mixed iterables. It offers a unified interface for both standard and async iterators, wrapping standard iterables in async generators. The current version is 0.13.0, and it follows a feature release cadence.

Warnings

deprecated The `loop` parameter to `aioitertools.asyncio` functions was deprecated in v0.9.0 and completely removed in v0.11.0. This aligns with modern `asyncio` practices.
Fix: Remove the `loop` argument when calling any `aioitertools.asyncio` functions. The event loop is now managed automatically.
breaking Support for Python 3.6 and 3.7 was dropped in version 0.12.0.
Fix: Upgrade your Python environment to version 3.8 or newer.
gotcha `aioitertools.asyncio.as_completed` yields actual results of awaitables, unlike the standard `asyncio.as_completed` which yields futures.
Fix: Be aware that `aioitertools.asyncio.as_completed` directly gives you the resolved values, so you do not need to `await` on the yielded items.
gotcha Functions like `aioitertools.permutations` or `aioitertools.combinations` may consume the entire input iterable into memory before yielding any results. This can be inefficient for very large or infinite iterables.
Fix: For very large iterables, consider if these functions are suitable, or use them with a limited subset of data. Be mindful of memory usage.
gotcha Similar to standard `itertools`, infinite iterators (e.g., `aioitertools.count`, `aioitertools.cycle`) will run indefinitely.
Fix: Always combine infinite iterators with a 'stopping' function, such as `aioitertools.islice`, `aioitertools.takewhile`, or an explicit `break` condition in an `async for` loop, to prevent unbounded iteration.

Install

pip install aioitertools Install stable version

Imports

iter
```
from aioitertools import iter
```
Aioitertools shadows standard library builtins directly at the top level for convenience.
map
```
from aioitertools import map
```
Aioitertools shadows standard library builtins directly at the top level for convenience.
chain
```
from aioitertools import chain
```
Aioitertools shadows standard library itertools directly at the top level for convenience.
as_completed
```
from aioitertools.asyncio import as_completed
```
Asynchronous specific utilities are found under the `aioitertools.asyncio` submodule.

Quickstart

This quickstart demonstrates how to use `aioitertools.iter` to get an async iterator from both synchronous and asynchronous iterables, `aioitertools.next` to retrieve the next item, and `aioitertools.map` to apply an asynchronous function concurrently over an iterable.

import asyncio
from aioitertools import iter, next, map

async def async_generator():
    for i in range(3):
        yield i
        await asyncio.sleep(0.01)

async def process_item(item):
    # Simulate an async operation
    await asyncio.sleep(0.005)
    return item * 2

async def main():
    # Using aioitertools.iter with a standard iterable
    it_sync = iter([1, 2, 3])
    first_item = await next(it_sync)
    print(f"First item from sync: {first_item}")

    # Using aioitertools.iter with an async iterable
    it_async = iter(async_generator())
    print("Items from async generator:")
    async for item in it_async:
        print(f"  {item}")

    # Using aioitertools.map for concurrent async operations
    urls = ["url1", "url2", "url3"]
    async def fetch_url(url):
        # Replace with actual async HTTP request
        await asyncio.sleep(0.01)
        return f"Processed {url}"

    print("\nMapping async functions:")
    async for result in map(fetch_url, urls):
        print(f"  {result}")

if __name__ == "__main__":
    asyncio.run(main())

view raw JSON →