PQDM - Parallel TQDM

0.2.0 · active · verified Wed Apr 15

PQDM is a TQDM and concurrent futures wrapper designed to provide enjoyable parallelization of iterating through an Iterable with a progress bar. It supports both process-based and thread-based parallel execution and automatically integrates with `tqdm.notebook` in Jupyter environments. The current version is 0.2.0, with the last PyPI release in February 2022, suggesting a mature or slower development cadence.

Warnings

gotcha Choosing the correct backend (processes vs. threads) is crucial for optimal performance. Use `from pqdm.processes import pqdm` for CPU-bound tasks and `from pqdm.threads import pqdm` for I/O-bound tasks. Incorrect selection can lead to suboptimal performance, e.g., using threads for CPU-bound tasks in Python due to the GIL.
Fix: Always explicitly import `pqdm` from either `pqdm.processes` or `pqdm.threads` based on the nature of your workload.
gotcha When passing arguments to the target function, `pqdm` defaults to passing iterable elements directly. For functions expecting multiple positional arguments or keyword arguments, you must specify the `argument_type` parameter (e.g., `'args'` for `func(a, b)` with iterable `[[a1, b1], [a2, b2]]` or `'kwargs'` for `func(a=x, b=y)` with iterable `[{'a': x1, 'b': y1}, {'a': x2, 'b': y2}]`).
Fix: Review the 'Different ways to pass arguments to function' section in the documentation and set `argument_type='args'` or `argument_type='kwargs'` as needed.
gotcha The latest PyPI release (v0.2.0) for `pqdm` was in February 2022. While the library is functional, this suggests a slower or potentially maintenance-focused development cycle. Users seeking actively developed or frequently updated features might want to check the GitHub repository directly for the latest commits or consider alternative libraries if frequent updates are a requirement.
Fix: Be aware of the project's update cadence. For critical bug fixes or new features, monitor the GitHub repository's commit history and issues page.

Install

pip install pqdm Install stable release

Imports

pqdm
```
from pqdm.processes import pqdm
```
Users must explicitly import from `pqdm.processes` for multiprocessing or `pqdm.threads` for multithreading, depending on their use case. The top-level `pqdm` module does not directly expose the callable.

Quickstart

This example demonstrates how to use `pqdm` to parallelize a simple CPU-bound task using 4 processes, displaying a TQDM progress bar. For I/O-bound tasks, `from pqdm.threads import pqdm` would be more appropriate.

from pqdm.processes import pqdm
import time

def long_running_task(number):
    time.sleep(0.1) # Simulate work
    return number * number

items = list(range(1, 101))

# Parallelize the task with 4 processes and a progress bar
results = pqdm(items, long_running_task, n_jobs=4, desc="Processing numbers")

print(f"First 5 results: {results[:5]}")

view raw JSON →