{"id":8393,"library":"p-tqdm","title":"p_tqdm: Parallel Processing with Progress Bars","description":"p_tqdm is a Python library that simplifies parallel processing by wrapping `pathos.multiprocessing` with `tqdm` progress bars. It provides a flexible parallel map functionality, supporting lambda functions, nested functions, and class methods, while displaying a clear progress bar with estimated time to completion. The library is actively maintained, with version 1.4.2 being the current release as of August 2024, and receives updates to ensure compatibility and address issues.","status":"active","version":"1.4.2","language":"en","source_language":"en","source_url":"https://github.com/swansonk14/p_tqdm","tags":["parallel-processing","progress-bar","tqdm","multiprocessing","pathos"],"install":[{"cmd":"pip install p-tqdm","lang":"bash","label":"PyPI"}],"dependencies":[{"reason":"Provides the progress bar functionality, which p_tqdm wraps for parallel operations.","package":"tqdm"},{"reason":"Provides the flexible multiprocessing framework (pathos.multiprocessing) that p_tqdm uses for parallel execution.","package":"pathos"}],"imports":[{"symbol":"p_map","correct":"from p_tqdm import p_map"},{"symbol":"p_imap","correct":"from p_tqdm import p_imap"},{"symbol":"p_umap","correct":"from p_tqdm import p_umap"},{"symbol":"p_uimap","correct":"from p_tqdm import p_uimap"}],"quickstart":{"code":"from p_tqdm import p_map\nimport time\n\ndef my_function(x, y):\n    time.sleep(0.01) # Simulate work\n    return x + y\n\nl1 = [i for i in range(100)]\nl2 = [i * 2 for i in range(100)]\n\nresults = p_map(my_function, l1, l2)\nprint(f\"First 5 results: {results[:5]}\")","lang":"python","description":"This example demonstrates how to use `p_map` to apply a function in parallel to elements from multiple iterables, displaying a progress bar. The `my_function` simulates a task by sleeping briefly and then adding two numbers."},"warnings":[{"fix":"Upgrade your Python interpreter to 3.5+ or use p_tqdm v1.3 or higher. `pip install --upgrade p-tqdm`.","message":"p_tqdm versions prior to 1.3 dropped support for Python 2.x and versions prior to 3.5. Ensure you are using Python 3.5 or newer.","severity":"breaking","affected_versions":"< 1.3"},{"fix":"For single, repeated arguments, use `functools.partial`. For example, instead of `p_map(func, iterable, fixed_arg)`, use `p_map(partial(func, fixed_arg=fixed_arg), iterable)`.","message":"The handling of non-list iterables changed significantly in v1.3.3. Previously, single items or `num_iters` might have been implicitly repeated; now, singleton items (and the `num_iters` kwarg) should no longer be passed directly. Use `functools.partial` for repeated arguments.","severity":"breaking","affected_versions":"< 1.3.3"},{"fix":"Use `tqdm.write('Your message')` instead of `print()` to ensure messages are printed without interfering with the progress bar.","message":"Using `print()` statements inside functions processed by `p_tqdm` (or `tqdm`) can disrupt the progress bar's display, leading to messy output.","severity":"gotcha","affected_versions":"All versions"},{"fix":"Pass the `total` keyword argument to `p_map` (or other `p_tqdm` functions) with the expected number of iterations: `p_map(func, iterable, total=expected_count)`.","message":"If you are using generators or iterables without a known `len()`, the progress bar will not accurately display the total or estimated time remaining unless you explicitly provide the `total` argument.","severity":"gotcha","affected_versions":"All versions"}],"env_vars":null,"last_verified":"2026-04-16T00:00:00.000Z","next_check":"2026-07-15T00:00:00.000Z","problems":[{"fix":"Ensure the function passed to `p_tqdm` is a top-level function. If using class methods, convert them to static or standalone functions, or ensure the class itself is picklable. Consider using `multiprocess.Pool` with `tqdm.tqdm` manually if `p_tqdm`'s `pathos` integration proves problematic for complex objects, or simplify the function being pickled.","cause":"This error often arises from issues with `dill` (used by `pathos`) when trying to pickle certain objects, especially class methods or complex closures, for multiprocessing.","error":"TypeError: __init__() takes 1 positional argument but 2 were given"},{"fix":"Upgrade `p-tqdm` to version 1.4.0 or higher: `pip install --upgrade p-tqdm`. This version includes a fix for `collections` compatibility with Python 3.9+.","cause":"Older versions of `p-tqdm` (prior to v1.4.0) used deprecated imports from the `collections` module. This can cause `ModuleNotFoundError` or `DeprecationWarning` on Python 3.9+ environments. [GitHub release notes]","error":"ModuleNotFoundError: No module named 'collections.abc'"}]}