{"id":2596,"library":"mpire","title":"MPIRE - MultiProcessing Is Really Easy","description":"MPIRE is a Python package that simplifies multiprocessing, offering a faster and more user-friendly alternative to the standard `multiprocessing` module. It provides an intuitive API with map-like functions, support for worker state, progress bars (via tqdm), worker insights, and efficient handling of shared objects. The library is actively maintained with regular updates and aims to make parallelizing CPU-bound tasks straightforward and performant.","status":"active","version":"2.10.2","language":"en","source_language":"en","source_url":"https://github.com/sybrenjansen/mpire","tags":["multiprocessing","parallel-processing","performance","worker-pool","tqdm"],"install":[{"cmd":"pip install mpire","lang":"bash","label":"Install MPIRE"}],"dependencies":[{"reason":"Provides progress bar functionality.","package":"tqdm","optional":false},{"reason":"Used for code highlighting in some features (e.g., dashboard).","package":"pygments","optional":false},{"reason":"Required for Windows-specific features.","package":"pywin32","optional":false},{"reason":"Required for Python versions prior to 3.9.","package":"importlib_resources","optional":false},{"reason":"Optional serialization backend through `multiprocess`, enabling parallelizing more exotic objects (e.g., lambdas, nested functions).","package":"dill","optional":true},{"reason":"Optional dependency for the MPIRE dashboard feature.","package":"Flask","optional":true}],"imports":[{"note":"MPIRE's core class is `WorkerPool`, which extends `multiprocessing.Pool` with additional features and a different API. Directly using `multiprocessing.Pool` will not provide MPIRE's enhancements.","wrong":"from multiprocessing import Pool","symbol":"WorkerPool","correct":"from mpire import WorkerPool"}],"quickstart":{"code":"import time\nfrom mpire import WorkerPool\n\ndef my_function(x):\n    time.sleep(0.01) # Simulate some work\n    return x * 2\n\n# Run with 4 worker processes\nwith WorkerPool(n_jobs=4) as pool:\n    results = pool.map(my_function, range(100))\n    print(f\"Results: {results[:5]}...\")","lang":"python","description":"This quickstart demonstrates how to use `mpire.WorkerPool` to parallelize a simple function across multiple processes. It initializes a pool with 4 workers and uses the `map` function, similar to Python's built-in `map` but executed in parallel."},"warnings":[{"fix":"Update your code to use `func` instead of `func_pointer` when calling `WorkerPool.map`, `map_unordered`, `imap`, or `imap_unordered`.","message":"The `func_pointer` parameter in `map` functions was renamed to `func`.","severity":"breaking","affected_versions":">=2.0.0"},{"fix":"Remove the `restart_workers` parameter. Consider using `worker_lifespan` in the `WorkerPool` constructor for similar functionality of restarting workers after a specified number of tasks.","message":"The `restart_workers` parameter was deprecated and subsequently removed.","severity":"deprecated","affected_versions":"Deprecated in <1.0.0, removed in >=1.0.0"},{"fix":"Pass `enable_insights=True` to the `WorkerPool` constructor instead of individual `map` calls.","message":"The `enable_insights` parameter moved from `map` functions to the `WorkerPool` constructor.","severity":"deprecated","affected_versions":"Deprecated in >=2.3.0, removed in >=2.6.0"},{"fix":"On Windows, shared objects will be copied once per worker instead of using copy-on-write. This might still be more efficient than copying per task, but understand the performance implications compared to Linux/macOS.","message":"Copy-on-write shared objects, a key performance feature, are only available when using the `fork` start method, which is not supported on Windows.","severity":"gotcha","affected_versions":"All versions"},{"fix":"Avoid mixing different types of `map` calls within the same `WorkerPool` instance. If you need to use different map styles, create separate `WorkerPool` instances for each.","message":"Mixing multiple `map` calls (e.g., `pool.map` followed by `pool.imap`) can raise an error due to internal state management.","severity":"gotcha","affected_versions":">=2.7.0"},{"fix":"These `OSError` messages can generally be safely ignored as they are related to internal cleanup processes.","message":"When using `dill` as the serialization backend on Windows, exceptions raised by workers or exit functions might print additional `OSError` messages to the terminal.","severity":"gotcha","affected_versions":"All versions with `dill` on Windows"},{"fix":"If running on Windows and needing a progress bar, use a different start method (e.g., `spawn` or `forkserver` if available, though `fork` is not on Windows) or consider disabling the progress bar.","message":"The progress bar feature is not supported when using the `threading` start method on Windows.","severity":"gotcha","affected_versions":"All versions"}],"env_vars":null,"last_verified":"2026-04-10T00:00:00.000Z","next_check":"2026-07-09T00:00:00.000Z"}