{"id":6682,"library":"ipyparallel","title":"ipyparallel","description":"ipyparallel provides an architecture for interactive parallel and distributed computing within the IPython ecosystem, allowing users to interact with a cluster of Python kernels. It enables parallel execution of code, distributed data processing, and asynchronous task management. The current version is 9.1.0, and it generally follows a moderate release cadence, with major versions introducing significant features or compatibility changes.","status":"active","version":"9.1.0","language":"en","source_language":"en","source_url":"https://github.com/ipython/ipyparallel","tags":["parallel-computing","ipython","distributed-computing","jupyter","hpc"],"install":[{"cmd":"pip install ipyparallel","lang":"bash","label":"Install ipyparallel"}],"dependencies":[{"reason":"Core foundation for interactive Python and kernel management.","package":"ipython","optional":false}],"imports":[{"note":"Old import path from IPython itself; ipyparallel is now a separate library.","wrong":"from IPython.parallel import Client","symbol":"Client","correct":"from ipyparallel import Client"},{"note":"Programmatic cluster management, less common than the CLI 'ipcluster' for basic use.","symbol":"Cluster","correct":"from ipyparallel.cluster import Cluster"}],"quickstart":{"code":"import ipyparallel as ipp\nimport time\nimport os\n\n# NOTE: For this code to run, you must have an ipyparallel cluster running.\n#       Start one from your terminal using: ipcluster start\n#       (e.g., 'ipcluster start --n=4' for 4 engines)\n\ntry:\n    # Connect to the default cluster profile\n    client = ipp.Client()\n    \n    if not client.ids:\n        raise RuntimeError(\"No engines found in the cluster. Ensure 'ipcluster start' is running.\")\n\n    print(f\"Connected to cluster with {len(client.ids)} engines.\")\n\n    # Get a direct view (all engines, tasks are mapped across them)\n    dview = client[:]\n\n    # Execute a simple parallel task synchronously\n    def square(x):\n        time.sleep(0.01) # Simulate some work\n        return x * x\n\n    print(\"Executing map_sync on range(10)...\")\n    results = dview.map_sync(square, range(10))\n    print(f\"Parallel results (first 5): {results[:5]}...\")\n\n    # Execute an asynchronous task on each engine\n    print(\"Executing async getpid()...\")\n    ar = dview.apply_async(lambda : os.getpid())\n    pids = ar.get()\n    print(f\"PIDs from engines: {pids}\")\n\nexcept Exception as e:\n    print(f\"Could not connect to ipyparallel cluster or encountered an error: {e}\")\n    print(\"Please ensure 'ipcluster start' is running and no firewall is blocking access.\")\n","lang":"python","description":"This quickstart demonstrates how to connect to an existing ipyparallel cluster, obtain a direct view, and execute both synchronous and asynchronous parallel tasks. It assumes `ipcluster start` has been run in a separate terminal or managed process. If no cluster is running, the connection will fail."},"warnings":[{"fix":"Upgrade your Python environment to 3.10 or later, or use an older ipyparallel version (e.g., 8.x for Python 3.8-3.9).","message":"ipyparallel 9.x requires Python 3.10 or newer. Older Python versions are not supported.","severity":"breaking","affected_versions":"<9.0.0"},{"fix":"Ensure that the necessary ports (default 10100-10105 range, configurable) are open between the client and engine machines, or configure your profile to use SSH tunnels.","message":"Connections to the ipyparallel cluster can be blocked by firewalls or incorrect network configurations, especially when engines are on different machines.","severity":"gotcha","affected_versions":"all"},{"fix":"Always ensure the client's profile matches the cluster's profile. Check `~/.ipython/profile_*/security` for connection files.","message":"The Client must connect to the correct profile. If `ipcluster start` was run with `--profile=myprofile`, the client must specify `ipp.Client(profile='myprofile')`.","severity":"gotcha","affected_versions":"all"},{"fix":"`DirectView` assigns tasks round-robin (for `map`) or explicitly per-engine; `LoadBalancedView` sends tasks to the least busy engine. Choose the view appropriate for your workload.","message":"Understanding the difference between `DirectView` (`client[:]`) and `LoadBalancedView` (`client.load_balanced_view()`) is crucial for correct task distribution and performance.","severity":"gotcha","affected_versions":"all"},{"fix":"Always use `ipcluster stop` (or `client.shutdown()` programmatically) when you are finished with a cluster to clean up processes and free resources.","message":"Properly managing the cluster lifecycle (`ipcluster start`/`stop`) is essential. Forgetting to stop clusters can leave orphan processes and consume resources.","severity":"gotcha","affected_versions":"all"}],"env_vars":null,"last_verified":"2026-04-15T00:00:00.000Z","next_check":"2026-07-14T00:00:00.000Z","problems":[]}