{"id":1453,"library":"distributed","title":"Dask Distributed","description":"Distributed is the scheduler for Dask, providing a distributed computation engine for parallel and out-of-core analytics. It manages computations across a cluster of machines, featuring dynamic task scheduling, fault tolerance, and diagnostics. The library is actively maintained with frequent releases, typically on a monthly or bi-monthly schedule, following a `YYYY.MM.patch` versioning scheme. The current version is 2026.3.0.","status":"active","version":"2026.3.0","language":"en","source_language":"en","source_url":"https://github.com/dask/distributed","tags":["dask","distributed computing","parallelism","async","scheduler","data science","big data"],"install":[{"cmd":"pip install \"distributed[complete]\"","lang":"bash","label":"Recommended full installation"},{"cmd":"pip install distributed","lang":"bash","label":"Minimal installation"}],"dependencies":[{"reason":"Core dependency for Dask's array, dataframe, etc. objects.","package":"dask","optional":false},{"reason":"Required for certain data serialization and I/O operations (>=16 for recent versions).","package":"pyarrow","optional":false},{"reason":"Required to run the Dask diagnostics dashboard.","package":"bokeh","optional":true},{"reason":"Utility library (>=0.12.0 for recent versions).","package":"toolz","optional":false}],"imports":[{"note":"While `dask.distributed` is a valid module, `distributed` is the top-level package and preferred import path for its core components.","wrong":"from dask.distributed import Client","symbol":"Client","correct":"from distributed import Client"},{"symbol":"LocalCluster","correct":"from distributed import LocalCluster"},{"symbol":"fire_and_forget","correct":"from distributed import fire_and_forget"}],"quickstart":{"code":"from distributed import Client, LocalCluster\nimport time\n\ndef inc(x):\n    time.sleep(0.1) # Simulate work\n    return x + 1\n\n# Start a local Dask cluster\n# Using 'with' statement ensures proper cleanup\nwith LocalCluster(n_workers=2, threads_per_worker=2, dashboard_address=':8787') as cluster:\n    print(f\"Dashboard available at: {cluster.dashboard_link}\")\n\n    # Connect a client to the cluster\n    with Client(cluster) as client:\n        print(f\"Client connected to: {client.scheduler.address}\")\n\n        # Submit tasks\n        futures = client.map(inc, range(10))\n\n        # Gather results\n        results = client.gather(futures)\n\n        print(f\"Results: {results}\")\n        assert results == [1, 2, 3, 4, 5, 6, 7, 8, 9, 10]\n\n    print(\"Client disconnected.\")\nprint(\"LocalCluster closed.\")","lang":"python","description":"This quickstart demonstrates how to set up a local Dask cluster using `LocalCluster` and connect a `Client` to it. It then submits a simple `inc` function for parallel execution and gathers the results. The use of context managers (`with`) is crucial for proper resource management and cleanup. A dashboard link is printed if `bokeh` is installed."},"warnings":[{"fix":"Upgrade `pyarrow` to version 16 or newer: `pip install 'pyarrow>=16'`.","message":"Starting with version 2026.1.2, `distributed` requires `pyarrow>=16`. Older versions of `pyarrow` may lead to `ImportError` or serialization issues, especially with Dask DataFrames.","severity":"breaking","affected_versions":"2026.1.2+"},{"fix":"Ensure your Python environment is version 3.10 or greater. Upgrade Python or use a compatible environment.","message":"`distributed` now requires Python 3.10 or newer. Support for Python 3.9 and older has been dropped.","severity":"breaking","affected_versions":"2025.12.0+"},{"fix":"Upgrade `toolz` to version 0.12.0 or newer: `pip install 'toolz>=0.12.0'`.","message":"Starting with version 2025.12.0, `distributed` requires `toolz>=0.12.0`. Older versions may cause compatibility errors.","severity":"breaking","affected_versions":"2025.12.0+"},{"fix":"Install `distributed` with the `[dashboard]` or `[complete]` extra: `pip install \"distributed[dashboard]\"`.","message":"The Dask diagnostics dashboard requires `bokeh` to be installed. Without it, the dashboard link will still be provided but will not function correctly, often leading to a blank page or errors.","severity":"gotcha","affected_versions":"All versions"},{"fix":"Wrap `Client` and `LocalCluster` instantiation in `with` statements: `with LocalCluster(...) as cluster: ... with Client(cluster) as client: ...`.","message":"When using `Client` or `LocalCluster`, always use them as context managers (`with Client(...) as client:`) or explicitly call their `.close()` methods to ensure proper cleanup of resources, especially in scripts or long-running applications. Failing to do so can leave orphaned processes or open ports.","severity":"gotcha","affected_versions":"All versions"}],"env_vars":null,"last_verified":"2026-04-09T00:00:00.000Z","next_check":"2026-07-08T00:00:00.000Z"}