RAPIDS Dask Dependency Pinning

26.4.0 · active · verified Thu Apr 16

rapids-dask-dependency is a meta-package within the RAPIDS ecosystem that pins compatible versions of Dask and Distributed for a specific RAPIDS release. Its primary function is to ensure that users working with RAPIDS libraries like cuDF and Dask-cuDF have a consistent and stable Dask environment, preventing common version mismatch issues. It aligns with the RAPIDS release cadence, currently at version 26.4.0.

Common errors

```
distributed.scheduler.KilledWorker: Worker failed to start
```
cause Incompatible Dask and Distributed versions between the scheduler and workers, often due to an improperly managed environment or conflicting installations.

fix Ensure `rapids-dask-dependency` (and other RAPIDS packages) are installed consistently across all nodes and in the correct version for your RAPIDS release. Consider rebuilding your environment from scratch using `conda create -n myenv python=3.10` then `conda install -c rapidsai -c nvidia -c conda-forge -c nodejs rapids-dask-dependency=<version> dask-cudf=<version>`.
```
TypeError: 'distributed.protocol.pickle' object is not callable
```
cause This error or similar `TypeError` or `AttributeError` can occur when Dask and Distributed have subtle API incompatibilities due to mismatched versions, despite appearing to import correctly.

fix Verify that the versions of `dask` and `distributed` installed exactly match what `rapids-dask-dependency` specifies for your RAPIDS release. Re-run `pip install --upgrade --force-reinstall rapids-dask-dependency` to ensure these packages are correctly pinned. Check `pip list | grep dask` and `pip list | grep distributed`.

Warnings

breaking RAPIDS libraries (e.g., cudf, dask-cudf) and rapids-dask-dependency *must* belong to the same major RAPIDS release cycle (e.g., all 26.04.x). Mixing versions from different releases (e.g., 26.04 rapids-dask-dependency with 24.08 cudf) will lead to runtime errors, crashes, or incorrect behavior due to ABI/API mismatches.
Fix: Always install all RAPIDS components (including rapids-dask-dependency) within the same environment and specify the same major version number, e.g., `pip install 'rapids-dask-dependency==26.4.0' 'cudf==26.4.0' 'dask-cudf==26.4.0'`. Consider using `conda` for stricter environment management.
gotcha Installing `rapids-dask-dependency` *after* Dask or Distributed might not automatically downgrade/upgrade existing packages. Pip generally prefers not to downgrade unless explicitly forced, leading to a potentially mixed environment.
Fix: Always install `rapids-dask-dependency` *first* in a fresh virtual environment, or use `pip install --upgrade --force-reinstall rapids-dask-dependency` to ensure Dask/Distributed are properly managed. Even better, use `conda` for RAPIDS installations, as it handles dependencies more robustly.

Install

pip install rapids-dask-dependency==26.4.0 Install specific RAPIDS version

Quickstart

rapids-dask-dependency is installed to manage your environment, not imported for direct use. The quickstart shows how to install it to ensure compatible Dask/Distributed versions are present, which then allows you to seamlessly use Dask with other RAPIDS libraries.

# This package doesn't provide direct Python imports for functionality.
# Its purpose is to ensure compatible Dask/Distributed versions are installed.
# You install it to set up your environment *before* using Dask with RAPIDS.
#
# After installing, you can then import Dask components, which will be
# the versions specified by rapids-dask-dependency:

# Example of using Dask after installation:
import dask.array as da
import dask.dataframe as dd
from dask.distributed import Client

# This code will now run with Dask/Distributed versions compatible
# with the 26.04 RAPIDS release.

# client = Client()
# print(client)
# client.close()

print(f"Dask Array imported successfully: {da.__version__}")
print(f"Dask DataFrame imported successfully: {dd.__version__}")
# print(f"Dask Distributed imported successfully: {Client.__module__}")

view raw JSON →