{"id":7133,"library":"dask-kubernetes","title":"Dask Kubernetes","description":"dask-kubernetes provides native integration for Dask with Kubernetes, allowing users to deploy and manage Dask clusters programmatically using the Python API (KubeCluster) or declaratively using Kubernetes Custom Resources (Dask Operator). The current version is 2026.3.0, and it follows a rapid release cadence, often monthly or quarterly, in alignment with the broader Dask ecosystem.","status":"active","version":"2026.3.0","language":"en","source_language":"en","source_url":"https://github.com/dask/dask-kubernetes","tags":["dask","kubernetes","distributed-computing","orchestration","mlops"],"install":[{"cmd":"pip install dask-kubernetes","lang":"bash","label":"Install stable version"}],"dependencies":[{"reason":"Requires Python 3.10 or newer.","package":"python","optional":false},{"reason":"Required for the Dask Kubernetes Operator functionality. Minimum version 1.38.0 as of 2025.7.0.","package":"kopf","optional":true},{"reason":"Core dependency for Kubernetes interaction. Minimum version 0.20.* as of 2025.4.0.","package":"kr8s","optional":false}],"imports":[{"symbol":"KubeCluster","correct":"from dask_kubernetes import KubeCluster"},{"note":"The DaskKubernetesOperator is part of the 'operator' submodule.","wrong":"from dask_kubernetes import DaskKubernetesOperator","symbol":"DaskKubernetesOperator","correct":"from dask_kubernetes.operator import DaskKubernetesOperator"}],"quickstart":{"code":"from dask_kubernetes import KubeCluster\nfrom dask.distributed import Client\n\n# Ensure you have a kubectl context configured for a running Kubernetes cluster.\n# KubeCluster automatically detects the current context.\n\n# Create a Dask cluster on Kubernetes\ncluster = KubeCluster(name=\"my-dask-cluster\", n_workers=3)\nprint(f\"Dashboard link: {cluster.dashboard_link}\")\n\n# Connect a Dask client to the cluster\nclient = Client(cluster)\n\n# Perform a simple computation\ndef inc(x): return x + 1\ndef add(x, y): return x + y\n\nfutures = client.map(inc, range(10))\ntotal = client.submit(add, *futures)\nprint(f\"Result of computation: {total.result()}\")\n\n# Scale the cluster (optional)\ncluster.scale(5)\nprint(f\"Cluster scaled to {len(cluster.workers)} workers.\")\n\n# Close the client and cluster when done\nclient.close()\ncluster.close()\nprint(\"Dask cluster and client closed.\")","lang":"python","description":"This quickstart demonstrates how to create a Dask cluster on Kubernetes using `KubeCluster`, connect a Dask client, perform a simple computation, and then scale the cluster. It assumes you have `kubectl` configured and authenticated to a Kubernetes cluster."},"warnings":[{"fix":"Upgrade your Python environment to 3.10 or newer.","message":"Support for Python 3.9 was dropped in version 2024.8.0. Users on Python 3.9 must upgrade their Python environment.","severity":"breaking","affected_versions":">=2024.8.0"},{"fix":"For interactive Python sessions or quick programmatic deployments, use `KubeCluster`. For persistent, enterprise-grade, or CI/CD deployments, consider installing the Dask Kubernetes Operator and defining `DaskCluster` CRDs via YAML. Consult the official documentation for detailed guidance on each approach.","message":"dask-kubernetes offers two main deployment strategies: `KubeCluster` (programmatic client-side) and the Dask Kubernetes Operator (CRD-based, declarative via YAML). Users often confuse these or apply inappropriate configurations.","severity":"gotcha","affected_versions":"All versions"},{"fix":"Ensure your `kopf` installation meets or exceeds version `1.38.0` if you are using the Dask Kubernetes Operator or related features.","message":"The minimum required version for `kopf` (a core dependency for the operator) was bumped to `1.38.0` in `dask-kubernetes==2025.7.0`.","severity":"breaking","affected_versions":">=2025.7.0"},{"fix":"Ensure your `kr8s` installation meets or exceeds version `0.20.0` to avoid compatibility issues.","message":"The minimum required version for `kr8s` (a core dependency for Kubernetes interaction) was bumped to `0.20.*` in `dask-kubernetes==2025.4.0`.","severity":"breaking","affected_versions":">=2025.4.0"},{"fix":"Ensure the Kubernetes ServiceAccount used by your Dask scheduler and worker pods has adequate permissions (e.g., `list`, `watch`, `create`, `delete` for `pods`, `services`, `deployments`, etc.) in the target namespace. Refer to the Dask Kubernetes documentation for recommended RBAC configurations or use the provided Helm chart which configures default roles.","message":"Common errors arise from insufficient Kubernetes RBAC (Role-Based Access Control) permissions for the service account used by Dask pods. This can prevent pod creation, service exposure, or resource scaling.","severity":"gotcha","affected_versions":"All versions"}],"env_vars":null,"last_verified":"2026-04-16T00:00:00.000Z","next_check":"2026-07-15T00:00:00.000Z","problems":[{"fix":"Verify that your `kubectl` context is correctly configured and has the required permissions. If running `KubeCluster` inside a Kubernetes pod, ensure the pod's `serviceAccountName` has appropriate `RoleBindings` to access resources (pods, services, deployments) in its namespace. Consult Dask Kubernetes RBAC documentation for minimum required permissions.","cause":"The Kubernetes service account used by dask-kubernetes (or the user's `kubectl` context) lacks the necessary RBAC permissions to create or manage resources in the specified namespace.","error":"kubernetes.client.rest.ApiException: (403) Reason: Forbidden"},{"fix":"Ensure your `~/.kube/config` file is correctly set up and points to a running Kubernetes cluster. If running locally, confirm `kubectl config current-context` returns a valid context. If deploying within a Kubernetes pod, ensure it has access to the cluster's service account tokens.","cause":"dask-kubernetes could not find a valid Kubernetes configuration (kubeconfig file) or is not running inside a Kubernetes cluster environment.","error":"ValueError: No Kubernetes context found. Either provide a config file, or run within a Kubernetes Pod."},{"fix":"Check `kubectl describe pod <scheduler-pod-name>` and `kubectl logs <scheduler-pod-name>` for error messages. Verify the Docker image path and tag are correct and accessible. Review resource requests/limits defined in your `KubeCluster` arguments or `DaskCluster` YAML. Ensure no network policies are blocking communication to/from the scheduler.","cause":"The Dask scheduler pod failed to start or become ready. Common reasons include incorrect image name/tag, insufficient resource requests/limits, network policy restrictions, or a misconfigured `DaskCluster` CRD.","error":"Waiting for Dask scheduler pod to be ready... (TimeoutError or similar message indicating pod failure)"},{"fix":"Run `pip install dask-kubernetes` in your active Python environment. If using virtual environments, ensure you've activated the correct one.","cause":"The `dask-kubernetes` library is not installed in the current Python environment.","error":"ModuleNotFoundError: No module named 'dask_kubernetes'"}]}