KFP Kubernetes Platform Configuration

Common errors

• AttributeError: 'ContainerOp' object has no attribute 'add_node_selector'

  cause Attempting to use KFP v1 syntax (`ContainerOp`) with KFP v2. KFP v1 `ContainerOp` objects had direct methods for Kubernetes config, but KFP v2's `dsl.Task` (or equivalent) uses external utility functions.
  fix Update your pipeline code to KFP v2 syntax. For Kubernetes configurations, import `common` from `kfp_kubernetes` and use functions like `common.add_node_selector_to_component(task, key, value)` on your `dsl.Task` object.

• ModuleNotFoundError: No module named 'kfp_kubernetes.common'

  cause The `kfp-kubernetes` package is either not installed or not installed correctly within your Python environment, preventing the import of its modules.
  fix Run `pip install kfp-kubernetes` or `pip install kfp-kubernetes==X.Y.Z` ensuring the version matches your installed KFP SDK to resolve the missing module.

• TypeError: add_volume_to_component() missing 1 required positional argument: 'mount_path'

  cause The utility function `kfp_kubernetes.common.add_volume_to_component` (or similar) was called with an incorrect number or type of arguments, or the arguments were out of order.
  fix Consult the official `kfp-kubernetes` documentation or its source code for the correct function signature and required arguments. For `add_volume_to_component`, `volume_name` and `mount_path` are typically mandatory.

• Error: "The requested object store is not supported or not configured." (exact message may vary)

  cause The KFP pipeline is configured to use an object storage solution (e.g., MinIO, SeaweedFS) that is either not deployed, incorrectly configured, or inaccessible on the underlying Kubeflow cluster. This is often related to the default object store change in KFP 2.15.0.
  fix Verify the status and configuration of the object store on your Kubeflow cluster. If using KFP 2.15.0 or later, ensure SeaweedFS is operational, or explicitly configure your pipeline to use an alternative, correctly set up S3-compatible store or MinIO.

Warnings

• breaking The KFP SDK v2 introduced significant API changes for configuring components, replacing direct manipulation of `ContainerOp` attributes (common in KFP v1) with utility functions or decorators.
  Fix: Migrate pipeline code to KFP SDK v2. Use functions from `kfp_kubernetes.common` (e.g., `add_volume_to_component`) or `kfp.kubernetes` module functions (if directly provided by the `kfp` package itself) on `dsl.Task` objects.
• gotcha Ensuring strict version alignment between `kfp-kubernetes`, `kfp`, `kfp-server-api`, and `kfp-pipeline-spec` is critical. Mismatched versions can lead to runtime errors, unexpected behavior, or incorrect Kubernetes manifest generation.
  Fix: Always install all KFP SDK related packages with the same exact version string, e.g., `pip install kfp==2.16.0 kfp-kubernetes==2.16.0 kfp-server-api==2.16.0 kfp-pipeline-spec==2.16.0`.
• gotcha Starting with KFP 2.15.0, the default object store deployment for new installations changed from MinIO to SeaweedFS. Existing installations or specific configurations might still use MinIO, but this change can affect how persistent volumes are expected to be managed.
  Fix: Review your Kubeflow cluster's object store configuration. Ensure your pipeline's volume configurations (e.g., via `add_volume_to_component`) are compatible with the deployed object store or explicitly configure your preferred storage solution.
• gotcha While `kfp-kubernetes` helps generate Kubernetes manifests for pipeline components, the final execution is dependent on the actual Kubernetes cluster configuration. Discrepancies (e.g., missing StorageClasses, incorrect node labels, RBAC permissions) can cause pipeline failures.
  Fix: Validate the generated Kubernetes YAML (e.g., `kubernetes_config_pipeline.yaml`) and ensure your cluster has the necessary resources, permissions (RBAC), node labels/taints, and storage classes to support the requested configurations.

Install

• pip install kfp-kubernetes==2.16.0 Install specific version

Imports

• common
  wrong:

  from kfp.kubernetes import common

  correct:

  from kfp_kubernetes import common

  While `kfp.kubernetes` exists, `kfp-kubernetes` provides its own set of utilities, especially for KFP SDK v2. Direct import from `kfp_kubernetes.common` is recommended for explicit usage of this library's features.
• KubernetesPlatformConfig

  from kfp_kubernetes.kfp_kubernetes_platform import KubernetesPlatformConfig

Quickstart

This example demonstrates how to define a KFP pipeline and use `kfp_kubernetes.common` functions to add Kubernetes-specific configurations like a volume, node selector, and toleration to a component. This pipeline requires a KFP SDK v2 compatible environment for execution and an existing PVC named 'my-data-pvc' on the Kubernetes cluster.

import kfp
from kfp import dsl
from kfp_kubernetes import common

# Define a simple KFP component
@dsl.component
def hello_world_op(name: str) -> str:
    import os
    print(f"Hello, {name} from pod {os.environ.get('KUBERNETES_POD_NAME', 'unknown')}!")
    return f"Hello, {name}!"

# Create a pipeline
@dsl.pipeline(name="kubernetes-config-pipeline")
def kubernetes_config_pipeline():
    task = hello_world_op(name="World")

    # Add a Kubernetes volume to the component's pod
    # This assumes an existing PVC named 'my-data-pvc' on your cluster
    common.add_volume_to_component(
        task,
        volume_name="my-data-volume",
        mount_path="/mnt/data",
        existing_pvc_name="my-data-pvc"
    )

    # Add a node selector to schedule the task on a specific node
    common.add_node_selector_to_component(task, "kubernetes.io/hostname", "my-worker-node-label")

    # Add a toleration to allow scheduling on tainted nodes
    common.add_toleration_to_component(task, "key", "value", "Equal", "NoSchedule")

# Compile the pipeline (requires kfp to be installed)
if __name__ == "__main__":
    try:
        kfp.compiler.Compiler().compile(kubernetes_config_pipeline, "kubernetes_config_pipeline.yaml")
        print("Pipeline compiled successfully to kubernetes_config_pipeline.yaml")
    except Exception as e:
        print(f"Error compiling pipeline: {e}")