Kubeflow Pipelines SDK

Common errors

• ModuleNotFoundError: No module named 'kfp'

  cause The 'kfp' Python package is not installed in the current environment or the environment where the code is being executed.
  fix Install the Kubeflow Pipelines SDK using pip: `pip install kfp`.

• AttributeError: module 'kfp.components' has no attribute 'create_component_from_func'

  cause This error typically occurs when using KFP v1 API syntax (`kfp.components.create_component_from_func`) with a KFP v2 SDK installation, or when migrating from KFP v1 to v2 without updating component definitions.
  fix Migrate your component definitions to the KFP v2 style by using the `@kfp.dsl.component` decorator for Python functions instead of `kfp.components.create_component_from_func`.

• TypeError: Input argument supports only the following types: PipelineParam, str, int, float, bool, dict, and list. Got: "None".

  cause A component function in a KFP v2 pipeline received an argument with an unsupported type, often a `None` value, where a specific primitive or collection type was expected during pipeline compilation.
  fix Ensure all inputs passed to KFP v2 components are explicitly typed and are valid KFP parameter types (str, int, float, bool, dict, list). Avoid passing `None` or custom object types directly as component inputs.

• kfp or dsl-compile command not found

  cause The `kfp` or `dsl-compile` command-line executables, installed as part of the Kubeflow Pipelines SDK, are not in your system's PATH environment variable.
  fix If installed with `--user`, add the Python user base binary directory (e.g., `~/.local/bin` on Linux/macOS) to your PATH environment variable. For example: `export PATH=$PATH:~/.local/bin` in your shell's configuration file (like `~/.bashrc` or `~/.zshrc`), then `source` the file or restart your terminal.

Warnings

• breaking KFP SDK v2 is generally not backward compatible with user code written using the KFP SDK v1 main namespace. Key breaking changes include a new more Pythonic SDK with decorators like `@dsl.pipeline` and `@dsl.component`, and compilation to a generic Intermediate Representation (IR) YAML instead of Argo Workflow YAML.
  Fix: Rewrite pipeline and component definitions to use the new KFP SDK v2 decorators and API patterns. Refer to the official migration guides.
• breaking As of KFP 2.15.0, the default object store deployment for Kubeflow Pipelines has changed from MinIO to SeaweedFS. While MinIO is still supported, users upgrading from versions prior to 2.15.0 with existing or custom MinIO configurations for their backend may need to adjust their deployment manifests to maintain their desired object store configuration.
  Fix: Review deployment configurations and explicitly specify MinIO manifests if you wish to continue using it, or migrate to SeaweedFS or another S3-compatible store.
• breaking KFP 2.15.0 introduced a major upgrade to the underlying Gorm backend, necessitating an automated database index migration. This migration does not support rollback. It is strongly advised to back up production databases before initiating an upgrade from versions prior to 2.15.0.
  Fix: Perform a full backup of your KFP database before upgrading to KFP 2.15.0 or later to ensure data recovery in case of issues with the migration.
• gotcha In KFP 2.15.0, a regression was identified for AWS S3 authentication using IAM Roles for Service Accounts (IRSA). Specifically, the environment variables `OBJECTSTORECONFIG_ACCESSKEY` and `OBJECTSTORECONFIG_SECRETACCESSKEY` (which could previously be empty or omitted when using IRSA) became implicitly required, leading to authentication failures.
  Fix: Check for official patch releases (e.g., 2.15.1, 2.15.2, etc.) that address this specific IRSA regression. If not patched, ensure these environment variables are correctly configured or fall back to an earlier stable version or an alternative authentication method if IRSA is critical.
• gotcha The default base_image used by the `@dsl.component` decorator will switch from 'python:3.11' to 'python:3.12' on Oct 1, 2027. This change could affect component execution if not explicitly accounted for in future KFP SDK versions.
  Fix: To ensure future compatibility, explicitly provide a `base_image` argument to the `@dsl.component` decorator and verify component functionality with Python 3.12. For example, `@dsl.component(base_image='python:3.11')`.
• gotcha A FutureWarning is emitted for `@dsl.component` indicating that the default `base_image` will change from 'python:3.11' to 'python:3.12' on Oct 1, 2027. This may affect existing components that do not explicitly specify a `base_image` argument.
  Fix: To ensure future compatibility, explicitly provide a `base_image` argument to `@dsl.component` (e.g., `base_image='python:3.11'`) and verify that your component functions correctly with Python 3.12 if you plan to rely on the new default.

Install

• pip install kfp Core SDK
• pip install kfp[kubernetes] SDK with Kubernetes Extras

Imports

• Client

  from kfp import Client

  The Client class for interacting with the KFP backend API is exposed directly under the 'kfp' namespace for convenience, though its canonical path is 'kfp.client.Client'.
• dsl
  wrong:

  from kfp.v2 import dsl

  correct:

  from kfp import dsl

  For KFP SDK v2, DSL components are directly under the 'kfp.dsl' namespace. The 'kfp.v2' namespace for DSL was part of the v2-compatible mode and is now deprecated.
• component

  from kfp.dsl import component

• pipeline

  from kfp.dsl import pipeline

Quickstart

This quickstart defines a simple Python function as a KFP component using `@dsl.component` and then orchestrates it into a pipeline using `@dsl.pipeline`. It demonstrates the basic structure for authoring KFP v2 pipelines. To actually execute the pipeline, it needs to be compiled and submitted to a running Kubeflow Pipelines backend.

import kfp
from kfp import dsl
import os

# Define a lightweight Python component
@dsl.component
def add(a: float, b: float) -> float:
    '''Calculates sum of two arguments'''
    return a + b

# Define a pipeline using the component
@dsl.pipeline(
    name='Addition pipeline',
    description='An example pipeline that performs addition calculations.'
)
def add_pipeline(
    a: float = 1.0,
    b: float = 7.0,
):
    first_add_task = add(a=a, b=4.0)
    second_add_task = add(a=first_add_task.output, b=b)

# --- Running the pipeline (requires KFP backend) ---
# In a real environment, you'd configure the KFP client to connect to your KFP instance.
# For local testing without a KFP backend, you can use `kfp.local.init`.

# Example of compiling a pipeline (no KFP backend needed for this step)
# compiler = kfp.compiler.Compiler()
# compiler.compile(pipeline_func=add_pipeline, package_path='add_pipeline.yaml')

# Example of running a pipeline against a KFP endpoint
# client = kfp.Client(host=os.environ.get('KFP_HOST', 'http://localhost:8080'))
# run = client.create_run_from_pipeline_func(
#     add_pipeline,
#     arguments={'a': 7.0, 'b': 8.0}
# )
# print(f"Pipeline run initiated: {run.url}")

print("Pipeline 'add_pipeline' defined successfully. To run, compile and submit to a KFP backend.")