{"id":2089,"library":"kfp-pipeline-spec","title":"Kubeflow Pipelines Pipeline Specification","description":"kfp-pipeline-spec defines the canonical data structures and protobuf messages for Kubeflow Pipelines (KFP) v2 pipeline specifications. It is a core component of the KFP SDK, providing the underlying schema for pipeline definitions compiled by `kfp` and executed on the KFP backend. The library is actively maintained, with releases frequently synchronised with the main KFP SDK, currently at version 2.16.0.","status":"active","version":"2.16.0","language":"en","source_language":"en","source_url":"https://github.com/kubeflow/pipelines","tags":["kubeflow","mlops","pipeline","sdk","protobuf","machine learning"],"install":[{"cmd":"pip install kfp-pipeline-spec==2.16.0","lang":"bash","label":"Install specific version"}],"dependencies":[{"reason":"Used for defining and serializing the pipeline specification messages.","package":"protobuf","optional":false}],"imports":[{"note":"The v2 compiler path was common in older pre-GA KFP v2 releases; for direct access to the spec protobufs, import from `kfp_pipeline_spec`.","wrong":"from kfp.v2.compiler.pipeline_spec_pb2 import PipelineSpec","symbol":"PipelineSpec","correct":"from kfp_pipeline_spec.pipeline_spec_pb2 import PipelineSpec"},{"symbol":"ContainerSpec","correct":"from kfp_pipeline_spec.pipeline_spec_pb2 import ContainerSpec"}],"quickstart":{"code":"import kfp_pipeline_spec.pipeline_spec_pb2 as pb\nfrom google.protobuf import json_format\n\n# Create a minimal KFP PipelineSpec object directly using the protobuf definitions\npipeline_spec = pb.PipelineSpec(\n    pipeline_info=pb.PipelineInfo(name=\"my-direct-pipeline\"),\n    root=pb.PipelineTaskSpec(\n        executor_label=\"my-executor\",\n        inputs=pb.TaskInputsSpec(),\n        outputs=pb.TaskOutputsSpec(),\n    ),\n    components={\n        \"my-executor\": pb.ComponentSpec(\n            executor_label=\"my-executor\",\n            input_definitions=pb.ComponentInputsSpec(),\n            output_definitions=pb.ComponentOutputsSpec(),\n            implementation=pb.ComponentSpec.ContainerImplementation(\n                container=pb.ContainerSpec(\n                    image=\"python:3.9-slim\",\n                    command=[\"python\", \"-c\", \"print('Hello from kfp-pipeline-spec!')\"]\n                )\n            )\n        )\n    }\n)\n\n# Serialize the PipelineSpec to JSON (as it would be in a compiled YAML file)\njson_output = json_format.MessageToJson(pipeline_spec, indent=2)\nprint(json_output)\n\n# To verify it's a valid spec object\nassert isinstance(pipeline_spec, pb.PipelineSpec)\nprint(f\"Pipeline name: {pipeline_spec.pipeline_info.name}\")","lang":"python","description":"This quickstart demonstrates how to programmatically construct a barebones `PipelineSpec` object using the protobuf definitions from `kfp-pipeline-spec`. While most users generate this spec via the `kfp` SDK's compiler, this shows direct interaction with the underlying data model. It then serializes the spec to JSON, which is a common output format for compiled KFP pipelines."},"warnings":[{"fix":"Ensure your KFP SDK and backend are both on version 2.x or higher when using `kfp-pipeline-spec`. Old v1 pipelines need to be migrated or re-authored using KFP v2 SDK.","message":"The pipeline specification format underwent significant changes between Kubeflow Pipelines v1 and v2. `kfp-pipeline-spec` is designed for KFP v2 and is not compatible with pipeline definitions or compilers from KFP v1.","severity":"breaking","affected_versions":"<2.0.0"},{"fix":"For authoring pipelines, use the high-level `kfp` SDK (e.g., `@dsl.pipeline`, `@dsl.component`). Directly manipulating `kfp_pipeline_spec` is typically reserved for advanced use cases like custom compilers, introspection, or validation tools.","message":"`kfp-pipeline-spec` is primarily a definitions library for the KFP v2 pipeline schema. Most pipeline authors will interact with this schema indirectly through the `kfp` SDK's `kfp.dsl` and `kfp.compiler` modules, rather than directly constructing protobuf objects from `kfp_pipeline_spec`.","severity":"gotcha","affected_versions":">=2.0.0"},{"fix":"Always refer to the official KFP documentation for correct import paths and module responsibilities. Generally, `kfp` is for pipeline authoring, `kfp-pipeline-spec` for schema definitions, and `kfp-server-api` for interacting with the KFP API server.","message":"The KFP ecosystem consists of several related Python packages (`kfp`, `kfp-pipeline-spec`, `kfp-server-api`, `kfp-kubernetes`). It's a common mistake to confuse which package provides specific classes or functionalities. For example, the `Compiler` is in `kfp`, while `PipelineSpec` definition is in `kfp-pipeline-spec`.","severity":"gotcha","affected_versions":">=2.0.0"}],"env_vars":null,"last_verified":"2026-04-09T00:00:00.000Z","next_check":"2026-07-08T00:00:00.000Z"}