Flyte IDL 2 (Python Bindings)

2.0.12 · active · verified Thu Apr 16

flyteidl2 provides the Python bindings for the Interface Definition Language (IDL) of the Flyte platform. It encapsulates the protobuf definitions and generated client code necessary for defining and serializing Flyte's core entities like tasks, workflows, and literals. This library is a foundational dependency for the user-facing Flyte SDK (commonly installed as `flyte` or `flyte-sdk`), particularly in Flyte 2.x, enabling robust, type-safe communication and data exchange across distributed Flyte components. It is currently at version 2.0.12 and follows the Flyte SDK's release cadence.

Common errors

```
ModuleNotFoundError: No module named 'flyteidl'
```
cause You are likely using an older Flyte SDK version or attempting to import from a deprecated IDL package name. Flyte 2.x and its associated SDKs use `flyteidl2`.

fix Ensure you have `flyteidl2` installed (`pip install flyteidl2`). If using the Flyte SDK, ensure you have the correct version (`pip install flyte`). Update any old imports from `flyteidl` to `flyteidl2` where necessary.
```
TypeError: 'Literal' object is not iterable
```
cause This often occurs when trying to treat a low-level `flyteidl2.core.literals_pb2.Literal` object (which represents a single Flyte data entity) as a standard Python collection (like a list or dict). This is a common mistake when bridging between raw Flyte IDL types and Python native types.

fix Ensure you are correctly converting between Flyte's internal `Literal` types and Python native types using Flyte SDK's type transformers, or by correctly accessing the underlying values of the `Literal` object (e.g., `literal.scalar.primitive.integer`). Avoid direct manipulation of `Literal` objects unless you fully understand the Flyte type system.

Warnings

breaking Migration from Flyte 1.x to Flyte 2.x involves significant API changes in the Flyte SDK, which are underpinned by the transition from older IDL versions (e.g., `idl2`) to `flyteidl2`. This includes changes in decorators (`@task`, `@workflow` replaced by `@env.task`) and workflow definition patterns.
Fix: Refer to the official Flyte 1.x to 2.x migration guides for detailed steps. Update imports, task definitions using `TaskEnvironment`, and adopt the new 'tasks calling tasks' pattern for workflows.
gotcha Direct interaction with `flyteidl2` protobuf messages is generally not required for typical Flyte workflow authoring. It's a low-level dependency for the `flyte` SDK. Attempting to build workflows directly using `flyteidl2` types without the `flyte` SDK abstractions can lead to complex and unidiomatic code.
Fix: For authoring Flyte workflows, use the higher-level `flyte` Python SDK (`pip install flyte`). Interact with `flyteidl2` directly only when developing custom Flyte plugins, type transformers, or performing advanced introspection of Flyte's internal representations.

Install

pip install flyteidl2 Direct installation
pip install flyte Via Flyte SDK (recommended for users)

Imports

Literal
```
from flyteidl2.core import literals_pb2
```
Commonly imported for low-level interaction with Flyte's internal type system, especially when dealing with raw protobufs or extending Flyte's type transformers. Most users interact with higher-level Flyte SDK abstractions.
WorkflowTemplate
```
from flyteidl2.core import workflow_pb2
```
For directly interacting with the protobuf definition of a workflow template. Typically used internally by the Flyte SDK rather than by end-user workflow authors.

Quickstart

This quickstart demonstrates a basic Flyte workflow using the `flyte` SDK (which depends on `flyteidl2`). While `flyteidl2` itself is a low-level library, this example highlights how user-defined Python types are seamlessly translated into the Flyte IDL's protobuf messages, which `flyteidl2` provides. The explicit import at the end serves to demonstrate `flyteidl2`'s presence.

import asyncio
import flyte
import os

env = flyte.TaskEnvironment(
    name="hello_world_idl_example",
    image=flyte.Image.from_debian_base(python_version=(3, 11)),
)

@env.task
def calculate(x: int) -> int:
    # Flyteidl2 defines the underlying types and messages used by Flyte,
    # but users primarily interact with the high-level 'flyte' SDK.
    # For instance, the 'int' type here is automatically mapped to Flyte's
    # internal Literal type defined in flyteidl2.core.literals_pb2.
    return x * 2 + 5

@env.task
async def main_workflow(numbers: list[int]) -> float:
    results = await asyncio.gather(*[calculate.aio(num) for num in numbers])
    return sum(results) / len(results)

if __name__ == "__main__":
    # For local execution, you need to initialize Flyte.
    # This assumes a local Flyte setup (e.g., Flyte Sandbox via 'flytectl start')
    # or a configuration file (.flyte/config.yaml).
    # Ensure Docker is running if using sandbox.
    # For a quick local run without a full cluster, 'flyte.init()' is for local debugging.
    flyte.init()
    # To run this on a local Flyte sandbox, you'd typically use the CLI:
    # flyte run --local example_script.py main_workflow --numbers '[1,2,3]'

    # For a simple local Python execution:
    print("Running locally...")
    # The 'run' method creates a local execution environment.
    run_result = flyte.run(main_workflow, numbers=list(range(10)))
    print(f"Result: {run_result.result}")

    # To explicitly show flyteidl2 is a dependency and available:
    try:
        from flyteidl2.core import literals_pb2
        print(f"Successfully imported literals_pb2 from flyteidl2: {literals_pb2.__file__}")
    except ImportError:
        print("Could not import literals_pb2 from flyteidl2. Is it installed?")

view raw JSON →