Apache Airflow Task SDK

raw JSON →
1.2.0 verified Fri Apr 24 auth: no python

The Apache Airflow Task SDK provides Python-native interfaces for defining Directed Acyclic Graphs (DAGs), executing tasks in isolated subprocesses, and interacting with Airflow resources at runtime. Its primary goal is to decouple DAG authoring from Airflow internals, offering a forward-compatible and stable interface across Airflow versions. The library is currently at version 1.2.0 and receives frequent updates, aligning with the release cadence of Apache Airflow 3.x.

pip install apache-airflow-task-sdk
error ImportError: cannot import name 'DAG' from 'airflow' (unknown location)
cause This error occurs when the Python script is named 'airflow.py', causing a conflict with the Airflow package during import.
fix
Rename your script to avoid naming conflicts with the Airflow package.
error ImportError: cannot import name 'SUPERVISOR_COMMS' from 'airflow.sdk.execution_time.task_runner'
cause This error occurs when attempting to import 'SUPERVISOR_COMMS' from 'airflow.sdk.execution_time.task_runner', which may be due to an issue with the Airflow Task SDK.
fix
Ensure you are using the latest version of the Airflow Task SDK and check for any known issues or updates related to this import error.
error ModuleNotFoundError: No module named 'my_company'
cause This error occurs when attempting to import a module from the 'dags' folder without properly configuring the Python path.
fix
Ensure that the 'dags' folder is included in your Python path or use relative imports to access modules within the 'dags' directory.
error ModuleNotFoundError: No module named 'your_custom_module'
cause The Python environment where Airflow is running cannot find the specified module, which can be a custom module within your DAGs folder, a dependency for your tasks, or even the 'apache-airflow-task-sdk' itself if not properly installed.
fix
Ensure the required module is installed in the Airflow environment using pip install your_custom_module. If it's a local module, verify its path is correctly added to PYTHONPATH or placed in a discoverable location for Airflow's DAG processor.
error from airflow.decorators import dag, task
cause Developers are using the older TaskFlow API decorators from `airflow.decorators`, which are considered internal and less stable, instead of the recommended, forward-compatible interfaces provided by the Apache Airflow Task SDK (`airflow.sdk`).
fix
Update your DAG imports to use the Task SDK namespace: from airflow.sdk import dag, task.
breaking Direct metadata database access from task code is now restricted in Airflow 3.x. Tasks can no longer directly import and use Airflow database sessions or models. All runtime interactions must go through a dedicated Task Execution API via the Task SDK.
fix Refactor task code to use the Task SDK's interfaces for interacting with Airflow resources (e.g., Connections, Variables, XComs) instead of direct database access.
breaking All DAGs must update their imports to refer to `airflow.sdk` instead of using internal Airflow modules directly. Deprecated legacy import paths (e.g., `airflow.models.dag.DAG`, `airflow.decorator.task`) will be removed in future Airflow versions.
fix Update all DAG and task-related imports to use the `airflow.sdk` namespace. For example, `from airflow.models.dag import DAG` becomes `from airflow.sdk import DAG`.
breaking SubDAGs are replaced by TaskGroups, Assets, and Data Aware Scheduling in Airflow 3.x. Existing SubDAGs will need to be refactored.
fix Migrate SubDAG implementations to use TaskGroups for grouping tasks, and explore Assets and Data Aware Scheduling for managing complex data dependencies.
deprecated SLAs (Service Level Agreements) are deprecated and replaced with Deadline Alerts in Airflow 3.x.
fix Transition from using SLAs to implementing Deadline Alerts for monitoring task and DAG adherence to time constraints.
gotcha The `catchup_by_default` DAG parameter is now `False` by default in Airflow 3.x. This can lead to unexpected behavior if DAGs were implicitly relying on catchup being enabled.
fix Explicitly set `catchup=True` in your DAG definition if you intend for it to run for past missed schedules. Otherwise, review your DAGs to ensure `catchup=False` is the desired behavior.
gotcha When passing context to tasks, the traditional `def my_task(**context)` signature is being superseded. While still functional, the recommended approach in the Task SDK is to explicitly retrieve context using `airflow.sdk.get_current_context()`.
fix Update task definitions to import `get_current_context` from `airflow.sdk` and call it within the task function to retrieve the execution context.
runtime status import time mem disk
3.10-alpine 4.63s 64.3MB 242.8M
3.10-slim 3.21s 64.3MB 241M
3.11-alpine 6.13s 69.5MB 262.6M
3.11-slim 4.97s 69.5MB 261M
3.12-alpine 5.56s 68.4MB 252.8M
3.12-slim 5.56s 68.4MB 252M
3.13-alpine 5.09s 69.1MB 254.5M
3.13-slim 5.31s 69.1MB 254M
3.9-alpine 4.85s 56.0MB 209.6M
3.9-slim 4.49s 56.0MB 205M

This quickstart defines a simple Airflow DAG using the Task SDK's `@dag` and `@task` decorators. The `my_task` function is decorated as an Airflow task, and `example_simplest_dag` is a decorated function that defines the DAG structure and invokes the task. This pattern decouples DAG authoring from Airflow internals.

import pendulum
from airflow.sdk import dag, task

@task
def my_task():
    print("Hello from a Task SDK task!")

@dag(
    schedule=None,
    start_date=pendulum.datetime(2023, 1, 1, tz="UTC"),
    catchup=False,
    tags=["example"],
)
def example_simplest_dag():
    my_task()

example_dag_instance = example_simplest_dag()