Apache Airflow Provider for Singularity

3.9.3 verified Fri May 01 auth: no python

Apache Airflow provider for Singularity container runtime. Version 3.9.3 supports Airflow >=2.8.0 and Python >=3.10. The provider allows running tasks inside Singularity containers via the SingularityOperator. Release cadence is irregular, tracking Airflow provider releases.

pip install apache-airflow-providers-singularity

Common errors

error ModuleNotFoundError: No module named 'airflow.providers.singularity' ↓

cause Provider not installed or Airflow version <2.0

fix

Install the provider via pip install apache-airflow-providers-singularity. Ensure Airflow >=2.0.

error AirflowException: SingularityOperator requires singularity binary ↓

cause Singularity is not installed on the worker node

fix

Install Singularity (e.g., apt-get install singularity-container or build from source).

error AttributeError: 'SingularityOperator' object has no attribute 'image' ↓

cause Using outdated provider version with different parameter name

fix

Upgrade provider to latest version: pip install -U apache-airflow-providers-singularity.

Warnings

breaking Provider version 3.x requires Airflow >=2.8.0 and Python >=3.10. Older Airflow versions are incompatible. ↓

fix Upgrade Airflow to >=2.8.0 and Python to >=3.10.

deprecated The 'force_pull_image' parameter may be deprecated in future versions; use 'pull_policy' instead. ↓

fix Migrate to using 'pull_policy' parameter ("always", "if-not-present", "never").

gotcha SingularityOperator requires Singularity to be installed on the worker node. Without it, tasks fail with an executable not found error. ↓

fix Ensure Singularity is installed on all Airflow workers that run these tasks.

gotcha The provider does not automatically set environment variables for the container; use 'environment' parameter to pass variables. ↓

fix Pass environment variables via the 'environment' dict in the operator.

Imports

SingularityOperator

wrong

from airflow.contrib.operators.singularity import SingularityOperator

correct

from airflow.providers.singularity.operators.singularity import SingularityOperator

Old contrib module was removed in Airflow 2.x

SingularityHook

wrong

from airflow.hooks.singularity import SingularityHook

correct

from airflow.providers.singularity.hooks.singularity import SingularityHook

Hook moved to providers package in Airflow 2.0+

Quickstart

Minimal DAG using SingularityOperator to run a command in a container.

from datetime import datetime
from airflow import DAG
from airflow.providers.singularity.operators.singularity import SingularityOperator

with DAG(
    dag_id='singularity_example',
    start_date=datetime(2025, 1, 1),
    schedule=None,
    catchup=False,
) as dag:
    run_singularity = SingularityOperator(
        task_id='run_singularity_task',
        image='docker://busybox:latest',
        command='echo "Hello from Singularity"',
        force_pull_image=True,
    )