Apache Airflow Jenkins Provider

4.2.5 verified Thu Apr 16 auth: no python

The Apache Airflow Jenkins Provider package integrates Apache Airflow with Jenkins, a popular open-source automation server. It allows users to orchestrate and monitor Jenkins jobs directly from Airflow Directed Acyclic Graphs (DAGs), enabling seamless integration of CI/CD pipelines and data workflows. The current version is 4.2.5, and it follows the release cadence of the Apache Airflow provider ecosystem.

pip install apache-airflow-providers-jenkins

Common errors

error ModuleNotFoundError: No module named 'airflow.contrib.operators.jenkins_operator' ↓

cause Using an outdated import path for Jenkins operators/hooks that were part of `airflow.contrib` in Airflow 1.x. In Airflow 2.x, these components moved to dedicated provider packages.

fix

Update your import statements to use the correct provider path: from airflow.providers.jenkins.operators.jenkins_job_trigger import JenkinsJobTriggerOperator.

error airflow.exceptions.AirflowException: The conn_id `jenkins_default` isn't defined. ↓

cause The Jenkins connection specified in the operator's `jenkins_connection_id` parameter has not been configured in the Airflow UI or `airflow.cfg`.

fix

Navigate to Admin -> Connections in the Airflow UI, click + to create a new connection. Set Conn Id to jenkins_default, Conn Type to Jenkins, and fill in Host, Login, and Password for your Jenkins server.

error jenkins.JenkinsException: Error in request. Status code: 404 ↓

cause The specified Jenkins job name either does not exist on the configured Jenkins server or the Jenkins user lacks the necessary permissions to access/trigger it.

fix

Verify the job_name in your DAG matches an existing job in Jenkins. Also, ensure the Jenkins user configured in your Airflow connection has 'Build' and 'Read' permissions for that specific job.

Warnings

breaking Provider version 3.0.0 and above explicitly requires Apache Airflow 2.2+. Attempting to install on older Airflow versions may lead to unexpected behavior or automatic Airflow upgrades. ↓

fix Upgrade your Apache Airflow environment to version 2.2.0 or higher before installing or upgrading to `apache-airflow-providers-jenkins` 3.0.0+.

breaking Provider version 2.0.0 introduced breaking changes related to the removal of the `apply_default` decorator, requiring Airflow 2.1.0+. If your Airflow version is older, an automatic upgrade might occur, necessitating a manual `airflow upgrade db`. ↓

fix Ensure your Apache Airflow is at least version 2.1.0. If upgrading from pre-2.1.0, manually run `airflow upgrade db` after the upgrade.

gotcha When installing Airflow providers, especially with older pip versions (e.g., pip 20.3 in late 2020), dependency resolution issues might occur. This was due to changes in pip's dependency resolver. ↓

fix For modern Airflow (2.x+), ensure you have a recent pip version (>=21.0). If encountering issues, try upgrading pip (`pip install --upgrade pip`) or in older environments, use `pip install --use-deprecated=legacy-resolver` or downgrade pip to `20.2.4`.

Imports

JenkinsHook

from airflow.providers.jenkins.hooks.jenkins import JenkinsHook

JenkinsJobTriggerOperator

from airflow.providers.jenkins.operators.jenkins_job_trigger import JenkinsJobTriggerOperator

JenkinsBuildSensor

wrong

from airflow.contrib.sensors.jenkins_build_sensor import JenkinsBuildSensor

correct

from airflow.providers.jenkins.sensors.jenkins_build import JenkinsBuildSensor

Old contrib imports are deprecated and should use the new provider path for Airflow 2+.

Quickstart

This quickstart demonstrates how to trigger a Jenkins job using the `JenkinsJobTriggerOperator`. Before running, configure a Jenkins connection named 'jenkins_default' in your Airflow UI with the appropriate Jenkins server URL, username, and password. The example assumes a Jenkins job named 'my_sample_jenkins_job' exists and accepts 'PARAM1' and 'PARAM2' as build parameters.

from __future__ import annotations
import pendulum
from airflow.models.dag import DAG
from airflow.providers.jenkins.operators.jenkins_job_trigger import JenkinsJobTriggerOperator

# Ensure you have a Jenkins connection configured in Airflow UI:
# Conn Id: jenkins_default
# Conn Type: Jenkins
# Host: http://your-jenkins-server.com:8080
# Login: your_jenkins_username
# Password: your_jenkins_password

with DAG(
    dag_id='jenkins_job_example',
    start_date=pendulum.datetime(2023, 1, 1, tz="UTC"),
    catchup=False,
    schedule=None,
    tags=['jenkins', 'example'],
) as dag:
    trigger_jenkins_job = JenkinsJobTriggerOperator(
        task_id='trigger_my_jenkins_job',
        jenkins_connection_id='jenkins_default',
        job_name='my_sample_jenkins_job',
        parameters={'PARAM1': 'value1', 'PARAM2': 'value2'},
        allowed_jenkins_states=['SUCCESS', 'UNSTABLE'],
        # Optional: You can also specify build_params for a specific build trigger
        # build_params={'token': 'YOUR_JENKINS_JOB_TOKEN'}
    )