{"library":"apache-airflow-providers-http","title":"HTTP Provider for Apache Airflow","description":"The Apache Airflow HTTP Provider enables seamless integration with HTTP APIs within Airflow DAGs. It offers operators and hooks to send HTTP requests, handle responses, and poke API endpoints. Released independently from core Airflow, it adheres to Semantic Versioning and is currently at version 6.0.1.","status":"active","version":"6.0.1","language":"en","source_language":"en","source_url":"https://github.com/apache/airflow/tree/main/airflow/providers/http","tags":["airflow","provider","http","api","integration","operator","sensor"],"install":[{"cmd":"pip install apache-airflow-providers-http","lang":"bash","label":"Install HTTP provider"}],"dependencies":[{"reason":"Core Airflow is required for running DAGs and tasks. Version >=2.11.0 is required for this provider version.","package":"apache-airflow","optional":false},{"reason":"Provides compatibility layers for cross-provider functionalities. Version >=1.12.0 is required.","package":"apache-airflow-providers-common-compat","optional":false},{"reason":"HTTP client library used by the provider. Version >=2.32.0,<3 is required.","package":"requests","optional":false},{"reason":"Additional tools for the requests library. Version >=1.0.0 is required.","package":"requests-toolbelt","optional":false},{"reason":"Asynchronous HTTP client/server framework, used for deferrable operators. Version >=3.12.14 is required.","package":"aiohttp","optional":false},{"reason":"ASGI specification utilities, conditional on Python version (>=2.3.0 for <3.14, >=3.11.1 for >=3.14).","package":"asgiref","optional":false}],"imports":[{"note":"Old `SimpleHttpOperator` (renamed to `HttpOperator`) from `airflow.operators.http_operator` is deprecated and removed in Airflow 2.x and later provider versions. Always use the provider-specific import path.","wrong":"from airflow.operators.http_operator import SimpleHttpOperator","symbol":"HttpOperator","correct":"from airflow.providers.http.operators.http import HttpOperator"},{"note":"Similar to HttpOperator, always use the provider-specific import path for HttpSensor.","wrong":"from airflow.sensors.http_sensor import HttpSensor","symbol":"HttpSensor","correct":"from airflow.providers.http.sensors.http import HttpSensor"},{"symbol":"HttpHook","correct":"from airflow.providers.http.hooks.http import HttpHook"}],"quickstart":{"code":"from __future__ import annotations\n\nimport pendulum\n\nfrom airflow.models.dag import DAG\nfrom airflow.providers.http.operators.http import HttpOperator\nfrom airflow.providers.http.sensors.http import HttpSensor\n\n# NOTE: You need to create an HTTP connection in Airflow UI\n# Admin -> Connections -> + New Record\n# Conn Id: http_default\n# Conn Type: HTTP\n# Host: httpbin.org\n# For HTTPS, see the 'Configuring HTTPS is counter-intuitive' warning.\n\nwith DAG(\n    dag_id=\"http_example_dag\",\n    start_date=pendulum.datetime(2023, 1, 1, tz=\"UTC\"),\n    catchup=False,\n    schedule=None,\n    tags=[\"http\", \"example\"],\n) as dag:\n    # Use HttpOperator to make a GET request\n    http_get_task = HttpOperator(\n        task_id=\"http_get_request\",\n        http_conn_id=\"http_default\",\n        method=\"GET\",\n        endpoint=\"get\", # This will resolve to httpbin.org/get\n        data={\"param1\": \"value1\", \"param2\": \"value2\"},\n        response_check=lambda response: \"param1\" in response.text,\n        log_response=True,\n    )\n\n    # Use HttpSensor to wait for a specific response\n    http_sensor_task = HttpSensor(\n        task_id=\"http_sensor_check\",\n        http_conn_id=\"http_default\",\n        endpoint=\"status/200\", # This will resolve to httpbin.org/status/200\n        response_check=lambda response: response.status_code == 200,\n        poke_interval=5,\n        timeout=60,\n    )\n\n    http_get_task >> http_sensor_task\n","lang":"python","description":"This quickstart demonstrates how to use `HttpOperator` to perform a GET request and `HttpSensor` to poll an endpoint until a condition is met. Ensure you configure an Airflow HTTP connection named `http_default` pointing to `httpbin.org` (or your target API) before running."},"warnings":[{"fix":"Upgrade Apache Airflow to at least version 2.1.0 before installing or upgrading `apache-airflow-providers-http`. For provider 6.0.1, Airflow >=2.11.0 is required.","message":"Provider versions 2.0.0 and above (including 6.0.1) require Airflow 2.1.0+. This is due to the removal of the `apply_default` decorator in core Airflow. Installing a newer provider version with an older Airflow might automatically upgrade Airflow, requiring a manual `airflow upgrade db`.","severity":"breaking","affected_versions":"Provider >=2.0.0, Airflow <2.1.0"},{"fix":"Ensure your Apache Airflow installation is at least version 2.2.0. For provider 6.0.1, Airflow >=2.11.0 is required.","message":"Provider versions 3.0.0 and above require Airflow 2.2+. This aligns with the Apache Airflow providers support policy.","severity":"breaking","affected_versions":"Provider >=3.0.0, Airflow <2.2.0"},{"fix":"If this change affects existing deployments (e.g., due to network policies), you can disable it by setting `tcp_keep_alive=False` or configure keep-alive parameters using `tcp_keep_alive_*` arguments in the operator/hook/sensor constructors.","message":"From provider version 4.0.0, `TCP_KEEPALIVE` is enabled by default for `HttpOperator`, `HttpSensor`, and `HttpHook`. This prevents firewalls from closing long-running, inactive connections.","severity":"breaking","affected_versions":"Provider >=4.0.0"},{"fix":"To use HTTPS, your connection URI should look like `http://your_host:443/https`, and the specific URL path should be passed via the `endpoint` parameter. Alternatively, you can percent-encode the `https://` prefix in the host: `http://https%3a%2f%2fyour_host:443/`.","message":"Configuring HTTPS via the `HttpOperator` can be counter-intuitive due to historical implementation. The operator defaults to `http` protocol.","severity":"gotcha","affected_versions":"All versions"},{"fix":"Utilize the `response_filter` parameter to process and reduce the response size before it's stored, or implement custom logic to handle pagination and data processing in smaller chunks outside the operator's direct memory management.","message":"When using `HttpOperator` with pagination, all API responses are stored in memory and returned as a single result. This can lead to high memory and CPU consumption for large datasets.","severity":"gotcha","affected_versions":"All versions"},{"fix":"Ensure your Airflow environment is running version 2.11.0 or newer to guarantee compatibility and access to all features.","message":"The minimum required Apache Airflow version for `apache-airflow-providers-http` 6.0.1 is 2.11.0.","severity":"gotcha","affected_versions":"Provider 6.0.1"}],"env_vars":null,"last_verified":"2026-04-05T00:00:00.000Z","next_check":"2026-07-04T00:00:00.000Z"}