{"library":"apache-airflow-providers-amazon","title":"Apache Airflow Amazon Provider","description":"The `apache-airflow-providers-amazon` package extends Apache Airflow by providing operators, hooks, and sensors for seamless integration with various Amazon Web Services (AWS). It enables orchestrating workflows involving services like Amazon S3, Redshift, EMR, SQS, and more. The provider is actively maintained by the Apache Airflow community with frequent releases, typically every few weeks.","status":"active","version":"9.24.0","language":"en","source_language":"en","source_url":"https://github.com/apache/airflow/tree/main/airflow/providers/amazon","tags":["airflow","aws","amazon","provider","cloud","orchestration","etl"],"install":[{"cmd":"pip install apache-airflow-providers-amazon","lang":"bash","label":"Install core provider"},{"cmd":"pip install 'apache-airflow-providers-amazon[s3fs,cncf.kubernetes]' # Example with optional extras","lang":"bash","label":"Install with optional extras"}],"dependencies":[{"reason":"Core Apache Airflow library; required for provider functionality.","package":"apache-airflow","optional":false},{"reason":"AWS SDK for Python; typically a transitive dependency but essential for AWS interactions.","package":"boto3","optional":false},{"reason":"Required by some operators (e.g., SQL to S3) for data manipulation, can be installed via `[pandas]` extra.","package":"pandas","optional":true}],"imports":[{"symbol":"S3Hook","correct":"from airflow.providers.amazon.aws.hooks.s3 import S3Hook"},{"symbol":"S3CopyObjectOperator","correct":"from airflow.providers.amazon.aws.operators.s3 import S3CopyObjectOperator"},{"symbol":"S3KeySensor","correct":"from airflow.providers.amazon.aws.sensors.s3 import S3KeySensor"},{"note":"The `ecs_operator` module was refactored, and components moved to `ecs` module in some versions.","wrong":"from airflow.providers.amazon.aws.operators.ecs_operator import EcsOperator","symbol":"EcsOperator","correct":"from airflow.providers.amazon.aws.operators.ecs import EcsOperator"}],"quickstart":{"code":"from __future__ import annotations\n\nimport os\n\nimport pendulum\n\nfrom airflow.models.dag import DAG\nfrom airflow.providers.amazon.aws.operators.s3 import S3CopyObjectOperator\n\n# Ensure AWS credentials are configured in Airflow Connection 'aws_default'\n# or via environment variables (AWS_ACCESS_KEY_ID, AWS_SECRET_ACCESS_KEY, AWS_REGION)\n# For local testing, you can mock this or use dummy values if S3 interaction isn't critical.\n\nwith DAG(\n    dag_id=\"example_s3_copy_dag\",\n    start_date=pendulum.datetime(2023, 1, 1, tz=\"UTC\"),\n    catchup=False,\n    schedule=None,\n    tags=[\"s3\", \"aws\", \"example\"],\n) as dag:\n    copy_s3_object = S3CopyObjectOperator(\n        task_id=\"copy_s3_object_task\",\n        source_bucket_key=f\"s3://{os.environ.get('SOURCE_BUCKET', 'my-source-bucket')}/source_file.txt\",\n        dest_bucket_key=f\"s3://{os.environ.get('DEST_BUCKET', 'my-dest-bucket')}/dest_file.txt\",\n        aws_conn_id=\"aws_default\", # Assumes 'aws_default' connection is configured in Airflow\n    )\n","lang":"python","description":"This example demonstrates a basic Airflow DAG that uses the `S3CopyObjectOperator` from the Amazon provider to copy an object between two S3 buckets. Ensure you have an Airflow connection named `aws_default` configured with appropriate AWS credentials and permissions, or set AWS environment variables."},"warnings":[{"fix":"Upgrade your Apache Airflow environment to at least version 2.11.0. Consult the Airflow documentation for upgrade paths.","message":"Provider versions 9.x and above (including 9.24.0) require Apache Airflow version 2.11.0 or greater. Attempting to install on older Airflow versions will lead to compatibility issues or automatic Airflow upgrades.","severity":"breaking","affected_versions":"9.0.0+"},{"fix":"Update existing DAGs to use `cloudformation_parameters` instead of `params` for these operators.","message":"For provider versions 3.0.0 and above, the `CloudFormationCreateStackOperator` and `CloudFormationDeleteStackOperator` had their `params` argument renamed to `cloudformation_parameters`. This was to avoid a clash with Airflow 2.2+'s DAG `params` field.","severity":"breaking","affected_versions":"3.0.0+"},{"fix":"Use the `impersonation_chain` parameter for impersonation in affected operators.","message":"In provider version 9.24.0, the deprecated `delegate_to` parameter has been removed from `GCSToS3Operator`, `GlacierToGCSOperator`, and `GoogleApiToS3Operator`.","severity":"breaking","affected_versions":"9.24.0"},{"fix":"Refer to the latest Airflow AWS provider documentation for recommended connection configurations, preferring `endpoint_url` in `extra` where applicable.","message":"The use of `aws_access_key_id` and `aws_secret_access_key` directly in AWS connection `extra` fields is being deprecated in favor of `endpoint_url` for some connections.","severity":"deprecated","affected_versions":"4.1.0+"},{"fix":"Always explicitly include a `--constraint` statement in your `requirements.txt` for MWAA environments.","message":"When running on Amazon MWAA with Airflow v2.7.2+, your `requirements.txt` file should include a `--constraint` statement. If omitted, MWAA will apply a default constraint file, which might not align with desired provider versions.","severity":"gotcha","affected_versions":"Airflow 2.7.2+"}],"env_vars":null,"last_verified":"2026-04-06T20:53:46.482Z","next_check":"2026-07-05T00:00:00.000Z"}