{"id":6987,"library":"apache-airflow-providers-grpc","title":"Apache Airflow gRPC Provider","description":"The `apache-airflow-providers-grpc` package extends Apache Airflow, enabling interaction with gRPC-based services. It provides operators and hooks to execute gRPC commands and manage connections within Airflow workflows. Currently at version 3.9.4, this provider is actively maintained with frequent releases aligning with the broader Apache Airflow provider release schedule, ensuring compatibility and introducing new features.","status":"active","version":"3.9.4","language":"en","source_language":"en","source_url":"https://github.com/apache/airflow/tree/main/airflow/providers/grpc","tags":["apache-airflow","grpc","provider","orchestration","data-pipeline"],"install":[{"cmd":"pip install apache-airflow-providers-grpc","lang":"bash","label":"Install latest version"}],"dependencies":[{"reason":"Core Airflow functionality; requires Airflow >=2.11.0 for provider version 3.9.0+.","package":"apache-airflow","optional":false},{"reason":"Underlying gRPC Python library for communication.","package":"grpcio","optional":false},{"reason":"Required for Google-specific gRPC authentication modes (e.g., JWT_GOOGLE, OATH_GOOGLE).","package":"google-auth","optional":true},{"reason":"Required for Google-specific gRPC authentication modes (e.g., JWT_GOOGLE, OATH_GOOGLE).","package":"google-auth-httplib2","optional":true}],"imports":[{"symbol":"GrpcOperator","correct":"from airflow.providers.grpc.operators.grpc import GrpcOperator"},{"note":"The `contrib` path was for Airflow 1.x backport providers. Airflow 2.x+ providers use `airflow.providers.grpc.hooks.grpc`.","wrong":"from airflow.contrib.hooks.grpc_hook import GrpcHook","symbol":"GrpcHook","correct":"from airflow.providers.grpc.hooks.grpc import GrpcHook"}],"quickstart":{"code":"from __future__ import annotations\n\nimport os\nfrom datetime import datetime\n\nfrom airflow.models.dag import DAG\nfrom airflow.providers.grpc.operators.grpc import GrpcOperator\nfrom airflow.providers.grpc.hooks.grpc import GrpcHook\n\n# NOTE: In a real scenario, you would generate protobuf stub classes\n# For this example, we assume a 'dummy_pb2_grpc' and 'dummy_pb2' exist\n# and define a 'DummyService' with a 'Ping' method that takes 'PingRequest'\n# and returns 'PingResponse'.\n# Replace with your actual generated proto files and service/method names.\n\ntry:\n    import grpc\n    from dummy_pb2_grpc import DummyServiceStub\n    from dummy_pb2 import PingRequest, PingResponse\nexcept ImportError:\n    # Placeholder for running the quickstart without actual protobufs\n    class DummyServiceStub:\n        def __init__(self, channel):\n            pass\n        def Ping(self, request):\n            print(f\"Mock Ping called with: {request.message}\")\n            return type('PingResponse', (object,), {'message': 'Pong from mock!'})\n    class PingRequest:\n        def __init__(self, message=None):\n            self.message = message\n    PingResponse = type('PingResponse', (object,), {'message': None})\n    print(\"Warning: Protobuf stubs (dummy_pb2_grpc, dummy_pb2) not found. Using mock classes.\")\n\n\nwith DAG(\n    dag_id='grpc_example_dag',\n    start_date=datetime(2023, 1, 1),\n    schedule=None,\n    catchup=False,\n    tags=['grpc', 'example'],\n) as dag:\n    # Define a gRPC connection in Airflow UI (Admin -> Connections)\n    # Conn Id: 'grpc_default'\n    # Conn Type: 'gRPC Connection'\n    # Host: 'localhost'\n    # Port: '50051'\n    # Auth Type: 'NO_AUTH' for insecure channel, or 'SSL'/'TLS' with 'Credential Pem File'\n    # For JWT_GOOGLE or OATH_GOOGLE, set these and ensure 'google-auth' is installed\n\n    # Example of using GrpcOperator\n    # This assumes 'grpc_default' connection is configured in Airflow\n    call_grpc_service = GrpcOperator(\n        task_id='call_grpc_service',\n        grpc_conn_id='grpc_default',\n        stub_class=DummyServiceStub, # Your generated gRPC stub class\n        call_func='Ping', # The method on the stub class to call\n        data={'request': PingRequest(message='Hello gRPC from Airflow!')},\n        # If the gRPC method expects a specific request object, pass it like this.\n        # The 'data' parameter maps to the keyword arguments of the gRPC method.\n    )\n\n    # Example of using GrpcHook directly in a PythonOperator (or for custom logic)\n    def _interact_with_grpc_hook(**kwargs):\n        hook = GrpcHook(grpc_conn_id='grpc_default')\n        # For this example, we're using a mock service. In a real scenario,\n        # hook.run() would return a gRPC response or yield streaming responses.\n        response = hook.run(stub_class=DummyServiceStub, call_func='Ping', data={'request': PingRequest(message='Hello from Hook!')})\n        # For unary calls, response is typically the deserialized message\n        # For streaming, it's an iterator.\n        print(f\"gRPC Hook Response: {response.message}\")\n\n    interact_with_hook = GrpcOperator(\n        task_id='interact_with_grpc_hook',\n        grpc_conn_id='grpc_default',\n        stub_class=DummyServiceStub, \n        call_func='Ping',\n        data={'request': PingRequest(message='Hello from Hook via Operator!')}\n    )","lang":"python","description":"This quickstart demonstrates a basic Airflow DAG using `GrpcOperator` to interact with a gRPC service. It assumes a gRPC connection named `grpc_default` is configured in the Airflow UI (Admin -> Connections). The example uses placeholder protobuf stubs (`dummy_pb2_grpc`, `dummy_pb2`) for illustration; in a real application, these would be generated from your `.proto` files. The `GrpcOperator` calls a specified gRPC method (`Ping` on `DummyServiceStub`) with structured data. A `GrpcHook` can also be used directly within a `PythonOperator` for more complex interactions."},"warnings":[{"fix":"Upgrade your Apache Airflow instance to at least version 2.1.0 before installing or upgrading to `apache-airflow-providers-grpc` version 2.0.0 or higher. For the latest provider version (3.9.0+), Airflow >=2.11.0 is required.","message":"Provider version 2.0.0+ requires Apache Airflow 2.1.0+ due to the removal of the `apply_default` decorator. Older Airflow versions will cause automatic upgrades or database migration issues.","severity":"breaking","affected_versions":">=2.0.0"},{"fix":"Always check the provider's changelog or documentation for the `minimum_airflow_version` compatible with the specific provider version you intend to use. Ensure your Airflow environment meets this requirement.","message":"Minimum supported Airflow version has incrementally increased. Provider version 3.0.0+ requires Airflow 2.2+, version 3.1.0+ requires Airflow 2.3+, and version 3.9.0+ requires Airflow 2.11+.","severity":"breaking","affected_versions":">=3.0.0"},{"fix":"Ensure your Python environment is version 3.10 or higher when using `apache-airflow-providers-grpc` versions 3.8.1 or newer, as indicated by the `requires_python: >=3.10` metadata on PyPI.","message":"Python 3.9 support was dropped in provider version 3.8.1.","severity":"breaking","affected_versions":">=3.8.1"},{"fix":"Always pass the gRPC method's expected request object within the `data` dictionary, e.g., `data={'request': YourProtoRequestObject(field='value')}`. Do not pass raw dictionaries directly if the gRPC method expects a protobuf message.","message":"The `data` parameter in `GrpcOperator` expects a dictionary where keys map to keyword arguments of the gRPC method, and values are the corresponding protobuf request objects. Passing incorrect types or structures will lead to serialization errors.","severity":"gotcha","affected_versions":"All"}],"env_vars":null,"last_verified":"2026-04-16T00:00:00.000Z","next_check":"2026-07-15T00:00:00.000Z","problems":[{"fix":"Construct the correct protobuf request object (e.g., `YourService_pb2.YourRequest(field='value')`) and pass it within the `data` dictionary, typically under a 'request' key: `data={'request': YourService_pb2.YourRequest(...) }`.","cause":"Attempting to pass a raw Python dictionary directly to a gRPC method via `GrpcOperator` or `GrpcHook` when the method expects a protobuf message. The `data` parameter needs the actual protobuf object.","error":"TypeError: 'dict' object is not callable or expecting protobuf message"},{"fix":"Verify the gRPC connection details in Airflow UI (Admin -> Connections -> `grpc_default` or your custom connection ID). Ensure the 'Host' and 'Port' are correct, the gRPC server is running and accessible from the Airflow worker, and the 'Auth Type' and associated credentials (e.g., 'Credential Pem File' or 'Scopes') are correctly configured and match the server's requirements.","cause":"Airflow's `GrpcHook` or `GrpcOperator` failed to establish a connection with the gRPC server. This could be due to an incorrect host/port, the server not running, network issues, or authentication failures.","error":"grpc._channel._InactiveRpcError: <_InactiveRpcError of RPC that terminated with:\n\tstatus = StatusCode.UNAVAILABLE\n\tdetails = \"Connect Failed\"\n\tdebug_error_string = \"...\""}]}