Apache Airflow Provider for Apache Pinot

4.10.2 verified Fri May 01 auth: no python

This provider allows Apache Airflow to integrate with Apache Pinot, enabling data transfers, SQL execution, and cluster management. Current version 4.10.2 works with Python >=3.10 and Airflow >=2.10. Releases are on demand.

pip install apache-airflow-providers-apache-pinot

Common errors

error ModuleNotFoundError: No module named 'airflow.hooks.pinot_hook' ↓

cause Trying to import PinotDbApiHook from the old path after upgrading provider.

fix

Change import to: from airflow.providers.apache.pinot.hooks.pinot import PinotDbApiHook

error airflow.exceptions.AirflowException: The conn_type 'pinot' is not supported ↓

cause The Pinot connection type is not registered because the provider is not installed or missing pinotdb dependency.

fix

Install the provider and its dependency: pip install apache-airflow-providers-apache-pinot

error AttributeError: module 'airflow.providers.apache.pinot.hooks.pinot' has no attribute 'PinotDbApiHook' ↓

cause The hook class is named differently (e.g., 'PinotHook' in older versions) or the module is corrupted.

fix

Use the correct class name: PinotDbApiHook (introduced in provider 2.0.0). Also ensure the provider is updated: pip install --upgrade apache-airflow-providers-apache-pinot

Warnings

breaking Removal of 'pinot_hook' import path: In provider version 2.0.0, the Pinot hook was moved from airflow.hooks.pinot_hook to airflow.providers.apache.pinot.hooks.pinot. Direct imports from the old path will fail. ↓

fix Update imports to use the new provider.module path: from airflow.providers.apache.pinot.hooks.pinot import PinotDbApiHook

breaking Minimum Airflow version increased to 2.10. Starting from provider version 4.0.0, Airflow >=2.10 is required. Older Airflow versions will raise ImportError. ↓

fix Upgrade Apache Airflow to version >=2.10.

gotcha Password provided via extra parameter not connection: The PinotDbApiHook does not support password through the standard connection ``password`` field. Instead, pass it via the ``extra`` parameter as JSON: ``{"password": "mypassword"}``. Otherwise authentication may silently fail. ↓

fix When creating a Pinot connection in Airflow UI, set 'Extra' to '{"password": "..."}' instead of using the 'Password' field.

Imports

PinotAdminHook

wrong

from airflow.hooks.pinot_admin import PinotAdminHook

correct

from airflow.providers.apache.pinot.hooks.pinot_admin import PinotAdminHook

Old import path removed in provider version 2.0.0

PinotDbApiHook

wrong

from airflow.hooks.pinot_hook import PinotDbApiHook

correct

from airflow.providers.apache.pinot.hooks.pinot import PinotDbApiHook

Old import path no longer supported

PinotTableCreateOrUpdateOperator

from airflow.providers.apache.pinot.operators.pinot_table_create_or_update import PinotTableCreateOrUpdateOperator

Quickstart

Basic DAG using Pinot operators and hooks. Requires a running Pinot cluster.

from datetime import datetime
from airflow import DAG
from airflow.providers.apache.pinot.hooks.pinot import PinotDbApiHook
from airflow.providers.apache.pinot.operators.pinot_table_create_or_update import PinotTableCreateOrUpdateOperator

with DAG(
    dag_id='pinot_example',
    start_date=datetime(2025, 1, 1),
    catchup=False,
) as dag:
    # Example: create a Pinot table
    create_table = PinotTableCreateOrUpdateOperator(
        task_id='create_pinot_table',
        table_config={
            'tableName': 'myTable',
            'tableType': 'OFFLINE',
            'segmentsConfig': {
                'replication': '1',
                'schemaName': 'mySchema'
            }
        },
        pinot_admin_hook=PinotAdminHook(pinot_admin_host='http://localhost:9000'),
    )