ADBC Driver for Apache Arrow Flight SQL

1.11.0 · active · verified Thu Apr 16

adbc-driver-flightsql is a Python driver for connecting to databases via Apache Arrow Flight SQL. It provides both a low-level ADBC API and a DBAPI 2.0-compatible interface for efficient, Arrow-native data transfer. The library is actively maintained as part of the Apache Arrow ADBC project, with frequent releases, typically on a monthly or bi-monthly cadence. The current version is 1.11.0.

Common errors

```
ModuleNotFoundError: No module named 'adbc_driver_flightsql.dbapi'
```
cause The `adbc-driver-flightsql` package is not installed or the import path is incorrect.

fix Ensure the package is installed: `pip install adbc-driver-flightsql`. Verify the import statement is `from adbc_driver_flightsql.dbapi import connect`.
```
gRPC RPC Error: 14 UNAVAILABLE: Connect Failed
```
cause The Flight SQL server specified in the URI is unreachable, not running, or blocking the connection (e.g., firewall).

fix Check the Flight SQL server's status, ensure the URI (host and port) is correct, and verify network connectivity between your client and the server. Also check for firewall rules.
```
gRPC RPC Error: 16 UNAUTHENTICATED: No authentication configured
```
cause The Flight SQL server requires authentication, but no credentials or an invalid authentication header were provided by the client.

fix Provide valid authentication details via `db_kwargs` in the `connect` function, such as `db_kwargs={'username': 'your_user', 'password': 'your_password'}` or `db_kwargs={'adbc.flight.sql.authorization_header': 'Bearer <your_token>'}`.
```
TypeError: Cannot convert value of type <class 'pandas.core.frame.DataFrame'> to an Arrow Table.
```
cause Attempting to use a pandas DataFrame (or similar non-Arrow object) directly with ADBC functions that expect an Arrow Table or RecordBatch, especially during bulk ingestion.

fix Convert your data to an `pyarrow.Table` or `pyarrow.RecordBatch` before passing it to ADBC functions. For example, `arrow_table = pa.Table.from_pandas(df)`.

Warnings

gotcha Authentication is not enabled by default. You must explicitly configure authentication using database options (e.g., 'username', 'password', or 'adbc.flight.sql.authorization_header') for secured Flight SQL endpoints.
Fix: Pass appropriate `db_kwargs` or `conn_kwargs` to the `connect` function, such as `db_kwargs={'username': 'user', 'password': 'pass'}` or `db_kwargs={'adbc.flight.sql.authorization_header': 'Bearer <token>'}`. Consult your Flight SQL server's documentation for required authentication schemes.
gotcha The driver implements various timeouts (connection, query execution, data fetching, data upload). Long-running operations may fail without explicit timeout configuration. Default timeouts might be too short for large datasets or complex queries.
Fix: Set specific timeout options via `db_kwargs` or `conn_kwargs`. For example, `db_kwargs={'adbc.flight.sql.rpc.timeout_seconds.connect': '60', 'adbc.flight.sql.rpc.timeout_seconds.query': '120'}` to increase connection and query timeouts respectively.
deprecated The API for bulk ingestion changed significantly in version 1.10.0 with the introduction of a new convenience API. Older patterns for bulk ingestion might still work but are less efficient or recommended.
Fix: Upgrade to version 1.10.0 or later and utilize the new `adbc_ingest` method for bulk data loading, which offers improved performance and automatic schema inference.
gotcha PyArrow is a core dependency for practical use, especially when working with query results. While `adbc-driver-flightsql` can be installed alone, most operations that return data will expect `pyarrow` to be present for Arrow Table/RecordBatch handling.
Fix: Always install `adbc-driver-flightsql` alongside `pyarrow`: `pip install adbc-driver-flightsql pyarrow`.

Install

pip install adbc-driver-flightsql pyarrow Install with PyArrow

Imports

connect
wrong:
```
import adbc_driver_flightsql
```
correct:
```
from adbc_driver_flightsql.dbapi import connect
```
The primary interface for creating connections is the 'connect' function from the dbapi submodule.

Quickstart

This quickstart demonstrates connecting to an Apache Arrow Flight SQL server using the DBAPI 2.0 interface, executing a simple query, and fetching results as an Arrow Table. Connection details and authentication are typically passed via the URI and `db_kwargs` parameter, using environment variables for sensitive information.

import os
from adbc_driver_flightsql.dbapi import connect

# Replace with your Flight SQL server URI and authentication details
FLIGHT_SQL_URI = os.environ.get('ADBC_FLIGHT_SQL_URI', 'grpc+tls://localhost:50050')
FLIGHT_SQL_USERNAME = os.environ.get('ADBC_FLIGHT_SQL_USERNAME', '')
FLIGHT_SQL_PASSWORD = os.environ.get('ADBC_FLIGHT_SQL_PASSWORD', '')

db_options = {}
if FLIGHT_SQL_USERNAME and FLIGHT_SQL_PASSWORD:
    db_options['username'] = FLIGHT_SQL_USERNAME
    db_options['password'] = FLIGHT_SQL_PASSWORD

try:
    with connect(FLIGHT_SQL_URI, db_kwargs=db_options) as conn:
        with conn.cursor() as cursor:
            cursor.execute('SELECT 1 as example_column')
            result = cursor.fetch_arrow_table()
            print('Query successful!')
            print(result.to_pandas())
except Exception as e:
    print(f"Error connecting or querying: {e}")
    print("Please ensure your Flight SQL server is running and accessible,")
    print("and ADBC_FLIGHT_SQL_URI, USERNAME, and PASSWORD are correctly set if required.")

view raw JSON →