PyAirbyte

Warnings

• breaking The PyAirbyte MCP (Model Context Protocol) Server is currently experimental. Its API and features may change significantly or be entirely refactored without notice between minor versions of PyAirbyte. Avoid using it in production environments where stability is critical.
  Fix: Monitor official PyAirbyte documentation and GitHub releases for updates on the MCP Server's stability. For production, rely on stable APIs.
• gotcha While PyAirbyte itself supports Python 3.10+, specific Airbyte connectors might have stricter Python version requirements or dependencies. If you encounter issues, consider explicitly setting the Python version for the connector via the `use_python` argument in `ab.get_source()` or `ab.get_destination()`. Alternatively, using `docker_image=True` can provide greater stability by leveraging Docker.
  Fix: When initializing a source or destination, use `ab.get_source(..., use_python="3.11")` or `ab.get_source(..., docker_image=True)` to control the connector's execution environment.
• gotcha Java-based Airbyte destination connectors (which typically run as Docker containers) require Docker to be installed and running on your system. For greater portability and Python-native execution, consider utilizing SQL-based caches (e.g., DuckDB, Postgres, Snowflake) directly within PyAirbyte, if your destination is a SQL database.
  Fix: Install Docker for Java-based destinations. For SQL databases, use PyAirbyte's built-in SQL caching mechanisms instead of destination connectors for a more Python-native experience: `cache = ab.get_destination_cache('postgres', config=...)`.
• gotcha Starting with PyAirbyte version 0.29.0, the library defaults to using `uv` instead of `pip` for installing Python-based connectors, offering significant speed improvements. If `uv` causes unexpected issues or conflicts in your environment, you can force PyAirbyte to fall back to `pip` for connector installations.
  Fix: Set the environment variable `AIRBYTE_NO_UV=true` before running your PyAirbyte code to revert to `pip`-based connector installation: `os.environ['AIRBYTE_NO_UV'] = 'true'`.

Install

• pip install airbyte Install PyAirbyte

Imports

• airbyte

  import airbyte as ab

  The 'ab' alias is the recommended and most common convention in PyAirbyte documentation and examples for concise and readable code.

Quickstart

This quickstart demonstrates how to connect to a data source (using `source-faker` for demonstration), verify its configuration, extract data, and load it into a Pandas DataFrame. It uses the recommended `import airbyte as ab` convention and shows basic steps for data ingestion.

import airbyte as ab
import os

# Configure a source (e.g., source-faker for demo purposes)
# For real connectors, replace 'source-faker' with the actual connector name
# and 'config' with your credentials/connection details.
# Use os.environ.get for sensitive information.
source = ab.get_source(
    "source-faker", 
    config={
        "count": 1000, # Number of records to generate
        "seed": 42    # Seed for reproducible data generation
    },
    install_if_missing=True # Automatically install the connector if not found
)

# Verify configuration and connection
check_result = source.check()
if check_result.status == "succeeded":
    print("Source connection successful!\n")
else:
    print(f"Source connection failed: {check_result.message}\n")
    exit()

# Select all available streams from the source
source.select_all_streams()

# Read data from the source into PyAirbyte's internal cache (DuckDB by default)
read_result = source.read()

# Access a specific stream and convert it to a Pandas DataFrame
# Replace 'users' with the actual stream name from your source
users_df = read_result["users"].to_pandas()

print("First 5 records from 'users' stream:")
print(users_df.head())