{"id":1331,"library":"apache-airflow-providers-sqlite","title":"Apache Airflow SQLite Provider","description":"The `apache-airflow-providers-sqlite` package provides the necessary components to interact with SQLite databases within Apache Airflow DAGs. It includes the `SqliteHook` for programmatic access and `SqliteOperator` for defining tasks. This provider is currently at version 4.3.1 and typically releases alongside major Apache Airflow provider updates.","status":"active","version":"4.3.1","language":"en","source_language":"en","source_url":"https://github.com/apache/airflow/tree/main/airflow/providers/sqlite","tags":["airflow","database","sqlite","provider"],"install":[{"cmd":"pip install apache-airflow-providers-sqlite","lang":"bash","label":"Install provider"}],"dependencies":[{"reason":"Required for provider functionality; this package extends Apache Airflow core.","package":"apache-airflow","optional":false}],"imports":[{"symbol":"SqliteHook","correct":"from airflow.providers.sqlite.hooks.sqlite import SqliteHook"},{"symbol":"SqliteOperator","correct":"from airflow.providers.sqlite.operators.sqlite import SqliteOperator"}],"quickstart":{"code":"from __future__ import annotations\nimport pendulum\nfrom airflow.models.dag import DAG\nfrom airflow.providers.sqlite.operators.sqlite import SqliteOperator\n\nwith DAG(\n    dag_id=\"sqlite_example_dag\",\n    start_date=pendulum.datetime(2023, 1, 1, tz=\"UTC\"),\n    schedule=None,\n    catchup=False,\n    tags=[\"sqlite\", \"example\"],\n) as dag:\n    create_table = SqliteOperator(\n        task_id=\"create_table\",\n        sqlite_conn_id=\"sqlite_default\",\n        sql=\"\"\"\n            CREATE TABLE IF NOT EXISTS test_table (\n                id INTEGER PRIMARY KEY,\n                name TEXT\n            );\n        \"\"\",\n    )\n\n    insert_data = SqliteOperator(\n        task_id=\"insert_data\",\n        sqlite_conn_id=\"sqlite_default\",\n        sql=\"INSERT INTO test_table (name) VALUES ('Airflow'), ('SQLite');\",\n    )\n\n    query_data = SqliteOperator(\n        task_id=\"query_data\",\n        sqlite_conn_id=\"sqlite_default\",\n        sql=\"SELECT * FROM test_table;\",\n        handler=lambda x: [print(row) for row in x] # Example handler to print results to logs\n    )\n\n    create_table >> insert_data >> query_data\n","lang":"python","description":"This quickstart demonstrates a basic Airflow DAG using the `SqliteOperator` to create a table, insert data, and query it. Ensure an Airflow connection named `sqlite_default` is configured, or it will default to an in-memory database. For a persistent database file, specify the path in the 'Host' field of the connection or in the 'Extra' field (e.g., `{\"database\":\"/path/to/my.db\"}`)."},"warnings":[{"fix":"Configure the `sqlite_default` Airflow connection to specify a file path for the 'Host' field or in the 'Extra' field as `{\"database\":\"/path/to/my.db\"}`.","message":"SQLite database persistence and location: By default, if the `sqlite_default` connection's 'Host' or 'Extra' field is not configured with a database path, the tasks will use an in-memory SQLite database (`:memory:`). This means all data will be lost after the task completes. For persistent storage, you *must* specify a file path (e.g., `/opt/airflow/dags/data/my_db.db`) in the Airflow connection.","severity":"gotcha","affected_versions":"All versions"},{"fix":"Use SQLite only for single-node Airflow environments or for temporary, task-local data. For shared, persistent state in distributed setups, consider a client-server database like PostgreSQL or MySQL.","message":"Not suitable for distributed Airflow: SQLite is a file-based database. In distributed Airflow setups (e.g., using CeleryExecutor or KubernetesExecutor with multiple workers), the SQLite database file must reside on a shared, mounted filesystem accessible by all workers. Otherwise, each worker will operate on its own independent copy of the database, leading to data inconsistency. It's generally recommended only for single-node Airflow deployments or for ephemeral, task-local data.","severity":"gotcha","affected_versions":"All versions"},{"fix":"Ensure your Airflow environment is version 2.0 or newer. Upgrade Airflow if necessary.","message":"Airflow 1.x incompatibility: This provider package (like all `apache-airflow-providers-*` packages) is designed exclusively for Apache Airflow 2.0 and later. Attempting to install or use it in an Airflow 1.x environment will result in import errors and general incompatibility.","severity":"breaking","affected_versions":"< 2.0.0 (Airflow)"},{"fix":"Implement custom sensing logic using a `PythonOperator` calling the `SqliteHook` or combine with other Airflow features like `ExternalTaskSensor` if monitoring external processes that modify the database.","message":"No SQLite Sensor: Unlike some other database providers (e.g., Postgres, MySQL), the `apache-airflow-providers-sqlite` package currently only offers `SqliteHook` and `SqliteOperator`. There is no dedicated `SqliteSensor` available for polling database states directly.","severity":"gotcha","affected_versions":"All versions"}],"env_vars":null,"last_verified":"2026-04-09T00:00:00.000Z","next_check":"2026-07-08T00:00:00.000Z"}