Apache Airflow Provider for Pinecone

2.4.4 verified Fri May 01 auth: no python

Apache Airflow provider for integrating with Pinecone vector database. Version 2.4.4 supports Airflow 2.x and Python >=3.10. Provides hooks and operators to manage Pinecone indexes, upsert vectors, and run queries. Release cadence follows Airflow's provider releases.

pip install apache-airflow-providers-pinecone

Common errors

error ModuleNotFoundError: No module named 'airflow.providers.pinecone' ↓

cause Provider package not installed or Airflow is too old (<2.0) and doesn't support providers.

fix

Install the provider: pip install apache-airflow-providers-pinecone. Ensure you are using Airflow 2.0+.

error AttributeError: 'PineconeHook' object has no attribute 'create_index' ↓

cause Using an older version of the provider (<2.4.0) where create_index was not exposed.

fix

Upgrade to the latest provider: pip install --upgrade apache-airflow-providers-pinecone

error Connection to Pinecone failed: SSL error ↓

cause Missing or incorrect Pinecone environment in connection. The environment must match your Pinecone project (e.g., 'us-west1-gcp').

fix

In Airflow UI, edit the Pinecone connection to include the correct environment, ensure API key is set, and check network/firewall settings.

Warnings

breaking In version 2.4.0, the hook signature changed: `PineconeHook` now requires `pinecone_conn_id` instead of `api_key` and `environment`. Old connections using raw keys will break. ↓

fix Create an Airflow connection with type 'Pinecone' containing the API key and environment, then pass the connection ID.

gotcha The operator `PineconeIngestOperator` expects input as a list of dictionaries with keys 'id' and 'values', not raw tuples or pandas DataFrames. Converting non-standard formats is the user's responsibility. ↓

fix Ensure your data is a list of dicts: [{'id': 'vec1', 'values': [0.1, 0.2, ...]}, ...].

deprecated The method `describe_index_stats` on `PineconeHook` was deprecated in 2.4.3 in favor of `describe_index` which returns full metadata including dimension and metric. ↓

fix Replace calls to `hook.describe_index_stats(index_name)` with `hook.describe_index(index_name)`.

Imports

PineconeHook

wrong

from airflow.providers.pinecone.hooks import PineconeHook

correct

from airflow.providers.pinecone.hooks.pinecone import PineconeHook

Incorrect parent module; PineconeHook is in hooks.pinecone, not directly in hooks

PineconeIngestOperator

wrong

from airflow.providers.pinecone.operators import PineconeIngestOperator

correct

from airflow.providers.pinecone.operators.pinecone import PineconeIngestOperator

Operator is in the pinecone submodule, not directly in operators

Quickstart

Initialize a PineconeHook and create an index. Ensure a Pinecone connection is defined in Airflow.

from airflow.providers.pinecone.hooks.pinecone import PineconeHook
hook = PineconeHook(pinecone_conn_id='pinecone_default')
hook.create_index(name='test-index', dimension=512, metric='cosine')