OpenLineage Integration Common Library

1.46.0 · active · verified Thu Apr 09

The `openlineage-integration-common` library provides shared data models, utilities, and provider interfaces for building OpenLineage integrations in Python. It is a foundational component often used by other OpenLineage libraries (like `openlineage-python`) and custom integrations. The current version is 1.46.0, and it follows a frequent release cadence, often updated alongside the main OpenLineage project.

Warnings

breaking Version 1.40.0 experienced a breaking change where `__version__` variables were missing in top-level modules, which could affect tools relying on programmatic version checks.
Fix: Upgrade to `openlineage-integration-common` version 1.40.1 or newer to restore `__version__` variables.
gotcha This library (`openlineage-integration-common`) provides common models and utilities primarily for *building* OpenLineage integrations, or as an internal dependency. It is NOT the primary client library for sending OpenLineage events.
Fix: For sending OpenLineage events to a collector, use the `openlineage-python` library and its `OpenLineageClient`. `openlineage-python` internally depends on `openlineage-integration-common`.
gotcha Many core OpenLineage 'Facet' definitions (e.g., `SchemaDatasetFacet`, `RunFacet`) are located in the `openlineage.client.facet` module, which is part of the `openlineage-python` package, not directly in `openlineage.common`.
Fix: When working with OpenLineage Facets, ensure you are importing from `openlineage.client.facet` (after installing `openlineage-python`), rather than searching exclusively within `openlineage.common`.

Install

pip install openlineage-integration-common Install latest version

Imports

DbTableSchema
```
from openlineage.common.models import DbTableSchema
```
A common model for representing database table schemas.
SQLStatement
```
from openlineage.common.provider import SQLStatement
```
Used for parsing and representing SQL statements within integrations.
get_common_config
```
from openlineage.common.config import get_common_config
```
Utility to retrieve common OpenLineage configuration.

Source

from openlineage.common.models import Source

Model for representing data sources.

Quickstart

This quickstart demonstrates how to use fundamental data models like `DbTableSchema`, `Source`, and `SQLStatement` provided by `openlineage-integration-common`. These models are essential building blocks when developing custom OpenLineage integrations or working with parsed metadata.

from openlineage.common.models import DbTableSchema, Source
from openlineage.common.provider import SQLStatement

# Example: Defining a database table schema
db_table = DbTableSchema(
    schema='public',
    table='my_table',
    fields=[
        {'name': 'id', 'type': 'int'},
        {'name': 'name', 'type': 'string'}
    ]
)
print(f"Defined DB Table: {db_table.json(indent=2)}")

# Example: Defining a data source
my_source = Source(scheme='postgresql', authority='localhost:5432', connection_url='jdbc:postgresql://localhost:5432/mydb')
print(f"Defined Source: {my_source.json(indent=2)}")

# Example: Representing a SQL statement (without actual parsing logic)
sql_statement = SQLStatement(query='SELECT * FROM public.my_table')
print(f"SQL Statement: {sql_statement.json(indent=2)}")

view raw JSON →