Dagster Spark

Dagster Spark is a Python library that provides integration components for orchestrating Apache Spark jobs within the Dagster data platform. It enables users to define, run, and monitor Spark-based data pipelines with Dagster's declarative programming model, offering capabilities for data management, lineage, and observability. The library is actively maintained and typically releases in sync with the core Dagster library.