Meltano

4.2.0 · active · verified Sat Apr 11

Meltano is an open-source command-line interface (CLI) for ELT+ (Extract, Load, Transform) data integration. It provides a flexible and scalable framework for moving, transforming, and testing data, streamlining data engineering workflows. Currently at version 4.2.0, Meltano maintains an active development cycle with frequent releases across major, minor, and patch versions.

Warnings

breaking Meltano v4 introduced significant breaking changes compared to v3, including a new CLI parser (Click), changes to the `meltano.yml` project file structure, and updated plugin definition schemas.
Fix: Refer to the official Meltano v4 migration guide for detailed steps on updating `meltano.yml` and adapting to the new CLI syntax. Review your `meltano.yml` for required changes, especially in plugin configurations.
breaking The command-line syntax, particularly for `meltano config`, changed in Meltano v4 due to the new CLI parser. Old commands might no longer work as expected.
Fix: Consult the updated documentation for the specific commands you use, especially `meltano config`. For example, `meltano config <plugin_name> set <key> <value>` is the new pattern.
gotcha Docker Hub `latest` and `latest-python` tags for `meltano/meltano` image inadvertently reverted to Meltano v3 for a period around the v3.9.2 release before being corrected to v4.
Fix: If you rely on the `latest` Docker tags, ensure you pull the image again to get the most recent v4 version. Consider pinning to specific version tags (e.g., `meltano/meltano:v4.2.0-python`) for production environments to avoid unexpected updates.
gotcha Meltano frequently updates its internal dependency requirements (e.g., `google-cloud-storage`, `uv`, `structlog`). While typically handled by `pip install meltano`, conflicts can arise in complex Python environments or if `meltano` is installed into a shared virtual environment.
Fix: Always install Meltano in a dedicated virtual environment. If dependency conflicts occur, try a fresh installation in a new environment or use `pip check` to identify specific conflicts. Consult release notes for explicit dependency bumps.

Install

pip install meltano Install Meltano

Imports

meltano
```
Meltano is primarily a CLI tool. Direct programmatic import of `meltano` for end-user applications is not a common pattern. Instead, users interact with `meltano` via its command-line interface or by developing custom plugins.
```
Meltano is predominantly used as a command-line interface. While its internal components can be imported for plugin development, it's not designed for general programmatic use as a library within other Python applications.

Quickstart

This quickstart demonstrates how to initialize a new Meltano project, add an extractor (data source) and a loader (data target), configure a setting for the extractor, and then run a basic ELT pipeline. The output will be stored by the chosen loader (e.g., in `output.jsonl` if using `target-jsonl`).

# Initialize a new Meltano project
meltano init my_meltano_project
cd my_meltano_project

# Add a data source (e.g., tap-carbon-intensity)
meltano add extractor tap-carbon-intensity

# Add a data target (e.g., target-jsonl)
meltano add loader target-jsonl

# Configure an extractor setting (optional, replace with actual config)
meltano config tap-carbon-intensity set country 'GB'

# Run the ELT pipeline
meltano elt tap-carbon-intensity target-jsonl

view raw JSON →