{"id":1709,"library":"singer-python","title":"Singer Python SDK","description":"The `singer-python` library provides essential utilities for implementing Singer protocol taps (data extractors) and targets (data loaders). It simplifies tasks like parsing configuration, managing state, and emitting/consuming standard Singer messages (schema, record, state, activate_version, log, batch). It is currently at version 6.8.0 and is actively maintained, though more opinionated frameworks like Meltano SDK are also available for higher-level abstractions built on the Singer protocol.","status":"active","version":"6.8.0","language":"en","source_language":"en","source_url":"https://github.com/singer-io/singer-python","tags":["ETL","data integration","Singer protocol","tap","target","data pipeline","data engineering"],"install":[{"cmd":"pip install singer-python","lang":"bash","label":"Install latest version"}],"dependencies":[],"imports":[{"symbol":"singer","correct":"import singer"},{"symbol":"get_logger","correct":"singer.get_logger()"},{"symbol":"write_schema","correct":"singer.write_schema(...)"},{"symbol":"write_record","correct":"singer.write_record(...)"},{"symbol":"write_state","correct":"singer.write_state(...)"},{"note":"Used for command-line interface utilities like `singer.cli.main` or `singer.cli.with_cli` decorator for taps and targets.","symbol":"cli","correct":"import singer.cli"}],"quickstart":{"code":"import singer\nimport json\nimport sys\n\n# Get a Singer logger (logs to stderr by default)\nLOGGER = singer.get_logger()\n\n# 1. Define a schema for a stream\nstream_name = \"users\"\nschema = {\n    \"type\": \"object\",\n    \"properties\": {\n        \"id\": {\"type\": \"integer\", \"key_properties\": [\"id\"]},\n        \"name\": {\"type\": \"string\"},\n        \"email\": {\"type\": \"string\", \"format\": \"email\"}\n    }\n}\nkey_properties = [\"id\"]\n\n# 2. Write the schema message to stdout\n# This is how a Tap declares the structure of data it will send\nsinger.write_schema(\n    stream_name=stream_name,\n    schema=schema,\n    key_properties=key_properties\n)\nLOGGER.info(f\"Schema written for stream '{stream_name}'\")\n\n# 3. Write record messages to stdout\n# These are the actual data rows\nrecords = [\n    {\"id\": 1, \"name\": \"Alice\", \"email\": \"alice@example.com\"},\n    {\"id\": 2, \"name\": \"Bob\", \"email\": \"bob@example.com\"}\n]\n\nfor record in records:\n    singer.write_record(\n        stream_name=stream_name,\n        record=record\n    )\n    LOGGER.debug(f\"Record written for '{stream_name}': {record['id']}\")\n\n# 4. Write a state message to stdout\n# This allows a Tap to checkpoint its progress for incremental syncs\nstate = {\"bookmarks\": {stream_name: {\"last_id\": records[-1][\"id\"]}}}\nsinger.write_state(state)\nLOGGER.info(f\"State written: {state}\")\n\n# The actual Singer messages are written to stdout.\n# To see them, run this script and redirect stdout:\n# python your_script.py > output.jsonl","lang":"python","description":"This quickstart demonstrates the core functionality of `singer-python`: emitting schema, record, and state messages to standard output, which is the standard mechanism for Singer taps."},"warnings":[{"fix":"Use `singer.get_logger()` for all informational and debug output. By default, `singer.get_logger()` writes to `stderr`, ensuring `stdout` remains clean for Singer messages. Avoid all other `print()` statements.","message":"Any non-Singer output (e.g., plain text via `print()`) to `stdout` will break downstream Singer targets. Singer targets expect `stdout` to contain only line-delimited JSON messages conforming to the Singer protocol.","severity":"breaking","affected_versions":"All versions"},{"fix":"Thoroughly understand the Singer spec for state management. Ensure state is written atomically and reliably, typically after a batch of records has been confirmed processed, to correctly bookmark progress.","message":"Incorrect or untimely state management can lead to unreliable incremental syncs, data loss, or reprocessing. State messages should accurately reflect processed data and be emitted only *after* all corresponding records for a given checkpoint have been successfully written.","severity":"gotcha","affected_versions":"All versions"},{"fix":"Consult the Singer spec regarding schema evolution. Only introduce additive changes (new fields) to a stream's schema or utilize the `activate_version` message for breaking changes, ensuring the target supports it. Plan schema changes carefully.","message":"`singer-python` provides utilities for writing schemas but does not automatically handle complex schema evolution logic (e.g., column renames, type changes requiring migration). Taps must adhere to the Singer spec for schema changes.","severity":"gotcha","affected_versions":"All versions"},{"fix":"Evaluate your project's needs: `singer-python` for barebones Singer protocol adherence and maximum control; `meltano-sdk` for a more complete, feature-rich development experience for taps and targets.","message":"For new projects requiring a more opinionated framework with robust CLI parsing, test helpers, and higher-level abstractions for Singer, consider using `meltano-sdk` which builds upon the Singer protocol, rather than `singer-python` directly.","severity":"gotcha","affected_versions":"N/A"}],"env_vars":null,"last_verified":"2026-04-09T00:00:00.000Z","next_check":"2026-07-08T00:00:00.000Z"}