yq: Command-line YAML/XML/TOML Processor

3.4.3 · active · verified Thu Apr 09

yq is a Python-based command-line processor for YAML, XML, and TOML documents, acting as a lightweight wrapper that transparently converts these formats to JSON and pipes them to the `jq` command-line tool for querying and manipulation. It's currently at version 3.4.3 and maintains an active release cadence with regular updates and fixes.

Warnings

gotcha The `yq` Python library depends on the external `jq` command-line tool. You must have `jq` installed and available in your system's PATH for `yq` to function correctly. This is not a Python dependency.
Fix: Install `jq` via your system's package manager (e.g., `brew install jq` on macOS, `sudo apt-get install jq` on Debian/Ubuntu).
breaking Version 3.4.0 changed the behavior of `yq -y` to induce quoting for string scalars that start with '08' or '09' to prevent them from being interpreted as octal numbers by some YAML parsers. This might change the output format for certain string values.
Fix: Review outputs for YAML documents containing strings starting with '08' or '09'. If consistent non-quoted output is required, consider post-processing or adjusting input data if possible.
gotcha Versions 3.3.0 and 3.3.1 had conflicting behaviors regarding the interpretation of characters that cannot be parsed in octal as integers. Version 3.3.0 attempted to prevent this, but 3.3.1 reverted that change. This could lead to inconsistent data interpretation if documents contain strings resembling octal numbers.
Fix: Be cautious when handling YAML/XML/TOML data that might contain strings like '08' or '09'. Ensure your version of `yq` (and `PyYAML` indirectly) handles these as strings or explicitly quote them in your source data.
deprecated Earlier versions used the `toml` library for TOML processing. Version 3.2.0 switched to `tomlkit` for better round-trip preservation and more robust handling. While this was an internal change, it's good to be aware of the underlying parser for TOML files.
Fix: Ensure you are using `yq` version 3.2.0 or newer for improved TOML parsing capabilities, especially if round-trip fidelity is important.

Install

pip install yq Install yq

Quickstart

The `yq` library primarily provides command-line utilities (`yq`, `xq`, `tomlq`). To use it programmatically in Python, you typically invoke these commands via `subprocess`. This example demonstrates how to create a YAML file, extract a value, and perform a transformation using `yq`.

import subprocess
import os

# Create a sample YAML file
yaml_content = """
name: Jane Doe
age: 28
city: San Francisco
details:
  occupation: Developer
  hobbies: [coding, gaming]
"""
with open("sample.yaml", "w") as f:
    f.write(yaml_content)

print("--- Original YAML ---")
print(yaml_content)

# Process the YAML file using yq to extract the name
try:
    # Example 1: Get the name
    result_name = subprocess.run(
        ["yq", ".name", "sample.yaml"],
        capture_output=True, text=True, check=True
    )
    print(f"\n--- Extracted Name ---
{result_name.stdout.strip()}")

    # Example 2: Update the age and city using a jq-like expression
    result_updated = subprocess.run(
        ["yq", '.age = 29 | .city = "Seattle"', "sample.yaml"],
        capture_output=True, text=True, check=True
    )
    print("\n--- Updated YAML (age 29, city Seattle) ---")
    print(result_updated.stdout.strip())

except FileNotFoundError:
    print("\nError: 'yq' command not found. Please ensure yq (this Python package and the external 'jq' binary) is installed and in your system PATH.")
except subprocess.CalledProcessError as e:
    print(f"\nError processing YAML with yq: {e}")
    print(f"Stderr: {e.stderr.strip()}")
finally:
    # Clean up the sample file
    if os.path.exists("sample.yaml"):
        os.remove("sample.yaml")

view raw JSON →