{"id":10043,"library":"pipestat","title":"Pipeline Status Reporter","description":"pipestat is a Python library that acts as a pipeline results reporter. It provides a flexible way to manage and track the status and outputs of computational pipelines, supporting various backends like YAML files, SQLite databases, and Pephub. The current version is 0.13.1, and it maintains a regular release cadence, with several minor versions and patches released annually.","status":"active","version":"0.13.1","language":"en","source_language":"en","source_url":"https://github.com/pepkit/pipestat","tags":["pipeline","workflow","status","results","reporting","bioinformatics"],"install":[{"cmd":"pip install pipestat","lang":"bash","label":"Install pipestat"}],"dependencies":[{"reason":"Used for schema validation and data modeling; pipestat requires Pydantic v2+.","package":"pydantic","optional":false},{"reason":"Commonly used for project configuration (e.g., sample sheets) when integrating pipestat into larger PEP-based workflows.","package":"peppy","optional":true},{"reason":"Required for using the SQLite or other SQL database backends.","package":"sqlmodel","optional":true}],"imports":[{"symbol":"PipestatManager","correct":"from pipestat import PipestatManager"}],"quickstart":{"code":"import os\nimport tempfile\n\nfrom pipestat import PipestatManager\n\n# Create dummy config and schema files in a temporary directory\ntmpdir = tempfile.TemporaryDirectory()\nconfig_file_path = os.path.join(tmpdir.name, \"pipestat_config.yaml\")\nschema_file_path = os.path.join(tmpdir.name, \"results_schema.yaml\")\ndb_file_path = os.path.join(tmpdir.name, \"pipestat_test.sqlite\")\n\nconfig_content = f\"\"\"\ndatabase:\n  db_file: {db_file_path}\npipeline_name: my_pipeline\nschema_path: {schema_file_path}\n\"\"\"\nwith open(config_file_path, \"w\") as f:\n    f.write(config_content)\n\nschema_content = \"\"\"\nproperties:\n  sample_name:\n    type: string\n  my_result:\n    type: string\n  my_numeric_result:\n    type: number\nrequired:\n  - sample_name\n  - my_result\n\"\"\"\nwith open(schema_file_path, \"w\") as f:\n    f.write(schema_content)\n\n# Initialize PipestatManager\npsm = PipestatManager(\n    config_file=config_file_path,\n    schema_path=schema_file_path\n)\n\n# Report a result\nrecord_identifier = \"sample1\"\nresult_name = \"my_result\"\nresult_value = \"SUCCESS\"\npsm.report(record_identifier=record_identifier, result_name=result_name, value=result_value)\n\n# Report another result\npsm.report(record_identifier=record_identifier, result_name=\"my_numeric_result\", value=123.45)\n\nprint(f\"Reported '{result_name}' for '{record_identifier}' as '{result_value}'\")\n\n# Retrieve results\nretrieved_result = psm.retrieve(record_identifier=record_identifier, result_name=result_name)\nprint(f\"Retrieved '{result_name}' for '{record_identifier}': {retrieved_result}\")\n\n# Clean up\ntmpdir.cleanup()\n","lang":"python","description":"This quickstart demonstrates how to initialize `PipestatManager` with a configuration and schema, report a result for a record, and then retrieve it. It uses temporary files for the configuration and schema to keep it self-contained and runnable."},"warnings":[{"fix":"Review and update your `results_schema.yaml` to conform to the new structure if you define 'samples' in your schema. Refer to the official documentation for updated schema examples.","message":"The schema structure for 'samples' results changed in v0.11.0. If your output schema previously defined 'samples' directly, it now needs to be an array type and nested under 'items'.","severity":"breaking","affected_versions":">=0.11.0"},{"fix":"Ensure your environment has `pydantic>=2` installed. You may need to upgrade Pydantic (`pip install --upgrade pydantic`) or work in a fresh virtual environment.","message":"pipestat requires Pydantic v2+. If you have Pydantic v1 installed in your environment, you may encounter `ValidationError` or import errors due to API changes between Pydantic major versions.","severity":"gotcha","affected_versions":">=0.10.0"},{"fix":"Always double-check the paths provided to `PipestatManager` for `config_file` and `schema_path`. Ensure the `pipeline_name` in the config matches your pipeline, and the schema accurately reflects your expected results.","message":"Pipestat relies heavily on configuration files (e.g., `pipestat_config.yaml`) and result schemas (`results_schema.yaml`). Misconfiguration or incorrect paths can lead to runtime errors or unexpected behavior.","severity":"gotcha","affected_versions":"All"}],"env_vars":null,"last_verified":"2026-04-17T00:00:00.000Z","next_check":"2026-07-16T00:00:00.000Z","problems":[{"fix":"Upgrade Pydantic: `pip install --upgrade pydantic` or ensure `pydantic>=2` is installed.","cause":"Using an older version of Pydantic (v1.x) while pipestat requires Pydantic v2+.","error":"pydantic.v1.ValidationError: ..."},{"fix":"Verify that the paths to your `pipestat_config.yaml` and `results_schema.yaml` are correct and accessible by the script.","cause":"The `config_file` or `schema_path` provided to `PipestatManager` does not point to an existing file.","error":"FileNotFoundError: [Errno 2] No such file or directory: '/path/to/config.yaml'"},{"fix":"Ensure the `db_file` specified in your config is valid. `PipestatManager` should create the necessary tables upon initialization if the schema is provided and the database file is new. If it's an existing file, ensure it's not corrupted or missing tables.","cause":"The SQLite database backend was specified, but the database file either does not exist or has not been properly initialized with the required tables for pipestat.","error":"sqlalchemy.exc.OperationalError: (sqlite3.OperationalError) no such table: record"}]}