{"library":"pypiper","title":"Pypiper","description":"Pypiper is a lightweight Python toolkit designed for building robust, restartable command-line pipelines. It simplifies the process of creating complex data processing workflows by handling logging, error recovery, and status tracking. The current version is 0.15.1, and it maintains an active release cadence with regular updates.","language":"python","status":"active","last_verified":"Fri Apr 17","install":{"commands":["pip install pypiper"],"cli":{"name":"pypiper","version":"sh: 1: pypiper: not found"}},"imports":["from pypiper import PipelineManager","from pypiper import ngs_pipe"],"auth":{"required":false,"env_vars":[]},"quickstart":{"code":"import pypiper\nimport os\n\n# Define pipeline name and output directory\nPIPELINE_NAME = \"my_pypiper_example\"\nOUTDIR = \"pypiper_output\"\nos.makedirs(OUTDIR, exist_ok=True)\n\n# Initialize PipelineManager\npm = pypiper.PipelineManager(name=PIPELINE_NAME, outdir=OUTDIR)\n\nprint(f\"\\n--- Starting Pypiper Pipeline: {PIPELINE_NAME} ---\")\n\n# Stage 1: Create an initial file\ninput_file = os.path.join(OUTDIR, \"raw_data.txt\")\ncmd1 = f\"echo 'Line 1\\nLine 2\\nLine 3' > {input_file}\"\npm.run(cmd1, target=input_file, stage_name=\"create_raw_data\")\n\n# Stage 2: Process the file (e.g., count lines)\noutput_file = os.path.join(OUTDIR, \"processed_data.txt\")\ncmd2 = f\"wc -l {input_file} > {output_file}\"\npm.run(cmd2, target=output_file, stage_name=\"count_lines\")\n\n# Report a result to pipestat (requires pipestat to be configured or just report to log)\npm.report_result(\"lines_counted\", os.path.getsize(output_file))\n\n# Close the pipeline manager (flushes logs, finishes reporting)\npm.close()\n\nprint(f\"--- Pipeline Finished! Check '{OUTDIR}' for results. ---\")\nprint(f\"Content of {output_file}:\")\nwith open(output_file, 'r') as f:\n    print(f.read().strip())\n\n# Clean up (optional for quickstart demonstration)\n# import shutil\n# shutil.rmtree(OUTDIR)\n","lang":"python","description":"This quickstart demonstrates how to initialize a `PipelineManager`, define stages using `pm.run()` with shell commands, specify target files for restartability, and report a simple result. It creates an output directory, generates a file, processes it, and then reports a simple metric.","tag":null,"tag_description":null,"last_tested":null,"results":[]},"compatibility":null}