Snakemake Workflow Management System

9.19.0 · active · verified Thu Apr 16

Snakemake is a Python-based workflow management system designed to create reproducible and scalable data analyses. It enables writing workflows in a Python-like DSL called Snakefile, allowing for automatic parallelization, dependency tracking, and execution on various platforms. The current version is 9.19.0, with frequent patch releases and major feature updates typically every few months.

Common errors

```
No rule to produce output file 'path/to/output.txt'
```
cause Snakemake cannot find a rule that generates the specified target file or an intermediate file required to produce it. This often means a typo in file paths, missing rule, or incorrect dependencies.

fix Check the spelling of file paths and rule names. Ensure all necessary input and output files are defined correctly across rules, and that a rule exists to produce the requested output.
```
MissingInputException: Missing input files for rule X: path/to/input.txt
```
cause A rule is trying to access an input file that does not exist and cannot be generated by any preceding rule in the workflow.

fix Verify that the input file `path/to/input.txt` either exists on disk before the workflow starts, or that there is another rule defined earlier in the Snakefile that produces it as its output.
```
Error: Directory cannot be locked. Please make sure that no other Snakemake process is running in the workflow directory.
```
cause Another Snakemake process, or a previously crashed process, still holds a lock on the `.snakemake` directory in your workflow.

fix If no other Snakemake process is running, manually remove the lock file using `snakemake --unlock` from the command line within your workflow directory.
```
Command 'conda' not found. Ensure that you have Conda or Mamba installed and available in your PATH.
```
cause Snakemake is configured to use Conda/Mamba for environment management (e.g., via `conda: "envs/myenv.yaml"` or `--conda-frontend`), but the `conda` or `mamba` executable is not found in the system's PATH.

fix Install Miniconda or Mambaforge and ensure its `bin` directory is added to your system's PATH environment variable. Alternatively, use a different `--conda-frontend` or disable Conda integration if not needed.

Warnings

deprecated The `--use-conda` CLI flag is deprecated. It has been replaced by `--conda-frontend conda` or `--conda-frontend mamba` for explicit control over the Conda solver.
Fix: Replace `--use-conda` with `--conda-frontend conda` or `--conda-frontend mamba` in your `snakemake` commands. Ensure `mamba` is installed for faster dependency resolution.
gotcha Snakemake profiles now default to `profile.yaml` instead of a `profile` directory containing multiple files. While the directory structure is still supported, the YAML file is preferred for simplicity.
Fix: If using a single profile, consolidate configuration into a `profile.yaml` file within your profile directory. If you explicitly pass a profile directory, Snakemake will still look for `config.yaml`, `cluster.yaml`, etc., but `profile.yaml` will take precedence if present.
gotcha Over-specification of `threads` or `resources` in rules can lead to deadlocks or inefficient scheduling, especially on cluster systems where resources might be requested but not used.
Fix: Only specify `threads` or `resources` if a rule truly benefits from or requires them. For CPU-bound tasks, `threads: 1` is often sufficient unless the tool explicitly uses multiple threads. Use `resources` to specify memory or other custom requirements.

Install

pip install snakemake Install Snakemake

Imports

SnakemakeApi
wrong:
```
import snakemake
```
correct:
```
from snakemake.api import SnakemakeApi
```
For programmatic interaction (e.g., running workflows from Python), use SnakemakeApi. Direct 'import snakemake' primarily loads the CLI entry point but isn't the preferred way for library usage.

Quickstart

Create a `Snakefile` (the workflow definition) and then execute it using the `snakemake` command-line tool. The example generates a simple file processing workflow and shows how the workflow rules are defined. To run the workflow, execute `snakemake -c 1` in your terminal in the same directory as the Snakefile.

import os

# Create a dummy Snakefile for demonstration
snakefile_content = """
rule all:
    input: "results/final.txt"

rule prepare_data:
    output: "data/input.txt"
    shell: "mkdir -p data && echo 'Hello, Snakemake!' > {output}"

rule process_data:
    input: "data/input.txt"
    output: "results/processed.txt"
    shell: "mkdir -p results && cat {input} | tr 'A-Z' 'a-z' > {output}"

rule analyze_results:
    input: "results/processed.txt"
    output: "results/final.txt"
    shell: "echo 'Analysis complete for: ' $(cat {input}) > {output}"
"""

with open("Snakefile", "w") as f:
    f.write(snakefile_content)

print("Snakefile created. Running workflow...")

# To run this, you would typically use the command line:
# snakemake -c 1
# Or for programmatic execution (more complex, but possible):
# from snakemake.api import SnakemakeApi
# snakemake_runner = SnakemakeApi(
#    snakefile='Snakefile',
#    target_files=['results/final.txt'],
#    cores=1
# )
# success = snakemake_runner.execute()
# print(f"Workflow execution {'succeeded' if success else 'failed'}")

# Clean up example files (optional)
# import shutil
# if os.path.exists("data"): shutil.rmtree("data")
# if os.path.exists("results"): shutil.rmtree("results")
# if os.path.exists("Snakefile"): os.remove("Snakefile")

view raw JSON →