{"id":7321,"library":"jobflow","title":"jobflow","description":"jobflow is a free, open-source Python library (v0.3.1) for writing and executing computational workflows. It enables defining complex workflows using simple Python functions and executing them locally or on remote resources via managers like `jobflow-remote` or `FireWorks`. Key features include dynamic workflows, easy compositing and nesting of workflows, and the ability to store workflow outputs across various databases (MongoDB, S3, GridFS, etc.) through the `Maggma` package. The library is actively maintained with regular updates.","status":"active","version":"0.3.1","language":"en","source_language":"en","source_url":"https://github.com/materialsproject/jobflow","tags":["workflow-management","computational-science","material-science","task-orchestration","scientific-computing","python-workflow"],"install":[{"cmd":"pip install jobflow","lang":"bash","label":"Basic installation"},{"cmd":"pip install jobflow[fireworks]","lang":"bash","label":"For FireWorks integration"},{"cmd":"pip install jobflow[vis]","lang":"bash","label":"For graph visualization"}],"dependencies":[{"reason":"Used for settings management and data validation.","package":"pydantic","optional":false},{"reason":"Enables integration with various databases (MongoDB, S3, GridFS) for storing job outputs, highly recommended for persistent storage.","package":"maggma","optional":true},{"reason":"Optional workflow manager for executing jobflow workflows on remote clusters.","package":"fireworks","optional":true},{"reason":"Optional package for running jobflow workflows on remote resources.","package":"jobflow-remote","optional":true}],"imports":[{"symbol":"job","correct":"from jobflow import job"},{"symbol":"Flow","correct":"from jobflow import Flow"},{"symbol":"run_locally","correct":"from jobflow.managers.local import run_locally"},{"note":"Configuration settings are usually accessed via `jobflow.SETTINGS` directly after `v0.1.3`.","wrong":"from jobflow.settings import SETTINGS_CLASS_NAME","symbol":"SETTINGS","correct":"from jobflow import SETTINGS"}],"quickstart":{"code":"from jobflow import job, Flow\nfrom jobflow.managers.local import run_locally\n\n@job\ndef add(a, b):\n    return a + b\n\n@job\ndef multiply(a, b):\n    return a * b\n\n# Create Job objects\njob1 = add(1, 2)\njob2 = multiply(job1.output, 3)\njob3 = add(job2.output, 10)\n\n# Create a Flow from the jobs\nflow = Flow([job1, job2, job3], name=\"my_first_flow\")\n\n# Run the Flow locally\n# For persistent storage, configure ~/.jobflow.yaml with a JobStore\n# e.g., JOB_STORE: { _fw_name: \"JSONStore\", path: \"./jobstore\" }\nresponses = run_locally(flow)\n\n# Access results\nfinal_result = responses[job3.uuid].output\nprint(f\"Final result: {final_result}\")","lang":"python","description":"This quickstart defines two simple jobs using the `@job` decorator. These jobs are then chained together using `Flow` objects, where the output of one job serves as the input for the next. The `run_locally` function executes the entire flow in your local environment. For persistent storage and more advanced execution, a `JobStore` (e.g., MongoDB, S3) should be configured, typically via a `~/.jobflow.yaml` file."},"warnings":[{"fix":"Create or update your `~/.jobflow.yaml` file according to the latest documentation, particularly for `JOB_STORE` configuration (e.g., `JOB_STORE: { _fw_name: \"JSONStore\", path: \"./jobstore\" }`).","message":"As of `v0.1.3`, jobflow migrated its settings handling to `Pydantic`. The default `JOB_STORE` and other configurations are now managed through a `~/.jobflow.yaml` file. Existing setups using older configuration methods may break.","severity":"breaking","affected_versions":">=0.1.3"},{"fix":"Only access the actual output value from the `JobResponse` object returned after a flow has been successfully executed (e.g., `responses[job.uuid].output`).","message":"Job outputs accessed via `.output` are `OutputReference` objects, not the direct computed values. These references are used by jobflow to build the workflow graph and automatically resolve dependencies during execution. Attempting to use `.output` as an immediate value before the job has run will result in errors.","severity":"gotcha","affected_versions":"All versions"},{"fix":"Install the visualization dependencies: `pip install jobflow[vis]`.","message":"The `draw_graph()` method for visualizing flows requires the `graphviz` package and the `jobflow[vis]` extra to be installed. Without it, attempting to draw a graph will likely result in an `AttributeError` or `ModuleNotFoundError`.","severity":"gotcha","affected_versions":"All versions"},{"fix":"Ensure you are using Python >=3.10 and avoid Python 3.14.1 if using `jobflow-remote`. Always check the specific Python compatibility for all `jobflow` ecosystem packages.","message":"While `jobflow` itself supports Python >=3.10, its companion package `jobflow-remote` (used for remote execution) dropped support for Python 3.9 as of its `v1.0` release. There were also warnings regarding Python 3.14.1 due to `networkx` issues.","severity":"breaking","affected_versions":"jobflow-remote >=1.0, Python 3.9, Python 3.14.1"},{"fix":"Carefully configure your `JOB_STORE` in `~/.jobflow.yaml`. For production, use robust backends like MongoDB or S3 via `Maggma` rather than the default `JSONStore`.","message":"Misconfiguration of the `JobStore` (the database where job inputs/outputs are stored) can lead to jobs not being persisted, not being found by managers, or data loss. This is especially critical for remote execution or long-running workflows.","severity":"gotcha","affected_versions":"All versions"}],"env_vars":null,"last_verified":"2026-04-16T00:00:00.000Z","next_check":"2026-07-15T00:00:00.000Z","problems":[{"fix":"Install jobflow with the FireWorks extra: `pip install jobflow[fireworks]`.","cause":"Attempting to use the FireWorks manager without installing the optional 'fireworks' dependencies.","error":"ModuleNotFoundError: No module named 'jobflow.managers.fireworks'"},{"fix":"Remember that `job.output` returns an `OutputReference` for dependency tracking. Access the actual result from the `JobResponse` after running the flow, e.g., `responses[my_job.uuid].output`.","cause":"Incorrectly attempting to call or directly access the value of a `job.output` object before the job has been executed and its output resolved.","error":"TypeError: 'OutputReference' object is not callable"},{"fix":"Create or update `~/.jobflow.yaml` in your home directory with a valid `JOB_STORE` configuration, for example:\n```yaml\nJOB_STORE:\n  _fw_name: \"JSONStore\"\n  path: \"./my_job_data\"\n```\nor a `MongoStore` configuration if `Maggma` is installed.","cause":"The `JOB_STORE` setting is not properly defined in the `~/.jobflow.yaml` configuration file, which is required for any persistent or remote workflow execution.","error":"jobflow.settings.JobflowSettingsValidationError: The job store must be configured."},{"fix":"Install the visualization dependencies: `pip install jobflow[vis]`.","cause":"Attempting to call `flow.draw_graph()` without having installed the optional visualization dependencies.","error":"AttributeError: 'Flow' object has no attribute 'draw_graph'"}]}