{"id":4454,"library":"b2luigi","title":"b2luigi - Belle II Luigi Extensions","description":"b2luigi extends the Luigi workflow management system, primarily integrating it with the Belle II software framework (basf2) for batch processing tasks. It provides specialized tasks, runners, and target classes for managing data within the Belle II environment and various remote file systems. The library is actively maintained with frequent releases, typically every few months, addressing bug fixes and adding new functionalities like WebDAV support.","status":"active","version":"1.2.8","language":"en","source_language":"en","source_url":"https://github.com/belle2/b2luigi","tags":["luigi","workflow","batch processing","physics","belle2","data science"],"install":[{"cmd":"pip install b2luigi","lang":"bash","label":"Install b2luigi"}],"dependencies":[{"reason":"b2luigi is built on top of Luigi and extends its core functionalities.","package":"luigi","optional":false},{"reason":"Required for installation and packaging; explicitly added as a dependency since v1.2.6.","package":"setuptools","optional":false}],"imports":[{"symbol":"Task","correct":"from b2luigi import Task"},{"note":"b2luigi provides its own enhanced LocalTarget and other file system targets; prefer these over raw luigi.LocalTarget within b2luigi workflows.","wrong":"from luigi import LocalTarget","symbol":"LocalTarget","correct":"from b2luigi import LocalTarget"},{"note":"Use this for tasks that require integration with the Belle II basf2 framework.","symbol":"Basf2Task","correct":"from b2luigi.basf2 import Basf2Task"},{"symbol":"run","correct":"from b2luigi import run"}],"quickstart":{"code":"import b2luigi\nimport luigi\nimport os\n\nclass GenerateData(b2luigi.Task):\n    filename = luigi.Parameter()\n\n    def output(self):\n        return b2luigi.LocalTarget(f'data/{self.filename}.txt')\n\n    def run(self):\n        os.makedirs(os.path.dirname(self.output().path), exist_ok=True)\n        with self.output().open('w') as f:\n            f.write(f'Generated data for {self.filename}')\n\nclass ProcessData(b2luigi.Task):\n    filename = luigi.Parameter()\n\n    def requires(self):\n        return GenerateData(filename=self.filename)\n\n    def output(self):\n        return b2luigi.LocalTarget(f'processed_data/{self.filename}_processed.txt')\n\n    def run(self):\n        os.makedirs(os.path.dirname(self.output().path), exist_ok=True)\n        with self.input().open('r') as infile,\n             self.output().open('w') as outfile:\n            content = infile.read()\n            outfile.write(f'Processed: {content.upper()}')\n\nif __name__ == '__main__':\n    # Use luigi.build for programmatic execution within a script.\n    # For command-line execution and parsing, b2luigi.run() is typically used.\n    luigi.build([\n        ProcessData(filename='example_file')\n    ], local_scheduler=True)\n\n    print(\"\\n--- Task completed ---\")\n    print(\"Check 'data/example_file.txt' and 'processed_data/example_file_processed.txt'\")\n    # Clean up generated files for repeated execution\n    # os.remove('data/example_file.txt')\n    # os.remove('processed_data/example_file_processed.txt')","lang":"python","description":"This quickstart demonstrates how to define and run a simple workflow using `b2luigi.Task` and `b2luigi.LocalTarget`. It involves two tasks: one to generate a data file and another to process it, showcasing task dependencies and ensuring output directories are created."},"warnings":[{"fix":"Upgrade to b2luigi v1.2.6 or newer, or ensure your Python environment is <3.12 if using an older b2luigi version.","message":"Prior to version 1.2.6, b2luigi had a strict `Python < 3.12` requirement. Attempting to use it with Python 3.12 or newer on older b2luigi versions would lead to installation or runtime errors.","severity":"gotcha","affected_versions":"<1.2.6"},{"fix":"Always use `b2luigi.LocalTarget` and other `b2luigi` provided target classes for consistency and access to extended features. Review documentation for `FileSystemTarget` and `WebDAVTarget` if integrating with remote storage.","message":"Version 1.2.0 introduced new `b2luigi.LocalTarget` and `b2luigi.FileSystemTarget` classes, and v1.2.8 added `WebDAVTarget`. While designed to be compatible, code directly manipulating `luigi.LocalTarget` instances, custom target implementations, or relying on prior internal target structures might require adjustments.","severity":"breaking","affected_versions":">=1.2.0"},{"fix":"Upgrade to b2luigi v1.2.7 or newer to ensure correct functionality of `runner.remove_outputs`.","message":"The `runner.remove_outputs` method had an incorrect keyword argument in versions prior to 1.2.7, leading to unexpected behavior or errors when attempting to remove task outputs programmatically.","severity":"gotcha","affected_versions":"<1.2.7"},{"fix":"Understand that the library's focus is specialized. For generic Luigi workflows without `basf2` integration, pure `luigi` might be a simpler alternative, or `b2luigi` can be used for its enhanced local/remote target handling.","message":"While `b2luigi` builds on `luigi`, its primary purpose is integration with the Belle II `basf2` framework. Many advanced features (e.g., specialized runners for batch systems) are tailored for this environment. Users outside the Belle II collaboration might find some functionalities less relevant or require custom setup.","severity":"gotcha","affected_versions":"All versions"}],"env_vars":null,"last_verified":"2026-04-12T00:00:00.000Z","next_check":"2026-07-11T00:00:00.000Z"}