{"id":3476,"library":"dvc-task","title":"dvc-task","description":"dvc-task is an extensible Python library for queuing, running, and managing background jobs (processes) from standalone applications. It is built on Celery but leverages a pre-configured filesystem transport, eliminating the need for a full AMQP messaging server. As of April 2026, the current version is 0.40.2, with active development and somewhat frequent, though irregular, releases.","status":"active","version":"0.40.2","language":"en","source_language":"en","source_url":"https://github.com/iterative/dvc-task","tags":["task queue","celery","dvc","background jobs","async"],"install":[{"cmd":"pip install dvc-task","lang":"bash","label":"Install dvc-task"}],"dependencies":[{"reason":"Core task queue framework, dvc-task is built on top of it. Requires version 5.3 or later.","package":"celery"},{"reason":"Messaging library used by Celery, requires version 5.3 or later.","package":"kombu"}],"imports":[{"note":"Main application class for a filesystem-based Celery app.","symbol":"FSApp","correct":"from dvc_task.app import FSApp"},{"note":"Manages background processes within tasks.","symbol":"ProcessManager","correct":"from dvc_task.proc import ProcessManager"}],"quickstart":{"code":"from dvc_task.app import FSApp\nfrom dvc_task.proc import ProcessManager\nimport time\nimport os\n\n# Create a filesystem-based Celery app\n# For a real application, consider a more persistent wdir\napp = FSApp(wdir=\"./dvc-task-data\")\n\n@app.task\ndef run_command_task(command_args, task_name):\n    manager = ProcessManager(wdir=os.path.join(app.wdir, 'processes'))\n    # Run a simple command as a background process\n    # The '.delay()' call queues the task\n    signature = manager.run(command_args, name=task_name).delay()\n    return signature.id\n\n# Example usage:\nif __name__ == \"__main__\":\n    print(\"Starting dvc-task example...\")\n    task_id = run_command_task.delay(command_args=[\"echo\", \"Hello, dvc-task!\"], task_name=\"greet_task\")\n    print(f\"Task 'greet_task' queued with ID: {task_id}\")\n\n    print(\"Starting temporary worker...\")\n    # In a real scenario, the worker would run in a separate process/daemon\n    # For quickstart, we run a temporary worker that exits when the queue is empty\n    worker = app.TemporaryWorker()\n    worker.start() # This blocks until the queue is empty or timeout\n\n    # Clean up (optional, but good for quickstart)\n    app.clean()\n    print(\"dvc-task example finished and cleaned up.\")\n","lang":"python","description":"This quickstart demonstrates how to initialize a filesystem-based Celery application (FSApp), define a task to run a background process using ProcessManager, and then start a temporary worker to process the task. It showcases how to queue a simple 'echo' command."},"warnings":[{"fix":"Upgrade Celery and Kombu to at least version 5.3.0: `pip install 'celery>=5.3.0' 'kombu>=5.3.0'`.","message":"Version 0.3.0 introduced a breaking change by bumping the minimum required versions for Celery and Kombu to 5.3.0. Users on older versions of these dependencies will need to upgrade.","severity":"breaking","affected_versions":"<0.3.0"},{"fix":"Upgrade to dvc-task version 0.40.1 or later to resolve Celery Monitor Shutdown issues.","message":"Users observed 'Celery Monitor Shutdowns' prior to version 0.40.1, indicating potential instability or unexpected behavior in monitoring. Version 0.40.1 includes a fix (dvc-10427) for this issue.","severity":"gotcha","affected_versions":"<0.40.1"},{"fix":"Be aware of potential platform-specific issues on Windows. Refer to Celery's documentation for known Windows limitations and workarounds.","message":"While `dvc-task` is tested on Windows, its upstream dependency Celery is not officially supported on Windows. This means that certain issues specific to Windows might stem from Celery itself and could have limited official support.","severity":"gotcha","affected_versions":"All"},{"fix":"Carefully review the changelog between `0.4.0` and `0.40.1` for unlisted breaking changes or behavioral differences, and thoroughly test your application after upgrading.","message":"The version numbering jumped from `0.4.0` to `0.40.1`. While not explicitly stated as breaking, such a large jump in the minor version often signals significant changes or potential incompatibilities, even if release notes are brief. Users should review the changelog carefully when upgrading across this boundary.","severity":"gotcha","affected_versions":"All versions before 0.40.1 when upgrading to 0.40.1+"}],"env_vars":null,"last_verified":"2026-04-11T00:00:00.000Z","next_check":"2026-07-10T00:00:00.000Z"}