{"id":4460,"library":"brickflows","title":"Brickflows","description":"Brickflows is a Python library and CLI tool designed to simplify the development and deployment of scalable workflows on Databricks. It enables users to define Databricks workflows declaratively using Python, leveraging decorators for tasks and integrating seamlessly with Databricks Asset Bundles (DABs) for deployment. The current version is 1.7.0, and it maintains an active release cadence with frequent updates and bug fixes.","status":"active","version":"1.7.0","language":"en","source_language":"en","source_url":"https://github.com/Nike-Inc/brickflow","tags":["Databricks","Workflow","MLOps","CI/CD","Orchestration"],"install":[{"cmd":"pip install brickflows","lang":"bash","label":"Install Brickflows"},{"cmd":"curl -fsSL https://raw.githubusercontent.com/databricks/setup-cli/main/install.sh | sudo sh\ndatabricks configure --token","lang":"bash","label":"Install and Configure Databricks CLI (Prerequisite)"}],"dependencies":[{"reason":"Requires Python versions 3.9 to 3.12 for compatibility and execution.","package":"python","optional":false},{"reason":"The Brickflows CLI relies on the Databricks CLI for configuration and interaction with Databricks workspaces.","package":"databricks-cli","optional":false}],"imports":[{"symbol":"Workflow","correct":"from brickflow import Workflow"},{"symbol":"TaskSettings","correct":"from brickflow import TaskSettings"},{"symbol":"Cluster","correct":"from brickflow import Cluster"},{"symbol":"Project","correct":"from brickflow import Project"},{"note":"Used for accessing Databricks utilities and secrets within tasks.","symbol":"ctx","correct":"from brickflow.context import ctx"}],"quickstart":{"code":"from datetime import timedelta\nfrom brickflow import Workflow, Cluster, TaskSettings\nimport os\n\n# Configure Databricks host and token via environment variables or databricks configure --token\n# For this example, ensure your ~/.databrickscfg is set up or environment variables are available.\n# Example: export DATABRICKS_HOST=\"https://<your-workspace-url>.cloud.databricks.com\" \n#          export DATABRICKS_TOKEN=\"dapi...\"\n\nwf = Workflow(\n    \"hello_world_workflow\",\n    default_cluster=Cluster(\n        name=\"brickflow-example-cluster\",\n        spark_version='12.2.x-scala2.12',\n        node_type_id='Standard_DS3_v2',\n        num_workers=1\n    ),\n    default_task_settings=TaskSettings(\n        timeout_seconds=timedelta(hours=2).seconds\n    )\n)\n\n@wf.task()\ndef hello_task():\n    print(f\"Hello from Databricks! Host: {os.environ.get('DATABRICKS_HOST', 'N/A')}\")\n    return \"Task completed successfully\"\n\n# To run this, you would typically use the brickflows CLI:\n# 1. Create a project: `mkdir my_brickflow_project && cd my_brickflow_project && brickflow projects add`\n# 2. Place this code in `workflows/hello_world_wf.py` (assuming workflows is your workflows directory).\n# 3. Deploy: `brickflow deploy --deploy-mode=bundle -p <your_databricks_profile>`","lang":"python","description":"This quickstart demonstrates how to define a simple 'Hello World' Databricks workflow using Brickflows. It includes defining a workflow with a default cluster and a single task. To deploy, you must first set up a Brickflow project using the `brickflow` CLI and ensure the Databricks CLI is configured to authenticate with your workspace."},"warnings":[{"fix":"Upgrade to v1.4.1 or later. The regression introduced by #237 was fixed in #245 (v1.4.1), which allows workflows to deploy without explicit health rules.","message":"Deployment failure if 'health' rules were not set (regression in some v1.4.x releases).","severity":"breaking","affected_versions":"v1.4.0 (and potentially immediate patches leading up to v1.4.1)"},{"fix":"Upgrade to v1.3.2 or later, which includes a fix for this behavior. Always thoroughly test dependency sensors.","message":"The `WorkflowDependencySensor` might not fail when an invalid `dependency_job_id` is provided, leading to silent failures or unexpected behavior.","severity":"gotcha","affected_versions":"Up to v1.3.1"},{"fix":"Upgrade to v1.3.3 or later to ensure default task settings are applied consistently across all task types.","message":"Default task settings like `timeout_seconds` might not apply correctly to 'If/Else' or 'For Each' task types in older versions.","severity":"gotcha","affected_versions":"Up to v1.3.2"},{"fix":"To run specific tasks for debugging or testing, use the Databricks UI's 'Run now with different parameters' option and set `brickflow_internal_only_run_tasks` with a comma-separated list of task names. Avoid modifying other parameters.","message":"Running a single task directly via programmatic methods in Databricks workflows is not straightforward due to architectural differences from tools like Airflow. Brickflows provides a specific UI-based mechanism.","severity":"gotcha","affected_versions":"All versions"},{"fix":"Always initialize your project using `brickflow projects add` or `bf init` and follow the recommended project structure. Ensure `path_from_repo_root_to_project_root` and `path_project_root_to_workflows_dir` in `.brickflow-project-root.yml` are correctly set.","message":"Project setup is crucial for correct module resolution and deployment. Incorrectly configured project roots or workflow directories can lead to import errors or deployment issues.","severity":"gotcha","affected_versions":"All versions"}],"env_vars":null,"last_verified":"2026-04-12T00:00:00.000Z","next_check":"2026-07-11T00:00:00.000Z"}