{"id":8453,"library":"pulumi-databricks","title":"Pulumi Databricks Provider","description":"The `pulumi-databricks` Python package is an Infrastructure as Code (IaC) tool that enables developers to define, deploy, and manage Databricks cloud resources programmatically. It wraps the Databricks Terraform provider, providing Pythonic access to resources such as notebooks, clusters, jobs, and Unity Catalog entities. The library is actively maintained with frequent updates, typically released multiple times per month, reflecting the rapid development of its upstream Terraform provider. The current version is 1.90.0.","status":"active","version":"1.90.0","language":"en","source_language":"en","source_url":"https://github.com/pulumi/pulumi-databricks","tags":["pulumi","databricks","iac","cloud","infrastructure-as-code","cloud-resources"],"install":[{"cmd":"pip install pulumi-databricks","lang":"bash","label":"Install Pulumi Databricks"}],"dependencies":[{"reason":"Core Pulumi SDK for IaC operations.","package":"pulumi"},{"reason":"This Python provider is a wrapper around the official Databricks Terraform Provider.","package":"terraform-provider-databricks","optional":true}],"imports":[{"note":"While technically possible, the idiomatic Pulumi Python usage is to import the package as 'databricks' and refer to resources and types via `databricks.<ResourceName>` or `databricks.get_...` functions.","wrong":"from pulumi_databricks.provider import Provider","symbol":"Provider","correct":"import pulumi_databricks as databricks"},{"note":"It is also common to use `import pulumi_databricks as databricks` and then `databricks.Notebook`.","symbol":"Notebook","correct":"from pulumi_databricks import Notebook"},{"note":"It is also common to use `import pulumi_databricks as databricks` and then `databricks.Cluster`.","symbol":"Cluster","correct":"from pulumi_databricks import Cluster"}],"quickstart":{"code":"import pulumi\nimport pulumi_databricks as databricks\nimport os\n\n# Configure Databricks authentication using environment variables\n# DATABRICKS_HOST and DATABRICKS_TOKEN are typically set.\n# Example: export DATABRICKS_HOST=\"https://adb-YOUR_WORKSPACE_ID.1.azuredatabricks.net\"\n#          export DATABRICKS_TOKEN=\"dapiXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX\"\n\n# Retrieve current user information (useful for creating resources in user's home directory)\ncurrent_user = databricks.get_current_user()\n\n# Create a Databricks Notebook\nhello_notebook = databricks.Notebook(\n    \"my-first-notebook\",\n    path=current_user.then(lambda user: f\"{user.home}/pulumi_hello_world_notebook\"),\n    language=\"PYTHON\",\n    content_base64=\"# Databricks notebook created by Pulumi\\nprint('Hello from Pulumi!')\\n\",\n    # Note: content_base64 is discouraged for large notebooks; consider 'source' argument instead.\n    # For real applications, you would typically load content from a file using base64.b64encode(open('path/to/notebook.py', 'rb').read()).decode('utf-8')\n)\n\npulumi.export(\"notebook_url\", hello_notebook.url)\n","lang":"python","description":"This quickstart program demonstrates how to configure the Pulumi Databricks provider and create a simple Databricks Notebook. It assumes Databricks authentication is handled via environment variables `DATABRICKS_HOST` and `DATABRICKS_TOKEN`. The example retrieves the current user's home directory to place the notebook, showcasing a common pattern for dynamic resource naming."},"warnings":[{"fix":"Consult the `terraform-provider-databricks` release notes for detailed breaking changes and adjust your Pulumi program accordingly. Test upgrades in a staging environment.","message":"Major version upgrades of the underlying `terraform-provider-databricks` (often monthly) can introduce breaking changes in resource properties, behavior, or required arguments, even if `pulumi-databricks` itself remains at v1.x. Always review the `terraform-provider-databricks` changelog when upgrading `pulumi-databricks`.","severity":"breaking","affected_versions":"All v1.x when underlying Terraform provider is upgraded."},{"fix":"Prefer the `source` argument for `databricks.Notebook` to point to a local notebook file. Example: `source=\"./notebooks/my_notebook.py\"`.","message":"Using `content_base64` for `databricks.Notebook` is discouraged for large notebooks as it increases the Pulumi state file size and memory footprint. Consider using the `source` argument instead to specify a local file path for the notebook content.","severity":"gotcha","affected_versions":"All v1.x"},{"fix":"Use `pulumi config set databricks:token YYYYYYYYYYYYYY --secret` when setting the token via Pulumi configuration. Alternatively, use `DATABRICKS_TOKEN` environment variable which is the recommended approach for CI/CD.","message":"When configuring authentication, ensure `databricks:token` is marked as a secret if using `pulumi config set` to prevent it from being stored in plaintext in the Pulumi state file.","severity":"gotcha","affected_versions":"All v1.x"},{"fix":"Only set the `databricks:accountId` configuration (or `DATABRICKS_ACCOUNT_ID` environment variable) when interacting with account-level Databricks resources (e.g., creating workspaces). Omit it for workspace-level resources.","message":"The `accountId` provider argument should *not* be set for workspace-level providers. Setting it incorrectly can lead to 'invalid Databricks Account configuration errors' when performing workspace-specific operations.","severity":"gotcha","affected_versions":"All v1.x"},{"fix":"Use recommended authentication methods such as Personal Access Tokens (`pat`), Azure Service Principal (`azure-client-secret`), or other modern OIDC-based methods.","message":"The `basic` authentication type for the provider, specified via `databricks:authType`, is deprecated.","severity":"deprecated","affected_versions":"All v1.x"}],"env_vars":null,"last_verified":"2026-04-16T00:00:00.000Z","next_check":"2026-07-15T00:00:00.000Z","problems":[{"fix":"Upgrade `pulumi-databricks` to v0.5.5 or later if possible. If not, manually remove the cluster from the Pulumi state using `pulumi state rm urn:pulumi:<stack>::<project>::databricks:index/cluster:Cluster::<resource_name>` and then import the current state of your desired cluster, or create a new cluster resource.","cause":"This error occurs when trying to manage a Databricks cluster that has been terminated or unpinned and its metadata is no longer available in the Databricks API, typically after 30 days. The Pulumi state still references this old cluster.","error":"Error: Cannot access cluster ####-######-####### that was terminated or unpinned more than 30 days ago"},{"fix":"Set the `DATABRICKS_HOST` environment variable (e.g., `export DATABRICKS_HOST=\"https://adb-YOUR_WORKSPACE_ID.1.azuredatabricks.net\"`) or configure it via Pulumi config (`pulumi config set databricks:host \"https://...\"`).","cause":"The Databricks provider needs to know the host URL of your Databricks workspace to authenticate and interact with it. This configuration is missing.","error":"Error: configuration for 'databricks:host' is required"},{"fix":"Set the `DATABRICKS_TOKEN` environment variable (e.g., `export DATABRICKS_TOKEN=\"dapiXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX\"`) or configure it securely via Pulumi config (`pulumi config set databricks:token YYYYYYYYYYYYYY --secret`).","cause":"The Databricks provider requires an authentication token (Personal Access Token or similar) to authorize requests to your Databricks workspace.","error":"Error: configuration for 'databricks:token' is required"},{"fix":"Ensure you are using only one set of authentication credentials. For example, if using `~/.databrickscfg`, only set `DATABRICKS_CONFIG_FILE` or `databricks:configFile` and `databricks:profile`. If using host/token, only set `DATABRICKS_HOST`/`DATABRICKS_TOKEN` or `databricks:host`/`databricks:token`.","cause":"This typically happens when multiple authentication-related environment variables or Pulumi configuration settings are present and conflict (e.g., `DATABRICKS_HOST` and `DATABRICKS_CONFIG_FILE` are both set).","error":"Error: more than one authorization method configured"}]}