{"id":5894,"library":"dagster-azure","title":"Dagster Azure","description":"dagster-azure provides a collection of Azure-specific components for the Dagster data orchestration framework, including resources for Blob Storage, Data Lake Gen2, and compute options. The current version is 0.29.0, which aligns with Dagster core 1.13.0. Dagster and its libraries typically follow a monthly release cadence for minor versions.","status":"active","version":"0.29.0","language":"en","source_language":"en","source_url":"https://github.com/dagster-io/dagster/tree/master/python_modules/libraries/dagster-azure","tags":["dagster","azure","cloud","data lake","blob storage","etl","orchestration"],"install":[{"cmd":"pip install dagster-azure","lang":"bash","label":"Install dagster-azure"}],"dependencies":[{"reason":"Core Dagster framework dependency.","package":"dagster"},{"reason":"Required for Azure Blob Storage and I/O manager functionalities.","package":"azure-storage-blob","optional":false},{"reason":"Required for Azure authentication, including DefaultAzureCredential.","package":"azure-identity","optional":false}],"imports":[{"symbol":"azure_blob_storage_resource","correct":"from dagster_azure.blob.resource import azure_blob_storage_resource"},{"symbol":"blob_storage_io_manager","correct":"from dagster_azure.blob.io_manager import blob_storage_io_manager"},{"symbol":"adls2_file_cache","correct":"from dagster_azure.adls2 import adls2_file_cache"},{"symbol":"AzureDataLakeGen2Resource","correct":"from dagster_azure.adls2.resources import AzureDataLakeGen2Resource"}],"quickstart":{"code":"import os\nfrom dagster import Definitions, asset, JobDefinition\nfrom dagster_azure.blob.io_manager import blob_storage_io_manager\n\n# Set these environment variables or replace with actual values\n# For authentication, 'DefaultAzureCredential' (used by dagster-azure) looks for:\n# AZURE_TENANT_ID, AZURE_CLIENT_ID, AZURE_CLIENT_SECRET or AZURE_FEDERATED_TOKEN_FILE\n# or uses Managed Identity.\nAZURE_STORAGE_ACCOUNT_NAME = os.environ.get(\n    \"AZURE_STORAGE_ACCOUNT_NAME\", \"your_storage_account_name\"\n)\nAZURE_BLOB_CONTAINER_NAME = os.environ.get(\n    \"AZURE_BLOB_CONTAINER_NAME\", \"your-dagster-container\"\n)\n\n@asset\ndef hello_blob_asset():\n    \"\"\"An asset that writes a simple string to Azure Blob Storage.\"\"\"\n    return \"Hello, Dagster Azure Blob Storage!\"\n\n# Create a job that materializes the asset\nhello_blob_job = JobDefinition(name=\"hello_blob_job\", assets=[hello_blob_asset])\n\ndefs = Definitions(\n    assets=[hello_blob_asset],\n    jobs=[hello_blob_job],\n    resources={\n        \"io_manager\": blob_storage_io_manager.configured({\n            \"storage_account_name\": AZURE_STORAGE_ACCOUNT_NAME,\n            \"container\": AZURE_BLOB_CONTAINER_NAME,\n            \"prefix\": \"dagster_output/\" # Optional: objects will be stored under this prefix\n        })\n    }\n)","lang":"python","description":"This example defines an asset `hello_blob_asset` that returns a string. It is configured to use the `blob_storage_io_manager` to persist this string to an Azure Blob Storage container. The `io_manager` automatically handles serialization and deserialization. Ensure you have the `AZURE_STORAGE_ACCOUNT_NAME` and `AZURE_BLOB_CONTAINER_NAME` environment variables set, and that your environment is configured for Azure authentication (e.g., via `AZURE_TENANT_ID`, `AZURE_CLIENT_ID`, `AZURE_CLIENT_SECRET`). To run this example, save it as `repo.py` and run `dagster dev` in the same directory, then launch the `hello_blob_job` from the UI."},"warnings":[{"fix":"Refer to the official Dagster release notes for version compatibility between core Dagster and its libraries. Always upgrade both core Dagster and its libraries together.","message":"The `dagster-azure` library's version (`0.x.x`) is aligned with the core `dagster` version (`1.x.x`). For example, `dagster-azure==0.29.0` is compatible with `dagster==1.13.0`. Always ensure that your `dagster` and `dagster-azure` versions are compatible, typically by installing them together (e.g., `pip install dagster==1.13.0 dagster-azure==0.29.0`).","severity":"gotcha","affected_versions":"All versions"},{"fix":"Ensure the necessary environment variables (`AZURE_TENANT_ID`, `AZURE_CLIENT_ID`, `AZURE_CLIENT_SECRET`) are set, or that your execution environment has a configured Managed Identity or Azure CLI login. Consult Azure identity documentation for `DefaultAzureCredential` setup.","message":"Azure authentication relies on `azure-identity`'s `DefaultAzureCredential`. This credential provider attempts various authentication methods (environment variables, managed identity, Azure CLI, etc.). Incorrectly configured authentication is a common source of errors.","severity":"gotcha","affected_versions":"All versions"},{"fix":"Always review the release notes for `dagster` and `dagster-azure` when upgrading. Pay close attention to changes in resource configuration, I/O manager behavior, and any deprecation warnings in the console or UI.","message":"Major version updates to core `dagster` (e.g., `1.x.x` to `2.x.x` when it eventually happens) or significant changes within `1.x.x` can introduce breaking changes to resource definitions, I/O manager interfaces, or configuration schema that `dagster-azure` components rely on. This might require updates to your resource and I/O manager definitions.","severity":"breaking","affected_versions":"Changes between `dagster` 1.x.x minor versions, and future major versions."}],"env_vars":null,"last_verified":"2026-04-14T00:00:00.000Z","next_check":"2026-07-13T00:00:00.000Z"}