Azure Synapse Artifacts Client Library

0.22.0 · active · verified Sat Apr 11

The Microsoft Azure Synapse Artifacts Client Library for Python allows interaction with Azure Synapse Analytics workspaces to manage data factory artifacts like pipelines, datasets, linked services, and data flows. As of its current version 0.22.0, it provides a programmatic interface for creating, reading, updating, and deleting these resources. The library follows the Azure SDK guidelines for Python, utilizing `azure-core` and `azure-identity` for consistent interaction and authentication. New versions are released as part of the broader Azure SDK for Python release train, typically on a monthly or bi-monthly cadence, with `0.x.x` versions indicating a preview status.

Common errors

```
ModuleNotFoundError: No module named 'azure.synapse.artifacts'
```
cause The `azure-synapse-artifacts` package is not installed or not available in the Python environment where the code is being executed, especially common within Azure Synapse Notebooks if not explicitly installed or configured for the Spark pool.

fix Install the package using pip: `pip install azure-synapse-artifacts`. For Synapse Notebooks, ensure the package is added to the Spark pool's environment packages via a `requirements.txt` file or wheel file, and restart the Spark session.
```
ClientAuthenticationError: DefaultAzureCredential failed to retrieve a token from the included credentials. Attempted credentials: ...
```
cause The `DefaultAzureCredential` in `azure-identity` attempts several authentication methods, and this error indicates that none of them successfully retrieved an access token. This often happens due to misconfigured environment variables (e.g., `AZURE_CLIENT_ID`, `AZURE_TENANT_ID`, `AZURE_CLIENT_SECRET`), missing managed identity assignments, or issues with Azure CLI/VS Code authentication.

fix Ensure that the necessary environment variables for service principal authentication are set, or that the managed identity is correctly assigned to the resource executing the code (e.g., Synapse workspace, Azure Function) and has the required permissions. For development, verify Azure CLI or Visual Studio Code are logged in with an active session.
```
BadRequest (Synapse artifacts deployment failing)
```
cause This error typically occurs during deployment of Synapse artifacts (e.g., using Azure DevOps pipelines or `az deployment group create`) due to an invalid configuration in the ARM template or parameters file. Common reasons include incorrect schema, invalid workspace names, non-existent storage account references, or unquoted strings in JSON.

fix Double-check the `TemplateParametersForWorkspace.json` and `TemplateForWorkspace.json` files for correct syntax, schema adherence, and valid resource references. Run the deployment with `--debug` locally (for `az CLI`) to get detailed inner errors which pinpoint the exact property causing the failure.
```
Error code: 403 Forbidden. ACL verification failed. Either the resource does not exist or the provided credentials do not have permission to perform the action.
```
cause This error indicates that the managed identity or service principal used by the Synapse pipeline lacks the necessary Access Control List (ACL) or Azure role-based access control (RBAC) permissions to access a linked resource, such as an Azure Data Lake Storage Gen2 account. Specific 'Execute' and 'Write' permissions on folders might be required.

fix Grant the Synapse workspace's managed identity or the configured service principal the appropriate data plane roles (e.g., 'Storage Blob Data Contributor' or 'Storage Blob Data Reader') via IAM on the storage account. Additionally, verify and set specific ACLs (Execute on parent folders, Write on target folder) using Azure Storage Explorer or Azure CLI for fine-grained access.

Warnings

breaking The library is currently in a preview (0.x.x) version. APIs are subject to change without notice before a 1.0.0 stable release, which may introduce breaking changes in future minor versions.
Fix: Be prepared for potential API changes when upgrading. Monitor release notes closely for updates and consider pinning to specific preview versions in production environments until 1.0.0 is released.
gotcha The Synapse workspace endpoint URL must be in the correct format, typically `https://{workspace_name}.dev.azuresynapse.net`. Incorrect URLs (e.g., missing `.dev` or `.net`) will lead to connection errors.
Fix: Always verify the endpoint format from your Synapse workspace overview in the Azure portal or by inspecting the workspace properties. Set the `endpoint` parameter of `SynapseArtifactsClient` accordingly.
gotcha Using `DefaultAzureCredential` for authentication requires proper configuration of Azure environment variables (e.g., `AZURE_CLIENT_ID`, `AZURE_TENANT_ID`, `AZURE_CLIENT_SECRET` for a service principal, or local login via `az login`).
Fix: Refer to the `azure-identity` documentation for setting up `DefaultAzureCredential`. Ensure the identity used has sufficient permissions (e.g., 'Synapse Contributor' or 'Synapse Artifact User') on the Synapse workspace.
gotcha The `azure-identity` package, which provides `DefaultAzureCredential`, is a separate dependency and must be installed. Not installing it will result in a `ModuleNotFoundError` when attempting to import or use it.
Fix: Ensure `azure-identity` is included in your project dependencies. Install it using `pip install azure-identity`.
gotcha The 'azure-identity' package is a required dependency for authentication using DefaultAzureCredential. It must be installed separately.
Fix: Install the 'azure-identity' package using pip: 'pip install azure-identity'.

Install

pip install azure-synapse-artifacts Install stable version

Imports

SynapseArtifactsClient
wrong:
```
from azure.synapse.data_factory import SynapseArtifactsClient
```
correct:
```
from azure.synapse.artifacts import SynapseArtifactsClient
```
The client for artifacts is directly under `azure.synapse.artifacts`, not an older 'data_factory' submodule or a top-level 'synapse' module.

DefaultAzureCredential

from azure.identity import DefaultAzureCredential

Quickstart

This quickstart demonstrates how to initialize the `SynapseArtifactsClient` using `DefaultAzureCredential` and list the pipelines within your Azure Synapse Analytics workspace. Ensure your `AZURE_SYNAPSE_WORKSPACE_URL` environment variable is set to your workspace's development endpoint (e.g., `https://{workspace_name}.dev.azuresynapse.net`) and your Azure credentials are configured for `DefaultAzureCredential` to function, typically via environment variables like `AZURE_CLIENT_ID`, `AZURE_TENANT_ID`, `AZURE_CLIENT_SECRET` for service principals.

import os
from azure.identity import DefaultAzureCredential
from azure.synapse.artifacts import SynapseArtifactsClient

# Set your Synapse workspace endpoint. It should look like: https://{workspace_name}.dev.azuresynapse.net
synapse_workspace_url = os.environ.get('AZURE_SYNAPSE_WORKSPACE_URL', 'https://your-synapse-workspace.dev.azuresynapse.net')

if not synapse_workspace_url:
    raise ValueError("Please set the 'AZURE_SYNAPSE_WORKSPACE_URL' environment variable.")

try:
    # Authenticate with DefaultAzureCredential
    credential = DefaultAzureCredential()

    # Create a SynapseArtifactsClient
    client = SynapseArtifactsClient(endpoint=synapse_workspace_url, credential=credential)

    print(f"Connected to Synapse workspace: {synapse_workspace_url}")

    # List pipelines in the workspace
    print("Listing pipelines:")
    pipelines = client.pipeline.get_pipelines_by_workspace()
    for pipeline in pipelines:
        print(f"  - {pipeline.name}")

except Exception as e:
    print(f"An error occurred: {e}")
    print("Please ensure 'AZURE_SYNAPSE_WORKSPACE_URL' is set and your Azure credentials are configured.")
    print("For DefaultAzureCredential, ensure AZURE_CLIENT_ID, AZURE_TENANT_ID, AZURE_CLIENT_SECRET (or other auth methods) are set.")

view raw JSON →