{"id":7969,"library":"azureml-pipeline","title":"Azure ML Pipeline","description":"The `azureml-pipeline` library is part of the Azure Machine Learning V1 Python SDK, used to build, optimize, and manage complex machine learning workflows as pipelines. It enables users to define sequences of steps, manage data dependencies, and run these pipelines on various Azure compute targets. The current version is 1.62.0, and it generally follows the release cadence of the broader Azure ML V1 SDK, with updates typically occurring monthly or bi-monthly.","status":"active","version":"1.62.0","language":"en","source_language":"en","source_url":"https://github.com/Azure/azure-sdk-for-python","tags":["azure","machine-learning","pipeline","mlops","v1-sdk"],"install":[{"cmd":"pip install azureml-pipeline azureml-core","lang":"bash","label":"Install `azureml-pipeline` and its core dependency"}],"dependencies":[{"reason":"Provides core Azure ML functionalities like Workspace, ComputeTarget, Datastore, and Experiment, which are essential for defining and running pipelines.","package":"azureml-core","optional":false}],"imports":[{"symbol":"Workspace","correct":"from azureml.core import Workspace"},{"symbol":"Experiment","correct":"from azureml.core import Experiment"},{"symbol":"ComputeTarget","correct":"from azureml.core.compute import ComputeTarget, AmlCompute"},{"symbol":"Pipeline","correct":"from azureml.pipeline.core import Pipeline"},{"symbol":"PythonScriptStep","correct":"from azureml.pipeline.steps import PythonScriptStep"},{"symbol":"PipelineData","correct":"from azureml.pipeline.core import PipelineData"}],"quickstart":{"code":"import os\nfrom azureml.core import Workspace, Experiment\nfrom azureml.core.compute import ComputeTarget, AmlCompute\nfrom azureml.pipeline.core import Pipeline\nfrom azureml.pipeline.steps import PythonScriptStep\n\n# --- 1. Get Azure ML Workspace ---\n# Authenticate via config.json or environment variables\ntry:\n    ws = Workspace.from_config()\n    print(f\"Workspace loaded from config: {ws.name}\")\nexcept Exception:\n    print(\"config.json not found or failed, trying environment variables...\")\n    # Replace with your actual subscription_id, resource_group, workspace_name\n    subscription_id = os.environ.get(\"AZURE_SUBSCRIPTION_ID\", \"<YOUR_SUBSCRIPTION_ID>\")\n    resource_group = os.environ.get(\"AZURE_RESOURCE_GROUP\", \"<YOUR_RESOURCE_GROUP>\")\n    workspace_name = os.environ.get(\"AZURE_WORKSPACE_NAME\", \"<YOUR_WORKSPACE_NAME>\")\n    if \"<YOUR_SUBSCRIPTION_ID>\" in subscription_id: # Check if placeholders are still present\n        raise ValueError(\"Please set AZURE_SUBSCRIPTION_ID, AZURE_RESOURCE_GROUP, AZURE_WORKSPACE_NAME or provide config.json\")\n    ws = Workspace(subscription_id, resource_group, workspace_name)\n    print(f\"Workspace loaded from environment: {ws.name}\")\n\n# --- 2. Define Compute Target ---\ncpu_cluster_name = \"cpu-cluster-qs\" # Name for your compute cluster\ntry:\n    cpu_cluster = ComputeTarget(workspace=ws, name=cpu_cluster_name)\n    print(f\"Found existing compute target: {cpu_cluster_name}\")\nexcept Exception:\n    print(f\"Creating a new compute target: {cpu_cluster_name}\")\n    compute_config = AmlCompute.provisioning_configuration(\n        vm_size=\"STANDARD_DS3_V2\",\n        min_nodes=0,\n        max_nodes=1\n    )\n    cpu_cluster = ComputeTarget.create(ws, cpu_cluster_name, compute_config)\n    cpu_cluster.wait_for_completion(show_output=True)\n\n# --- 3. Create a Python script for a pipeline step ---\nscript_name = \"my_pipeline_step_script.py\"\nwith open(script_name, \"w\") as f:\n    f.write(\"import os; print(f'Hello from Azure ML Pipeline step on {os.uname().nodename}')\")\n\n# --- 4. Define a Pipeline Step ---\nstep = PythonScriptStep(\n    name=\"HelloStep\",\n    script_name=script_name,\n    compute_target=cpu_cluster,\n    source_directory=\".\", # The directory containing the script\n    allow_reuse=True # Allows reuse of previous step runs if inputs/parameters are identical\n)\n\n# --- 5. Create and Submit the Pipeline ---\npipeline = Pipeline(workspace=ws, steps=[step])\nprint(\"Submitting pipeline...\")\npipeline_run = Experiment(ws, 'MyFirstPipelineExperiment').submit(pipeline)\nprint(f\"Pipeline submitted. Run ID: {pipeline_run.id}\")\n# Uncomment the line below to wait for the pipeline run to complete\n# pipeline_run.wait_for_completion(show_output=True)\n","lang":"python","description":"This quickstart demonstrates how to create and submit a simple Azure ML pipeline using `azureml-pipeline`. It involves obtaining a Workspace, defining a compute target, creating a Python script for a step, and then assembling and submitting these into a pipeline. Ensure your Azure ML Workspace details are accessible either via `config.json` or environment variables for authentication."},"warnings":[{"fix":"Decide whether to use the V1 SDK (all `azureml-*` packages) or the V2 SDK (`azure.ai.ml`). Do not mix them in the same codebase. For new projects, V2 is generally recommended.","message":"`azureml-pipeline` is part of the Azure ML V1 SDK. Microsoft's recommended SDK is V2 (`azure.ai.ml`). Mixing V1 and V2 objects or concepts will lead to runtime errors (e.g., `AttributeError`, `TypeError`).","severity":"breaking","affected_versions":"All versions of `azureml-pipeline` (V1 SDK) when attempting to use V2 SDK constructs."},{"fix":"Always install `azureml-pipeline` alongside `azureml-core` (e.g., `pip install azureml-pipeline azureml-core`). Alternatively, `pip install azureml-sdk` installs a compatible set of V1 SDK components.","message":"The `azureml-pipeline` library requires `azureml-core` for fundamental functionalities like `Workspace`, `ComputeTarget`, and `Experiment`. Installing only `azureml-pipeline` will result in `ModuleNotFoundError` for these core classes.","severity":"gotcha","affected_versions":"All versions"},{"fix":"Ensure your Python environment is within the supported range (e.g., `python -m venv .venv` and `source .venv/bin/activate` with Python 3.8 or 3.9).","message":"`azureml-pipeline` has specific Python version requirements. Current versions (1.62.0) support Python `3.8` and `3.9`. Using unsupported Python versions (e.g., Python 3.10+) will lead to installation failures or runtime errors.","severity":"gotcha","affected_versions":"All versions. Refer to PyPI metadata for exact range (`requires_python`)."},{"fix":"Verify that your compute target (e.g., AmlCompute cluster) is created and running in your Azure ML Workspace. Check permissions of the service principal or user identity used for authentication.","message":"Pipeline steps rely on Azure ML compute targets and datastores. Errors can occur if the specified compute target does not exist, is not correctly configured, or lacks necessary permissions.","severity":"gotcha","affected_versions":"All versions"}],"env_vars":null,"last_verified":"2026-04-16T00:00:00.000Z","next_check":"2026-07-15T00:00:00.000Z","problems":[{"fix":"Install the package: `pip install azureml-pipeline`","cause":"The `azureml-pipeline` library is not installed in the current Python environment.","error":"ModuleNotFoundError: No module named 'azureml.pipeline'"},{"fix":"Install the core package: `pip install azureml-core`. It's often best to install both: `pip install azureml-pipeline azureml-core`.","cause":"The core `azureml-core` library, which provides essential components like `Workspace`, is not installed.","error":"ModuleNotFoundError: No module named 'azureml.core'"},{"fix":"If using `azureml-pipeline` (V1 SDK), use `azureml.core.Workspace` and `azureml.pipeline.core.Pipeline` methods. If using V2, import `MLClient` and build pipelines using V2 constructs, avoiding `azureml-pipeline`.","cause":"Attempting to use V2 SDK (`azure.ai.ml`) constructs (like `MLClient`) with V1 SDK (`azureml-pipeline`) pipeline definitions. `MLClient` is from the V2 SDK, while `Pipeline` is from V1.","error":"AttributeError: 'MLClient' object has no attribute 'create_pipeline'"},{"fix":"Ensure `config.json` is in the working directory, or set `AZURE_SUBSCRIPTION_ID`, `AZURE_RESOURCE_GROUP`, `AZURE_WORKSPACE_NAME` environment variables. Alternatively, use interactive login with `ws = Workspace.from_config(auth=InteractiveLoginAuthentication())`.","cause":"The Python environment does not have correct credentials or configuration to connect to your Azure ML Workspace.","error":"azureml.core.authentication.AuthenticationException: Authentication failed."},{"fix":"Verify the name of your compute target in the Azure ML studio portal and ensure it's correctly passed to `ComputeTarget(workspace=ws, name='my-compute')`.","cause":"The specified compute target does not exist or is misspelled in the Azure ML Workspace.","error":"azureml.core.compute.compute.ComputeTargetException: Compute target 'my-compute' not found."}]}