ClearML Agent

2.0.7 · active · verified Mon Apr 13

ClearML Agent is a Python library that provides auto-magical DevOps capabilities for Deep Learning. It serves as a worker to execute machine learning tasks remotely, integrating with the ClearML Python package and ClearML Server. The library maintains an active release cadence with frequent updates.

Warnings

breaking Python 3.5 support was removed in `clearml-agent` v1.9.3. Ensure you are using Python 3.6 or newer.
Fix: Upgrade Python environment to 3.6 or later.
breaking As of `clearml-agent` v1.1.0, the agent no longer tries to use the demo server by default. Attempting to run without explicit server configuration will fail.
Fix: Run `clearml-agent init` to configure server credentials or set `CLEARML_NO_DEFAULT_SERVER=0` environment variable for backward compatibility with older demo server behavior, or define `api_server`, `web_server`, `files_server` in `clearml.conf`.
gotcha When running tasks in Docker mode, the agent by default installs all required Python packages. To use a pre-built Docker image without additional installations, you must explicitly disable this behavior.
Fix: Set environment variables `CLEARML_AGENT_SKIP_PIP_VENV_INSTALL=1` or `CLEARML_AGENT_SKIP_PYTHON_ENV_INSTALL=1` for the agent's environment, or clear the 'Python Packages' section of the task in the ClearML UI before enqueuing.
gotcha Git credentials for cloning repositories can be complex, involving different configuration options and environment variables for various Git providers (GitHub, GitLab, Bitbucket, Azure DevOps).
Fix: Configure Git credentials during `clearml-agent init`, or use `agent.git.extra_credentials`, `agent.git_host_match_prefix`, `agent.git_use_azure_pat`, `agent.git_use_ms_entra_token` in `clearml.conf`. Environment variables like `CLEARML_AGENT_GIT_USER` and `CLEARML_AGENT_GIT_PASS` can also be used.
gotcha If running the agent in virtual environment mode (pip, conda, poetry), the agent itself cannot be launched from an *already activated* virtual environment. It needs to create its own isolated environments.
Fix: Run `clearml-agent daemon` from the system Python environment, not within an activated virtual environment. The agent will then create and manage virtual environments for each task it executes.

Install

pip install clearml-agent Install latest version

Quickstart

The ClearML Agent is primarily a CLI tool. This quickstart guides through installation, interactive initialization to connect to a ClearML server and configure Git, and finally starting the agent daemon to listen for and execute tasks. For a full non-interactive setup, consider setting environment variables for credentials and pre-configuring `clearml.conf`.

# 1. Install ClearML Agent
pip install clearml-agent

# 2. Initialize the agent (interactive setup for server connection and git credentials)
# You'll be prompted to paste ClearML credentials and optionally configure Git.
# Ensure CLEARML_API_ACCESS_KEY and CLEARML_API_SECRET_KEY are set as environment variables
# or paste them when prompted.
# os.environ['CLEARML_API_ACCESS_KEY'] = os.environ.get('CLEARML_API_ACCESS_KEY', 'YOUR_ACCESS_KEY')
# os.environ['CLEARML_API_SECRET_KEY'] = os.environ.get('CLEARML_API_SECRET_KEY', 'YOUR_SECRET_KEY')
clearml-agent init

# 3. Start the ClearML Agent daemon (listening for tasks)
# For simple virtual environment mode (default):
# clearml-agent daemon

# For Docker mode (recommended for complex environments):
clearml-agent daemon --docker

view raw JSON →