ClearML

Warnings

• breaking ClearML v2.1.0 and later removed support for Python versions lower than 3.6. Additionally, work-in-progress to drop Python 2 support was mentioned in v2.1.4, consolidating to Python 3.6+ as the minimum requirement.
  Fix: Ensure your Python environment is running Python 3.6 or a newer version.
• breaking For users self-hosting ClearML Server, upgrading to ClearML Server v2.0.0 or later requires a MongoDB major version upgrade from v5.x to v6.x. If your server is older than v1.17, an intermediate upgrade to v1.17 is necessary before proceeding to v2.0.0. This affects the backend infrastructure rather than the client library directly, but is crucial for compatibility.
  Fix: Follow the ClearML Server upgrade guide, ensuring MongoDB is upgraded in stages if coming from older server versions. Refer to the ClearML Server documentation for detailed steps.
• gotcha It is highly recommended to call `Task.init()` at the very beginning of your script's execution, ideally right after `if __name__ == "__main__:".` Delaying `Task.init()` can lead to missed automatic logging (e.g., from PyTorch, TensorFlow, Matplotlib, Tensorboard), memory leaks, or hanging child processes.
  Fix: Place `task = Task.init(...)` as early as possible in your main script execution flow.
• gotcha The `output_uri` parameter in `Task.init()` defaults to `None` or `False`. If not explicitly set to a storage location (e.g., `'s3://your-bucket/clearml-models'`), models registered by the task will NOT be automatically uploaded to ClearML's backend storage, although their metadata will be logged.
  Fix: To ensure models are uploaded, set `output_uri='s3://your-bucket/clearml-models'` (or another supported URI) when calling `Task.init()`.
• gotcha Calling methods like `Task.force_requirements_env_freeze()` or `Task.force_store_standalone_script()` after `Task.init()` can lead to unexpected behavior and might trigger warnings. These methods are intended for pre-initialization configuration.
  Fix: If these functionalities are needed, ensure they are called *before* `Task.init()` or consider alternative approaches if the goal is to modify an already initialized task.

Install

• pip install clearml Install core library

Imports

• Task

  from clearml import Task

• Dataset

  from clearml import Dataset

Quickstart

This quickstart demonstrates how to initialize a ClearML Task to start tracking an experiment. It sets up ClearML credentials via environment variables and then creates a `Task` object. Metrics are logged using the task's logger. It is crucial to call `Task.init()` early in your script to ensure comprehensive tracking. Replace 'YOUR_ACCESS_KEY' and 'YOUR_SECRET_KEY' with your actual ClearML credentials, or run `clearml-init` in your terminal to configure them interactively.

import os
from clearml import Task

# Set ClearML credentials (replace with your actual keys or configure via `clearml-init` CLI)
os.environ['CLEARML_WEB_HOST'] = os.environ.get('CLEARML_WEB_HOST', 'https://app.clear.ml')
os.environ['CLEARML_API_HOST'] = os.environ.get('CLEARML_API_HOST', 'https://api.clear.ml')
os.environ['CLEARML_FILES_HOST'] = os.environ.get('CLEARML_FILES_HOST', 'https://files.clear.ml')
os.environ['CLEARML_API_ACCESS_KEY'] = os.environ.get('CLEARML_API_ACCESS_KEY', 'YOUR_ACCESS_KEY')
os.environ['CLEARML_API_SECRET_KEY'] = os.environ.get('CLEARML_API_SECRET_KEY', 'YOUR_SECRET_KEY')

# Initialize a ClearML Task (experiment)
task = Task.init(project_name='My Project', task_name='My First ClearML Task')

print(f"ClearML Task initialized: {task.name}")

# Log a simple metric
logger = task.get_logger()
for i in range(10):
    logger.report_scalar(series="loss", value=10 - i, iteration=i)

# Simulate some work
import time
time.sleep(2)

# Close the task (optional, often handled automatically on script exit)
task.close()
print("Task completed and closed.")