{"id":5200,"library":"dtlpymetrics","title":"Dataloop Metrics (dtlpymetrics)","description":"Dataloop Metrics (dtlpymetrics) is a Python library that provides scoring and metrics functionality specifically for Dataloop AI projects. It allows users to define, calculate, and report custom metrics on datasets and annotation sets within the Dataloop platform. The current version is 1.2.32. Its release cadence is tied to the Dataloop platform and SDK updates, typically receiving frequent minor updates.","status":"active","version":"1.2.32","language":"en","source_language":"en","source_url":"https://github.com/dataloop-ai/dtlpy-metrics","tags":["dataloop","metrics","scoring","ai","mlops"],"install":[{"cmd":"pip install dtlpymetrics","lang":"bash","label":"Install dtlpymetrics"}],"dependencies":[{"reason":"Required for all Dataloop API interactions (authentication, project/dataset management, entity IDs). Version `~=1.x.x` is typically required based on `dtlpymetrics` setup.","package":"dtlpy","optional":false},{"reason":"Numerical operations within the library.","package":"numpy","optional":false}],"imports":[{"symbol":"DataloopMetrics","correct":"from dtlpymetrics.dtlpm import DataloopMetrics"}],"quickstart":{"code":"import dtlpy as dl\nfrom dtlpymetrics.dtlpm import DataloopMetrics\nimport os\n\n# Ensure Dataloop SDK is logged in or configured\ntry:\n    if not dl.token_expired():\n        print(\"Already logged into Dataloop SDK.\")\n    else:\n        # Replace with your actual Dataloop token or ensure `dl.login()` is called elsewhere\n        dl.login(token=os.environ.get('DATALOOP_API_TOKEN', ''))\nexcept Exception as e:\n    print(f\"Failed to login to Dataloop SDK. Please ensure 'dtlpy' is installed and you are logged in or DATALOOP_API_TOKEN is set. Error: {e}\")\n    exit(1)\n\n# 1. Initialize the metrics client\nmetrics_client = DataloopMetrics()\n\n# 2. Define your target Dataloop Project and Dataset IDs\n#    Replace 'YOUR_PROJECT_ID' and 'YOUR_DATASET_ID' with actual IDs or set environment variables.\nproject_id = os.environ.get('DATALOOP_PROJECT_ID', 'YOUR_PROJECT_ID')\ndataset_id = os.environ.get('DATALOOP_DATASET_ID', 'YOUR_DATASET_ID')\n\n# Check if placeholder IDs are still present to guide the user\nif project_id == 'YOUR_PROJECT_ID' or dataset_id == 'YOUR_DATASET_ID':\n    print(\"Warning: Please replace 'YOUR_PROJECT_ID' and 'YOUR_DATASET_ID' with actual Dataloop IDs or set the DATALOOP_PROJECT_ID/DATALOOP_DATASET_ID environment variables.\")\n    exit(1)\n\n# For demonstration, use a placeholder metric_id and score\nmetric_id = \"example_accuracy_score\"\nscore_value = 0.85\n\n# 3. Report a score\ntry:\n    # Before reporting, ensure the metric_id exists in your Dataloop project/dataset.\n    # You can create metrics using metrics_client.metrics_create() if needed.\n    metrics_client.metrics_report_score(\n        metric_id=metric_id,\n        score=score_value,\n        project_id=project_id,\n        dataset_id=dataset_id\n        # Optional: Link to specific entities with filters or entity_id/entity_type\n        # filters=dl.Filters(resource=dl.FiltersResource.ITEM).add(field='filename', values=['my_image.jpg']),\n        # entity_id='your-item-id',\n        # entity_type='item'\n    )\n    print(f\"Successfully reported score {score_value} for metric '{metric_id}' to Dataloop.\")\nexcept Exception as e:\n    print(f\"Failed to report score: {e}. Make sure the metric_id exists and you have correct permissions for project '{project_id}' and dataset '{dataset_id}'.\")\n","lang":"python","description":"Initializes the DataloopMetrics client, ensures Dataloop SDK login, and reports a sample score to a specific project and dataset. Requires an active Dataloop account, `dtlpy` SDK, and a pre-defined metric on the Dataloop platform."},"warnings":[{"fix":"Always install `dtlpymetrics` alongside a compatible `dtlpy` version, paying attention to the `~=` dependency specified in `dtlpymetrics`'s `setup.py`. Upgrade both libraries in tandem to maintain compatibility.","message":"The `dtlpymetrics` library is tightly coupled with the `dtlpy` SDK. Breaking changes or version mismatches in `dtlpy` (especially major versions or specific `~=` ranges) can lead to unexpected behavior or API errors in `dtlpymetrics`.","severity":"breaking","affected_versions":"All versions"},{"fix":"Use `metrics_client.metrics_create()` to programmatically define new metrics, or ensure they are configured manually through the Dataloop platform UI, before attempting `metrics_report_score()`.","message":"Before reporting scores, the specified `metric_id` must be pre-defined within the Dataloop platform (associated with your project/dataset). Attempting to report to a non-existent metric will result in a `DLServerError`.","severity":"gotcha","affected_versions":"All versions"},{"fix":"Ensure `dtlpy.login()` is successfully called prior to using `DataloopMetrics`. Double-check all Dataloop entity IDs to ensure they are valid and belong to the correct context (e.g., project_id for dataset_id) and that the authenticated user has the necessary permissions.","message":"All operations in `dtlpymetrics` require active authentication with the Dataloop platform via the `dtlpy` SDK. Additionally, providing correct Dataloop entity IDs (project_id, dataset_id, item_id, annotation_id, etc.) is crucial for successful metric operations.","severity":"gotcha","affected_versions":"All versions"}],"env_vars":null,"last_verified":"2026-04-13T00:00:00.000Z","next_check":"2026-07-12T00:00:00.000Z"}