Multiple Object Tracking Metrics (motmetrics)

Common errors

• AttributeError: module 'motmetrics' has no attribute 'create'

  cause The `create` function for instantiating a metrics host is not directly in the top-level `motmetrics` module.
  fix Import it from the `metrics` submodule: `from motmetrics.metrics import create` or access it as `mm.metrics.create()` if you imported `motmetrics as mm`.

• TypeError: 'numpy.ndarray' object is not callable

  cause This typically occurs when attempting to call `iou_matrix` or other distance functions without providing the correct bounding box arguments, or if an argument is mistakenly treated as a function.
  fix Ensure you are passing numpy arrays of shape `(N, 4)` (for N bounding boxes in `xywh` format) as the first two arguments to `iou_matrix`, not attempting to call an array object itself.

• ValueError: operands could not be broadcast together with shapes (X) (Y)

  cause This error commonly arises within `acc.update()` if the number of ground truth IDs, hypothesis IDs, and the dimensions of the distance matrix do not align.
  fix Verify that the number of ground truth IDs (`gt_ids.shape[0]`) matches the number of rows in your distance matrix (`C.shape[0]`), and similarly for hypothesis IDs (`ts_ids.shape[0] == C.shape[1]`) for the `acc.update(gt_ids, ts_ids, C)` call. The distance matrix `C` must precisely match the counts of ground truth and hypothesis objects for the current frame.

Warnings

• deprecated Older versions of motmetrics (prior to 1.4.0) may raise `FutureWarning: In the future `np.bool` will be a zero-dimensional array instead of a scalar instance.` when used with newer NumPy versions.
  Fix: Upgrade motmetrics to version 1.4.0 or higher to resolve this: `pip install --upgrade motmetrics`.
• gotcha The `motmetrics.utils.compare_to_groundtruth` function had a correction in version 1.4.0 regarding the usage of Euclidean distances. This could lead to subtle correctness issues or different results for users explicitly relying on that specific distance metric with older motmetrics versions.
  Fix: Ensure you are using motmetrics 1.4.0 or newer for accurate distance calculations, especially when using `motmetrics.utils.compare_to_groundtruth` with non-IoU metrics.
• gotcha Bounding box formats are crucial. `iou_matrix` and related distance functions primarily expect bounding boxes in `[x, y, width, height]` format. Misinterpreting this (e.g., using `x1, y1, x2, y2`) will lead to incorrect distance calculations and subsequently wrong metric results.
  Fix: Always ensure your bounding box data is in `[x, y, width, height]` format (often referred to as `xywh`) when passed to `motmetrics` distance functions like `iou_matrix`. Adjust your data preprocessing if necessary.
• gotcha When processing data frame-by-frame, it's critical to ensure that object IDs within a single frame are unique for both ground truth and hypotheses. Duplicate IDs within a frame can lead to unexpected behavior or incorrect associations.
  Fix: Before passing ground truth and hypothesis IDs to `acc.update()`, verify that `gt_frame_ids` and `ts_frame_ids` contain unique identifiers for objects present in that specific frame.

Install

• pip install motmetrics Install latest version

Imports

• MOTAccumulator

  from motmetrics import MOTAccumulator

• iou_matrix

  from motmetrics.distances import iou_matrix

• create
  wrong:

  from motmetrics import create

  correct:

  from motmetrics.metrics import create

  The 'create' function for metrics is located within 'motmetrics.metrics'. Importing from the top-level 'motmetrics' will result in an AttributeError.

Quickstart

Initialize a `MOTAccumulator`, then iterate frame by frame. For each frame, compute a distance matrix (e.g., IoU) between ground truth and tracker hypotheses bounding boxes, and update the accumulator. Finally, use `motmetrics.metrics.create()` to compute and display standard MOT metrics.

import motmetrics as mm
import numpy as np

# Dummy data: Ground truth and tracker hypotheses for two frames
# Format: [id, x, y, width, height] for bounding box coordinates
gt_frame1 = np.array([[1, 10, 10, 5, 5], [2, 20, 20, 5, 5]])
ts_frame1 = np.array([[1, 10, 10, 5, 5], [2, 20, 20, 5, 5]])

gt_frame2 = np.array([[1, 11, 11, 5, 5], [2, 21, 21, 5, 5]])
ts_frame2 = np.array([[1, 11, 11, 5, 5], [2, 21, 21, 5, 5]])

# Create an accumulator to store tracking results
acc = mm.MOTAccumulator(auto_id=True)

# Process Frame 1:
# Calculate Intersection over Union (IoU) distances between ground truth and hypotheses
# Bounding boxes are expected as [x, y, width, height]
C1 = mm.distances.iou_matrix(gt_frame1[:, 1:], ts_frame1[:, 1:], max_iou=0.5)
acc.update(gt_frame1[:, 0], ts_frame1[:, 0], C1)

# Process Frame 2:
C2 = mm.distances.iou_matrix(gt_frame2[:, 1:], ts_frame2[:, 1:], max_iou=0.5)
acc.update(gt_frame2[:, 0], ts_frame2[:, 0], C2)

# Compute and display metrics
mh = mm.metrics.create()
summary = mh.compute(acc, metrics=['mota', 'motp', 'num_frames'], name='dummy_tracking')
print(summary)