mir_eval: Music Information Retrieval Evaluation

0.8.2 · active · verified Tue Apr 14

mir_eval is a Python library providing a transparent, standardized, and straightforward way to evaluate Music Information Retrieval (MIR) systems. It includes common metrics for tasks such as beat detection, chord estimation, melody extraction, and onset detection. The library is actively maintained, with the current version being 0.8.2, and sees regular updates to ensure compatibility and introduce minor enhancements.

Warnings

breaking Older versions of `mir_eval` (pre-0.8.0) used deprecated NumPy type aliases like `np.int` and `np.float`. These aliases were deprecated in NumPy 1.20 and removed entirely in NumPy 1.24+ and NumPy 2.0+.
Fix: Upgrade `mir_eval` to version 0.8.0 or newer. If you are stuck on an older `mir_eval` version, you may need to downgrade NumPy to <1.24. Otherwise, update your custom code to use Python built-in types (`int`, `float`) or specific NumPy dtypes (e.g., `np.int64`, `np.float64`) instead of the deprecated aliases.
gotcha NumPy 2.0 introduced significant changes to its numeric promotion rules, which can affect the precision of results when combining scalars of different types (e.g., `float32` with Python `float`). This could lead to unexpected lower precision or integer overflows in some edge cases.
Fix: Review any operations where `mir_eval` outputs are combined with other NumPy arrays or Python scalars, especially if precision is critical. Explicitly cast to desired dtypes (e.g., `np.float64`) or convert to Python scalars (`int()`, `float()`) before operations to ensure consistent behavior.
gotcha The `mir_eval.display` submodule, used for plotting, has had compatibility updates (e.g., in version 0.5, it updated to use Matplotlib 2). Using older `mir_eval` versions with newer Matplotlib or vice versa might lead to plotting errors or unexpected visual outputs.
Fix: Ensure `mir_eval` and `matplotlib` versions are reasonably aligned. For best compatibility, use `mir_eval` 0.5 or newer with `matplotlib` 2.0 or newer. If issues persist, check the `mir_eval` documentation for specific `matplotlib` version requirements if you rely heavily on `mir_eval.display`.

Install

pip install mir-eval Install latest stable version

Imports

mir_eval
```
import mir_eval
```
The primary import for accessing submodules and evaluation functions.
mir_eval.beat
```
import mir_eval.beat
```
Commonly imported submodule for beat tracking evaluation.
mir_eval.onset.evaluate
```
from mir_eval import onset
scores = onset.evaluate(reference_onsets, estimated_onsets)
```
While direct import `from mir_eval.onset import evaluate` works, it's generally recommended to import the submodule `onset` and call `onset.evaluate` for consistency and clarity, especially when multiple metrics from the same submodule might be used.

Quickstart

This quickstart demonstrates how to compute beat tracking evaluation metrics using `mir_eval.beat`. It initializes synthetic reference and estimated beat times and then calculates common metrics like F-measure, alternate period accuracy, and the Cemgil metric.

import numpy as np
import mir_eval.beat

# Reference beat times (ground truth)
reference_beats = np.array([0.5, 1.0, 1.5, 2.0, 2.5, 3.0])

# Estimated beat times from an algorithm
estimated_beats = np.array([0.51, 1.02, 1.48, 2.03, 2.55, 3.01])

# Calculate F-measure for beat tracking
f_measure = mir_eval.beat.f_measure(reference_beats, estimated_beats)

# Calculate other beat metrics
alternate_accuracy = mir_eval.beat.alternate_period_accuracy(reference_beats, estimated_beats)
cml_t = mir_eval.beat.cemgil_metric(reference_beats, estimated_beats)

print(f"Beat F-measure: {f_measure:.4f}")
print(f"Alternate Period Accuracy: {alternate_accuracy:.4f}")
print(f"Cemgil Metric (T): {cml_t:.4f}")

view raw JSON →