ASV Runner

0.2.1 · active · verified Wed Apr 15

ASV Runner is the pure Python core of ASV (Airspeed-Velocity), providing essential functionality for running Python benchmarks with minimal dependencies. It focuses on the execution, timing, and memory measurement of benchmark code. The library's current version is 0.2.1, and it typically releases updates a few times a year, often in conjunction with the main `asv` package.

Common errors

```
ModuleNotFoundError: No module named 'asv_runner'
```
cause The 'asv_runner' package is not installed in the Python environment.

fix Install the package using pip: 'pip install asv_runner'.
```
ImportError: cannot import name 'Benchmark' from 'asv_runner'
```
cause The 'Benchmark' class is not available in the 'asv_runner' module.

fix Ensure you are importing the correct class or function from 'asv_runner'. Refer to the official documentation for the correct import statements.
```
TypeError: 'NoneType' object is not callable
```
cause Attempting to call a function or method that is None, possibly due to a failed import or incorrect assignment.

fix Verify that all necessary modules and functions are correctly imported and initialized before use.
```
AttributeError: module 'asv_runner' has no attribute 'run_benchmark'
```
cause The 'asv_runner' module does not have a 'run_benchmark' attribute, possibly due to a version mismatch or incorrect import.

fix Check the module's documentation to confirm the available attributes and ensure you are using the correct version of 'asv_runner'.
```
ValueError: Invalid benchmark configuration
```
cause The benchmark configuration provided is invalid or missing required parameters.

fix Review the benchmark configuration settings and ensure all required parameters are correctly specified as per the documentation.

Warnings

breaking With the release of `asv` version 0.6.0, the project's functionality was split into the main `asv` package and `asv_runner`. `asv_runner` now strictly handles the core benchmark execution. Users or developers who previously relied on internal `asv` mechanisms for benchmark running prior to 0.6.0 may find their code no longer compatible if they attempt to interact directly with these components.
Fix: Migrate any direct internal calls to `asv`'s benchmark execution logic to use the `asv_runner` API where appropriate, or preferably, use the `asv` command-line tool for orchestration. Review the `asv` and `asv_runner` documentation for updated interaction patterns.
deprecated Raising `NotImplementedError` directly from a benchmark function to skip it is deprecated. While it may still work in some contexts for backward compatibility, the recommended approach is to raise `asv_runner.benchmark.mark.SkipNotImplemented` or use the `skip_benchmark_if`/`skip_params_if` decorators.
Fix: Replace `raise NotImplementedError` with `raise SkipNotImplemented()` (after importing it) or apply the `@skip_benchmark_if` or `@skip_params_if` decorators to your benchmark functions or classes.
gotcha ASV Runner (`asv-runner`) is primarily a low-level component designed to be used by the higher-level `asv` benchmarking tool. Most users should interact with the `asv` command-line interface (e.g., `asv run`, `asv quickstart`) or import from the main `asv` package for comprehensive benchmarking, environment management, and result analysis. Attempting to directly import and use `asv_runner` modules for high-level tasks will result in incomplete functionality and a lack of orchestration.
Fix: For full benchmarking workflows, install the `asv` package (`pip install asv`) and use its command-line tools. Only delve into `asv_runner` directly for extending ASV's core capabilities or for very specific, low-level interactions.
gotcha ASV Runner, when invoked by `asv`, automatically computes a 'version number' for each benchmark based on its source code. If the benchmark's code changes, this version number updates, causing `asv` to ignore previously collected results for that benchmark. This can lead to unexpected re-benchmarking if not understood.
Fix: Be aware that modifying benchmark code will invalidate old results for that benchmark. If you need fine-grained control over benchmark versioning (e.g., to group different versions of a benchmark), you can explicitly set the `.version` attribute on your benchmark function or class.

Install

pip install asv-runner Install stable version

Imports

Benchmark
```
from asv_runner.benchmarks._base import Benchmark
```
While direct import is possible, most users interact with `asv` via the CLI. This import is for advanced customization or extending ASV's core benchmark types.
time_range
```
from my_project.benchmarks import time_range
```
Benchmarks are typically defined in user-created files (e.g., `benchmarks.py`) within a project and then discovered and run by the `asv` command-line tool. Direct import of user benchmarks is part of regular Python module loading.
SkipNotImplemented
```
from asv_runner.benchmark.mark import SkipNotImplemented
```
Used to programmatically skip benchmarks within their execution context, preferred over raising `NotImplementedError`.

Quickstart

ASV Runner provides the core logic for executing benchmarks. In a typical setup, users define benchmark functions or classes within Python files (e.g., `benchmarks.py`) which are then discovered and run by the `asv` command-line tool. This example demonstrates a simple benchmark suite that `asv` (using `asv-runner` internally) would execute.

# my_project/benchmarks.py
class TimeSuite:
    def setup(self):
        self.data = list(range(1_000_000))

    def time_iterate_list(self):
        for _ in self.data:
            pass

    def time_sum_list(self):
        sum(self.data)

# To run this, you would typically use the `asv` command-line tool:
# 1. Initialize an ASV project: `asv quickstart` (creates asv.conf.json, etc.)
# 2. Place this file in your benchmark directory (e.g., 'benchmarks/')
# 3. Run benchmarks: `asv run`

view raw JSON →