pytest-random-order

1.2.0 · active · verified Fri Apr 10

pytest-random-order is a pytest plugin that randomises the order of tests. This can be useful to detect a test that passes just because it happens to run after an unrelated test that leaves the system in a favourable state. The plugin allows users to control the level of randomness they want to introduce and to disable reordering on subsets of tests. Tests can be rerun in a specific order by passing a seed value reported in a previous test run. Current version is 1.2.0.

Warnings

breaking From v1.0.0 onwards, `pytest-random-order` no longer randomizes tests by default. You must explicitly enable it using `--random-order`, `--random-order-bucket=<bucket_type>`, or `--random-order-seed=<seed>` command-line options, or by setting `addopts` in your pytest configuration file (e.g., `pytest.ini`).
Fix: Add `addopts = --random-order` to your `pytest.ini` or always pass `--random-order` when invoking `pytest`.
gotcha When using `pytest-random-order` with `pytest-xdist` for parallel test execution, the order configured by `pytest-random-order` might not be preserved. To make them work together effectively, use `pytest-xdist` with the `--dist=loadfile` option. This ensures all tests from one file are run in the same worker, allowing `pytest-random-order` to manage ordering within those files.
Fix: Run `pytest` with `--dist=loadfile --random-order`.
gotcha Using the `--random-order-bucket=global` or `--random-order-bucket=package` options can significantly increase test execution time, especially if your tests rely on module- or session-scoped fixtures. Higher levels of randomization can cause these fixtures to be set up and torn down multiple times across a test run, leading to performance degradation.
Fix: Start with smaller bucket types like `class` or `module` and only switch to `package` or `global` if necessary and after ensuring fixtures handle repeated setup/teardown efficiently.
gotcha Coexistence with other pytest plugins that actively modify test order (e.g., `pytest-ordering`, `pytest-randomly`) can lead to unpredictable test execution sequences, as plugins may conflict or partially revert each other's effects.
Fix: Avoid using multiple test-ordering plugins simultaneously unless their compatibility is explicitly documented. Prioritize `pytest-random-order` if its functionality is desired.

Install

pip install pytest-random-order Install with pip

Imports

pytest_random_order
```
No direct Python import is typically used for plugin activation; it's managed by pytest. For specific test control, markers are used.
```
pytest-random-order is a pytest plugin that is automatically discovered and loaded by pytest. Direct Python imports are not used to activate the plugin's core functionality.
pytest.mark.random_order
```
import pytest

@pytest.mark.random_order(disabled=True)
def test_something_not_randomized():
    pass
```
The `random_order` marker is exposed via the `pytest.mark` namespace for controlling test randomization at the module or class level. It is not imported directly from the plugin package.

Quickstart

Install the plugin, then run pytest with the `--random-order` command-line flag. For persistent randomization, configure `addopts` in your `pytest.ini` or `pyproject.toml` file. This example demonstrates basic test functions which will be randomized when the plugin is active.

# test_example.py
def test_alpha():
    assert True

def test_beta():
    assert True

def test_gamma():
    assert True

# Run from your terminal:
# pip install pytest pytest-random-order
# pytest --random-order
#
# To always enable it, add to pytest.ini (or pyproject.toml):
# [pytest]
# addopts = --random-order

view raw JSON →