pytest-randomly

Warnings

• gotcha While pytest-randomly shuffles test order and resets the global `random.seed()` before each test, multiple calls to `random` *within a single test function* will produce the same sequence if run with the same overall test session seed. This is by design for reproducibility of a failing test run. If truly unique random sequences per invocation within a single test are desired, the test itself needs to manage its random state more granularly or explicitly disable `pytest-randomly`'s seed reset for that test using `--randomly-dont-reset-seed`.
  Fix: To get different random numbers within a single test, manage `random.seed()` manually inside that test function, or use the `--randomly-dont-reset-seed` flag if you want less control over per-test random reproducibility. Understand that `pytest-randomly` aims for *test run* reproducibility, not unique random streams per `random` call across test runs.
• gotcha If you use other randomness generators in third-party packages (not explicitly supported by pytest-randomly out-of-the-box, e.g., `factory-boy`, `Faker`, `Model Bakery`, `NumPy`), their random states might not be reset by default, leading to inconsistent test results. You can check the documentation for a list of currently supported libraries.
  Fix: For unsupported randomness generators, you can register an entry point (`pytest_randomly.random_seeder`) referring to a function that takes the new seed and resets the library's random state. This allows `pytest-randomly` to manage its state.
• gotcha When using `pytest-randomly` with other plugins that affect test collection or ordering, such as `pytest-order`, ensure their configurations do not conflict. For example, `pytest-order` can be used to fix the order of specific tests while still allowing `pytest-randomly` to randomize others.
  Fix: Consult both `pytest-randomly` and the other plugin's documentation for compatible usage patterns. `pytest-order` documentation, for instance, has a section on usage with `pytest-randomly`.
• gotcha Using the `--random-order-bucket=global` option (which shuffles all tests completely without regard to module or class boundaries) can significantly increase test run times if you have class-scoped or module-scoped fixtures with expensive setup/teardown. This is because these fixtures might be initialized and torn down multiple times as tests from different scopes interleave.
  Fix: Evaluate if global randomization is strictly necessary. For most cases, the default randomization (at module and class level) is sufficient. If global randomization is required, ensure your fixtures are efficient and can handle multiple setup/teardown cycles gracefully, or consider adjusting fixture scopes.

Install

• pip install pytest-randomly Install stable version

Imports

• pytest_randomly

  This is a pytest plugin and is automatically loaded. Direct imports for its core functionality are not typically needed. Advanced users can register entry points for custom random seeders via 'pytest_randomly.random_seeder'.

  pytest-randomly primarily operates via pytest's plugin discovery and command-line options.

Quickstart

Install `pytest-randomly` and `pytest`. Create a test file, e.g., `test_example.py`, with standard pytest functions. Run pytest from your terminal. The plugin will automatically randomize the test order and print the seed used. You can pass this seed back via `--randomly-seed` to reproduce a specific test run order and random number sequence.

import random

def test_first_random():
    print(f"\nTest 1 random: {random.randint(1, 100)}")
    assert True

def test_second_random():
    print(f"\nTest 2 random: {random.randint(1, 100)}")
    assert True

# To run this, save as test_example.py and execute in your terminal:
# pytest -s test_example.py
# To reproduce a run, use the seed printed in the output, e.g.:
# pytest -s test_example.py --randomly-seed=123456789