pytest-parallel

0.1.1 · abandoned · verified Sun Apr 12

pytest-parallel is a pytest plugin that enables parallel and concurrent testing by utilizing both multiprocessing (workers) and multithreading (tests per worker). It is designed for specific use cases, such as Selenium tests, where tests can be thread-safe, benefit from non-blocking I/O, and manage minimal state in the Python environment. It is not intended as a direct replacement for pytest-xdist, which focuses solely on process-based parallelism for general test distribution. The current version is 0.1.1. The project appears to be unmaintained.

Warnings

breaking The pytest-parallel project (version 0.1.1) is explicitly marked as unmaintained on its GitHub repository. It relies on patching pytest internals, which can lead to instability or breakage with newer versions of pytest as the internal structure changes.
Fix: Consider alternatives like pytest-xdist for general parallel execution, or thoroughly test pytest-parallel's compatibility with your specific pytest version before relying on it in production. Be aware that the `pytest-run-parallel` project (from Quansight-Labs) is a different, more actively maintained project with a similar goal, but distinct features.
gotcha Starting with Python 3.8 on macOS, forking behavior for multiprocessing is forced at the expense of safety, and `spawn` is the default start method. Using `pytest-parallel` (which leverages multiprocessing) on macOS with Python 3.8+ might expose subprocess stability issues due to this change.
Fix: Monitor for unexpected crashes or hangs in parallel test runs on macOS. You may need to investigate alternative multiprocessing start methods or adjust your test environment if issues arise, though direct control over this within `pytest-parallel` is not explicitly documented.
gotcha Parallel and concurrent testing with `pytest-parallel` requires careful test isolation. Tests that modify shared global state, rely on specific test ordering, or use non-thread-safe fixtures (e.g., `capsys`, `monkeypatch` as warned by similar plugins) will likely lead to flaky tests, unpredictable failures, or incorrect results.
Fix: Ensure all tests are atomic, independent, and properly isolated. Avoid shared mutable state between tests. If a test is not thread-safe, it should not be run concurrently. Consider marking such tests to run serially if your test suite design cannot guarantee thread safety.
gotcha While `pytest-parallel` enables concurrent execution, it's not a universal speedup solution. For test suites consisting of many very fast unit tests that don't involve I/O waits, the overhead of managing processes and threads can sometimes make parallel execution slower than sequential execution.
Fix: Profile your test suite's execution time with and without `pytest-parallel`. Use it primarily for I/O-bound tests (like web requests in Selenium) that can genuinely benefit from concurrency and parallelization, rather than CPU-bound unit tests.

Install

pip install pytest-parallel Install stable release

Quickstart

Create a test file (e.g., `test_example.py`) and then execute pytest from your terminal, specifying the number of workers (processes) and tests per worker (concurrent threads). This example demonstrates how to set up simple tests for parallel and concurrent execution.

import time
import pytest

# test_example.py

def test_fast_operation_1():
    time.sleep(0.01)
    assert True

def test_fast_operation_2():
    time.sleep(0.01)
    assert True

def test_fast_operation_3():
    time.sleep(0.01)
    assert True

def test_slow_operation_1():
    time.sleep(0.1)
    assert True

def test_slow_operation_2():
    time.sleep(0.1)
    assert True

# Run with: pytest --workers 2 --tests-per-worker 2
# This will execute tests using 2 processes, with each process running up to 2 tests concurrently.

view raw JSON →