pytest-xprocess: Pytest Plugin for External Process Management

1.0.2 · active · verified Thu Apr 16

pytest-xprocess is a pytest plugin designed for robust management of external processes across test runs. It ensures that essential external processes, upon which an application depends, are started and properly terminated during pytest execution. The current version is 1.0.2, and the project maintains an active release cadence, with several updates per year, often coinciding with new Python or pytest versions.

Common errors

```
fixture 'xprocess' not found
```
cause `pytest-xprocess` is installed but the `xprocess` fixture is not being discovered or is out of scope. This often happens if the `conftest.py` file defining the fixture is not in a pytest-discoverable location or if pytest's import mechanisms are misconfigured.

fix Ensure `conftest.py` is in your test root directory or a parent directory accessible by pytest. If using a source layout, ensure pytest is configured to find your test modules (e.g., by setting `pythonpath = .` in `pytest.ini` or `pyproject.toml` for `pytest >= 7`).
```
ImportError: cannot import name 'ProcessStarter' from 'xprocess' (or similar ImportErrors)
```
cause This error typically indicates that the `pytest-xprocess` library is either not installed, or there's a problem with the Python environment's `sys.path` and how modules are being located. It could also happen if you are trying to import from `pytest_xprocess` instead of `xprocess`.

fix Verify that `pytest-xprocess` is installed (`pip show pytest-xprocess`). Ensure your Python environment is correctly activated. Double-check your import statement: it should be `from xprocess import ProcessStarter`.
```
TimeoutError: Process 'myserver' did not start within 120 seconds
```
cause The external process managed by `pytest-xprocess` failed to produce its expected 'pattern' output within the default (or configured) timeout period, or the process crashed before emitting the pattern.

fix First, manually run your external process to debug its startup behavior and confirm the expected 'pattern' is indeed printed. If the process takes longer, increase the `timeout` in your `ProcessStarter` subclass (e.g., `timeout = 300`). If the pattern is not reliable, implement a `startup_check` method for a more active health check.

Warnings

breaking `ProcessStarter.pattern` became optional in version 1.0.0. Users must now provide either `pattern` or `startup_check` (or both) to detect process initialization. Previously, `pattern` was implicitly required.
Fix: Ensure your `ProcessStarter` subclass explicitly defines either a `pattern` regex, a `startup_check` method, or both to signal when the process is ready. If neither is provided, `xprocess.ensure` will not know when the process is successfully started.
deprecated Explicit dependency on the `py` package was added in version 0.21.0 to fix issues with `pytest >= 7.2.0` which no longer implicitly depends on `py`. Older versions of `pytest-xprocess` might fail if `py` is not installed alongside `pytest >= 7.2.0`.
Fix: Upgrade `pytest-xprocess` to 0.21.0 or newer. If upgrading is not possible, manually `pip install py` in your environment.
gotcha By default, `pytest-xprocess` will wait for a maximum of 120 seconds for a process to start. If your process takes longer, or if no output is generated for a long time, it might time out prematurely.
Fix: Adjust the `timeout` class variable within your `ProcessStarter` subclass (e.g., `timeout = 300`) to provide more time for your process to initialize. Consider using the `startup_check` method for more robust readiness checks beyond simple pattern matching.
gotcha Prior to version 0.22.2, log file persistency was not optional, meaning log files were always retained. In 0.22.2, this became optional, defaulting to `True`. The behavior prior to 0.21.0 (where logs were overwritten) can be enabled by setting `persist_logs=False` in `XProcess.ensure()`.
Fix: If you require logs to be overwritten or to not persist, explicitly set `persist_logs=False` when calling `xprocess.ensure("my_process_name", Starter, persist_logs=False)`.
gotcha The `terminate_on_interrupt` attribute in `ProcessStarter` (introduced in 0.18.0) defaults to `False`. This means `pytest-xprocess` will NOT attempt to terminate processes upon interruptions (like CTRL+C) unless explicitly enabled.
Fix: To ensure processes are cleaned up during interruptions, set `terminate_on_interrupt = True` in your `ProcessStarter` subclass.

Install

pip install pytest-xprocess Install stable version

Imports

ProcessStarter
wrong:
```
from pytest_xprocess import ProcessStarter
```
correct:
```
from xprocess import ProcessStarter
```
The main class for defining how to start and manage an external process is imported directly from the 'xprocess' package, not 'pytest_xprocess'.
xprocess
wrong:
```
def my_fixture(pytest_xprocess):
```
correct:
```
def my_fixture(xprocess):
```
The primary fixture provided by the plugin is named 'xprocess', not 'pytest_xprocess'.

Quickstart

This quickstart demonstrates how to define a process fixture using `pytest-xprocess` in a `conftest.py` file. It sets up a simple dummy server, waits for a specific pattern in its output to confirm readiness, and then allows tests to run against it. The `xprocess.ensure()` method guarantees the process is started and managed, including automatic cleanup after the test session.

import pytest
import os
import time

# This content would typically be in conftest.py
# For demonstration, we'll create a dummy server script
# and then use it in the quickstart.

dummy_server_script = """
# dummy_server.py
import time
import sys

if __name__ == '__main__':
    print("Dummy server starting...")
    sys.stdout.flush()
    time.sleep(1) # Simulate startup time
    print("Dummy server ready on port 12345")
    sys.stdout.flush()
    try:
        while True:
            time.sleep(0.5)
    except KeyboardInterrupt:
        print("Dummy server shutting down.")
        sys.stdout.flush()
"""

# Ensure the dummy server script exists for the quickstart to run
with open("dummy_server.py", "w") as f:
    f.write(dummy_server_script)

@pytest.fixture(scope="session")
def dummy_server(xprocess):
    class Starter(xprocess.ProcessStarter):
        # Path to the dummy server script
        # Use absolute path to ensure it's found regardless of cwd
        script_path = os.path.abspath("dummy_server.py")
        pattern = "Dummy server ready on port 12345"
        args = ["python", script_path]

        # Optional: increase timeout if the server takes longer to start
        # timeout = 5 # Default is 120 seconds

    # Ensure the process is running; get its logfile and info object
    logfile = xprocess.ensure("dummy_server", Starter)
    
    # You can interact with the server here if needed, e.g., check its status
    # process_info = xprocess.getinfo("dummy_server")
    # assert process_info.isrunning()
    
    print(f"\nDummy server log: {logfile.name}")
    
    # Yield control to the tests that depend on this fixture
    yield logfile

    # xprocess automatically handles termination after tests, but you can
    # also get the info object and terminate manually if needed.
    # xprocess.getinfo("dummy_server").terminate()

def test_server_is_up(dummy_server):
    # The dummy_server fixture ensures the server is running.
    # You can read its log for verification or perform other checks.
    with open(dummy_server.name, "r") as f:
        logs = f.read()
    assert "Dummy server ready on port 12345" in logs
    print("Test: Dummy server is confirmed ready!")

# To run this, save as a .py file (e.g., test_quickstart.py) and run `pytest -s`
# The -s flag is important to see print statements from the fixture and tests.

view raw JSON →