pytest-ignore-test-results

0.3.0 · active · verified Thu Apr 16

pytest-ignore-test-results is a pytest plugin that enables selective test result ignoring while maintaining test execution. It allows users to specify test cases (by name or pattern) whose failures should not result in a non-zero exit code from pytest. The current version is 0.3.0, and it is actively maintained by Espressif, with releases occurring as needed for updates and new features.

Common errors

```
Pytest exits with a success code (0) even though some tests are explicitly failing.
```
cause The `pytest-ignore-test-results` plugin successfully ignored the specified test failures, leading pytest to report a successful run by default.

fix If you need a non-zero exit code to indicate that failures *were* present, even if ignored, use the `--strict-exit-code` command-line option: `pytest --ignore-result-cases "your_test_name" --strict-exit-code`
```
Tests are still failing in CI/CD despite configuring 'pytest-ignore-test-results'.
```
cause The tests are indeed failing, and the plugin is likely ignoring their *results*, but your CI/CD system might be looking for a different signal or the specific tests you want to ignore are not correctly matched by the patterns/names provided.

fix Double-check that the patterns passed to `--ignore-result-cases` or `--ignore-result-files` exactly match the full node IDs of the failing tests. Also, ensure the CI/CD pipeline is interpreting the `pytest` exit code correctly, especially if `--strict-exit-code` is not used.
```
My tests are running much slower than expected, even with 'pytest-ignore-test-results' enabled.
```
cause The `pytest-ignore-test-results` plugin *executes* all tests, even those whose results are ignored. It does not prevent tests from running. Native pytest skipping mechanisms prevent execution.

fix If the goal is to prevent slow tests from running entirely, consider using standard `pytest` mechanisms like `@pytest.mark.skip` or `pytest --ignore` for paths/modules that should not be executed. This plugin is for scenarios where tests *must* run, but their failures should not halt the build.

Warnings

gotcha This plugin *runs* tests and then *ignores their results*, which is distinct from `pytest.mark.skip` or `pytest --ignore`. Skipped tests (`@pytest.mark.skip`) are not executed at all, while ignored tests via this plugin will still consume execution time and resources.
Fix: Understand the difference: use `@pytest.mark.skip` or `pytest --ignore` for tests that should not run, and `pytest-ignore-test-results` for tests that must run but whose failures should not break the build.
gotcha The default exit code behavior can be confusing. If tests fail but all failures are ignored by the plugin, `pytest` will typically exit with code 0 (success) unless `--strict-exit-code` is used.
Fix: Use the `--strict-exit-code` option if you want `pytest` to exit with a specific non-zero code (e.g., 6) when all actual failures have been ignored by the plugin. This provides a clear signal in CI/CD that issues were present but suppressed.
gotcha Using `--ignore-no-tests-collected` will suppress the `pytest` error when no tests are found and return exit code 0. While useful in specific scenarios, this can mask issues where tests were expected but not collected.
Fix: Only use `--ignore-no-tests-collected` when you explicitly expect zero tests in a given run, such as in conditional test environments. Otherwise, investigate why tests are not being collected.

Install

pip install -U pytest-ignore-test-results Install latest version

Imports

pytest_ignore_test_results

This plugin is primarily configured via command-line options or pytest hooks, rather than direct Python imports for its core functionality. For example, to customize test case names, you might implement the 'pytest_custom_test_case_name' hook in a conftest.py file.

The plugin auto-registers with pytest upon installation. There are no common user-facing imports for its core ignoring mechanism.

Quickstart

This quickstart demonstrates how to use `pytest-ignore-test-results` via command-line arguments to selectively ignore test failures. It shows how to specify individual test cases or patterns, load ignore lists from files, and control exit codes for CI/CD pipelines. Note that ignored tests still execute.

# First, ensure you have pytest and the plugin installed:
# pip install pytest pytest-ignore-test-results

# Create a test file (e.g., test_example.py)
# ----------------------------------------
# def test_passing():
#     assert True
#
# def test_failing_critical():
#     assert False, "This test MUST NOT be ignored"
#
# def test_failing_but_ignorable():
#     assert False, "This test's failure can be ignored"
# ----------------------------------------

# Run pytest, ignoring the result of 'test_failing_but_ignorable':
# The test will still run, but its failure won't cause pytest to exit with a non-zero code
# unless --strict-exit-code is used and ALL failures are ignored.
# Expected output will show 'xfailed' or 'ignored' for the specified test.

print("Run with: pytest --ignore-result-cases 'test_failing_but_ignorable' test_example.py")
# Example of ignoring tests based on a pattern:
print("Run with: pytest --ignore-result-cases 'test_failing_*' test_example.py")

# Example of loading ignore patterns from a file (e.g., ignore_list.txt):
# Content of ignore_list.txt:
# test_failing_but_ignorable
# test_feature_*
print("Run with: pytest --ignore-result-files ignore_list.txt test_example.py")

# To exit with code 6 if all failed tests are ignored:
print("Run with: pytest --ignore-result-cases 'test_failing_but_ignorable' --strict-exit-code test_example.py")

view raw JSON →