{"id":7614,"library":"pytest-harvest","title":"pytest-harvest","description":"pytest-harvest is a pytest plugin that allows you to store data created during your pytest tests execution and retrieve it at the end of the session. This is particularly useful for applicative benchmarking purposes, collecting custom metrics, or generating reports. The current version is 1.10.5, and it maintains a regular release cadence, often to ensure compatibility with new pytest versions and address bug fixes.","status":"active","version":"1.10.5","language":"en","source_language":"en","source_url":"https://github.com/smarie/python-pytest-harvest","tags":["pytest","testing","benchmarking","plugins","results","xdist","data-collection"],"install":[{"cmd":"pip install pytest-harvest","lang":"bash","label":"Install pytest-harvest"}],"dependencies":[{"reason":"Core dependency for a pytest plugin.","package":"pytest","optional":false},{"reason":"Required if you wish to retrieve test results as pandas DataFrames (e.g., via `module_results_df`, `session_results_df`). It is lazy-loaded.","package":"pandas","optional":true},{"reason":"Enables distributed test execution with `pytest-harvest`'s result consolidation features.","package":"pytest-xdist","optional":true},{"reason":"Mentioned as compatible, especially concerning `lazy_value` parameters. Not strictly required for basic `pytest-harvest` usage.","package":"pytest-cases","optional":true}],"imports":[{"note":"Used to retrieve all collected results from the session at the end of the test run, typically in a `pytest_sessionfinish` hook.","symbol":"get_session_synthesis_dct","correct":"from pytest_harvest import get_session_synthesis_dct"},{"note":"This is a pytest fixture provided by `pytest-harvest` to store and access results within a test module context. It's injected, not explicitly imported.","symbol":"module_results_dct","correct":"def test_something(module_results_dct):"},{"note":"This is a pytest fixture provided by `pytest-harvest` to store and access results as a pandas DataFrame within a test session context. Requires `pandas` to be installed. It's injected, not explicitly imported.","symbol":"session_results_df","correct":"def test_something(session_results_df):"},{"note":"A pytest fixture for easily sharing data across tests within a session. It's injected, not explicitly imported.","symbol":"saved_shared_data","correct":"def test_something(saved_shared_data):"}],"quickstart":{"code":"import pytest\nfrom pytest_harvest import get_session_synthesis_dct\n\n# conftest.py\n@pytest.fixture\ndef custom_metric_value():\n    return 'important_data'\n\ndef pytest_sessionfinish(session):\n    print(\"\\n--- Session Synthesis Results ---\")\n    results = get_session_synthesis_dct(session)\n    for test_id, res in results.items():\n        custom = getattr(res, 'custom_metric_value', 'N/A')\n        print(f\"Test: {test_id}, Outcome: {res.outcome}, Custom: {custom}\")\n    print(\"---------------------------------\")\n\n# test_example.py\ndef test_success(request, custom_metric_value):\n    request.node.add_pytest_stats(custom_metric_value=custom_metric_value)\n    assert True\n\ndef test_failure(request, custom_metric_value):\n    request.node.add_pytest_stats(custom_metric_value=custom_metric_value + '_failed')\n    pytest.fail(\"Intentionally failed test\")\n\n# To run these tests and see the output:\n# 1. Save the conftest.py content as 'conftest.py' in your test directory.\n# 2. Save the test_example.py content as 'test_example.py' in the same directory.\n# 3. Run from your terminal: pytest -s","lang":"python","description":"This quickstart demonstrates how to define a custom metric using a fixture, store its value during test execution with `request.node.add_pytest_stats`, and then retrieve all collected data at the end of the session using `get_session_synthesis_dct` within a `pytest_sessionfinish` hook. Run `pytest -s` from your terminal to see the results printed."},"warnings":[{"fix":"Upgrade to pytest-harvest>=1.10.5 to ensure compatibility with pytest 8.1 and later: `pip install --upgrade pytest-harvest`.","message":"pytest-harvest versions prior to 1.10.5 are incompatible with pytest>=8.1 due to changes in pytest internals, leading to `AttributeError` or signature mismatches.","severity":"breaking","affected_versions":"<1.10.5"},{"fix":"Explicitly install pandas if you intend to use DataFrame result fixtures: `pip install pandas`.","message":"Using result fixtures like `module_results_df` or `session_results_df` without `pandas` installed will raise an `ImportError`. `pytest-harvest` uses lazy loading for `pandas`.","severity":"gotcha","affected_versions":"All versions"},{"fix":"Ensure you are using `pytest-harvest>=1.10.4` and the latest compatible `pytest-xdist` version for robust distributed testing result collection.","message":"`pytest-xdist` integration, especially with older `pytest` versions or `python 3.5`, had known issues with result consolidation or `pathlib` object handling.","severity":"gotcha","affected_versions":"<1.10.3 (for xdist hook), <1.10.4 (for python 3.5 xdist bugfix)"},{"fix":"Upgrade to `pytest-harvest>=1.10.1` to correctly parse test IDs with `::` delimiters.","message":"Test IDs containing `::` were not properly handled by `get_session_synthesis_dct` and related fixtures, leading to incomplete or incorrect results.","severity":"gotcha","affected_versions":"<1.10.1"},{"fix":"Upgrade `pytest-harvest` to `>=1.10.0` and `pytest-cases` to `>=2.3.0` to resolve this specific interaction issue.","message":"When using `pytest-cases` with `lazy_value` parameters, `module_results_df` in `pytest < 5.3` might wrongly insert values as integers instead of objects.","severity":"gotcha","affected_versions":"<1.10.0 with pytest-cases and pytest < 5.3"}],"env_vars":null,"last_verified":"2026-04-16T00:00:00.000Z","next_check":"2026-07-15T00:00:00.000Z","problems":[{"fix":"Upgrade `pytest-harvest` to version `1.10.5` or higher: `pip install --upgrade pytest-harvest`.","cause":"Incompatibility between older `pytest-harvest` versions and `pytest>=8.1` due to internal `pathlib` API changes.","error":"AttributeError: 'str' object has no attribute 'iter_parents'"},{"fix":"Install the `pandas` library: `pip install pandas`.","cause":"Attempting to use `module_results_df` or `session_results_df` fixtures when `pandas` is not installed.","error":"ModuleNotFoundError: No module named 'pandas'"},{"fix":"Ensure `pytest-harvest>=1.10.4` is installed. Verify `pytest-xdist` is also up-to-date and compatible with your `pytest` version. Check `pytest_sessionfinish` hook for data retrieval.","cause":"Issues with `pytest-xdist` hook execution or data transfer, especially in older `pytest-harvest` versions or specific Python/xdist combinations.","error":"pytest-xdist: results not consolidated or missing across workers/master node"},{"fix":"When using `request.node.add_pytest_stats(key=value)`, data is accessible as `result_object.key`. For `request.node.add_report_section('call', 'category', value)`, access might involve parsing `result_object.sections`.","cause":"Incorrectly accessing collected data via `request.node.add_pytest_stats` or `request.node.add_report_section` where the key/attribute name does not match retrieval logic in `get_session_synthesis_dct` or result fixtures.","error":"KeyError: 'custom_data_field' or attribute error when accessing collected data"}]}