{"id":2721,"library":"pytest-codspeed","title":"pytest-codspeed","description":"pytest-codspeed is a pytest plugin that integrates performance benchmarking into your Python test suite. It allows developers to measure and track the performance of their code, providing support for CPU simulation, wall-time, and memory usage measurements. The library is actively maintained with frequent releases, currently at version 4.3.0, and aims to provide highly consistent benchmark results, especially when run in CI environments.","status":"active","version":"4.3.0","language":"en","source_language":"en","source_url":"https://github.com/CodSpeedHQ/pytest-codspeed","tags":["pytest","benchmarking","performance","CI","profiling","python-performance"],"install":[{"cmd":"pip install pytest-codspeed","lang":"bash","label":"Install with pip"}],"dependencies":[{"reason":"As a pytest plugin, it requires pytest to function.","package":"pytest","optional":false}],"imports":[{"note":"Used as a decorator to mark an entire test function for benchmarking.","symbol":"pytest.mark.benchmark","correct":"import pytest\n\n@pytest.mark.benchmark\ndef test_my_function(): ..."},{"note":"The `benchmark` fixture is automatically provided by pytest-codspeed for fine-grained control over which parts of a test function are measured.","symbol":"benchmark fixture","correct":"def test_my_function(benchmark):\n    result = benchmark(lambda: my_code_to_measure())\n    assert result is not None"}],"quickstart":{"code":"import pytest\n\ndef sum_squares(arr):\n    \"\"\"Sum the squares of the numbers in an array.\"\"\"\n    total = 0\n    for x in arr:\n        total += x * x\n    return total\n\n@pytest.mark.benchmark\ndef test_sum_squares_full_function():\n    \"\"\"Benchmark an entire test function using the decorator.\"\"\"\n    # This entire function's execution will be measured\n    assert sum_squares(range(1000)) == 332833500\n\ndef mean(data):\n    return sum(data) / len(data)\n\ndef test_mean_performance_fine_grained(benchmark):\n    \"\"\"Benchmark a specific part of a test function using the fixture.\"\"\"\n    data = list(range(1_000_000)) # Setup not measured\n    \n    # Only the lambda function's execution will be measured\n    result = benchmark(lambda: mean(data))\n    assert result == 499999.5\n\n# To run locally: pytest your_test_file.py --codspeed","lang":"python","description":"This example demonstrates two common ways to create benchmarks with pytest-codspeed: using the `@pytest.mark.benchmark` decorator to measure an entire test function, and using the `benchmark` fixture for precise control over the code block to be measured within a test. Benchmarks can be run locally using `pytest --codspeed`."},"warnings":[{"fix":"Review existing benchmarks after upgrading and adjust any baselines if slight performance shifts are observed, particularly for very small, fast functions.","message":"Upgrading to v4.0.0 introduced `CodSpeedHQ/instrument-hooks`. This may cause slight performance changes in tiny microbenchmarks compared to previous versions due to changes in how instrument states are controlled.","severity":"breaking","affected_versions":">=4.0.0"},{"fix":"Integrate CodSpeedHQ/action into your CI/CD workflow to enable performance reporting and historical tracking. Local runs are primarily for functional verification of benchmarks.","message":"Running benchmarks locally with the `--codspeed` flag will display results in the console but will *not* produce performance reports or integrate with the CodSpeed dashboard. For full performance reporting, regression tracking, and consistent measurements, benchmarks must be run in a CI/CD environment using the official CodSpeed runner (e.g., GitHub Action).","severity":"gotcha","affected_versions":"All"},{"fix":"Configure your CI environment to run benchmarks in a single process when using `walltime` mode, or use CodSpeed's dedicated macro runners if available.","message":"When using the `walltime` instrument (enabled via `--codspeed-mode walltime`), avoid running multiple benchmark processes in parallel (e.g., with `pytest-xdist`). Parallel execution can lead to noisy and inconsistent measurements. For optimal consistency in walltime mode, use isolated machines like CodSpeed Hosted Macro Runners.","severity":"gotcha","affected_versions":"All versions with walltime instrument (>=3.0.0)"},{"fix":"Ensure your project's Python environment is 3.9 or later. Upgrade Python if necessary.","message":"Python 3.9 or newer is required. Older Python versions are not supported.","severity":"gotcha","affected_versions":"<3.9"},{"fix":"Add `assert` statements to your benchmark functions to validate their behavior, just like any other unit test.","message":"Benchmarks are still tests: always include assertions in your benchmark functions to verify that the code being measured is producing the expected output. This ensures that you're not just measuring speed, but the speed of correct functionality.","severity":"gotcha","affected_versions":"All"}],"env_vars":null,"last_verified":"2026-04-10T00:00:00.000Z","next_check":"2026-07-09T00:00:00.000Z"}