Buildkite Test Collector for Python

1.4.1 · active · verified Sun Apr 12

The `buildkite-test-collector` is the official Python adapter for Buildkite Test Engine, designed to collect and report test results, primarily from `pytest` suites, to the Buildkite Test Engine dashboard. It integrates seamlessly with CI environments and is actively maintained with frequent updates.

Warnings

breaking Python 3.8 is no longer supported. The library now requires Python >= 3.9.
Fix: Upgrade your Python environment to 3.9 or newer.
gotcha The requirement for the `CI` environment variable for uploads has changed multiple times. While v1.3.1 introduced automatic detection for common CI systems (Buildkite, GitHub Actions, CircleCI), older versions or less common CI systems might still require it.
Fix: For common CI systems, upgrade to v1.3.1 or later. For unknown CI systems or older versions, ensure the `CI` environment variable is set to any value, e.g., `CI=true`.
breaking Teardown failures were previously ignored and not reported as failed tests. This was fixed in v1.3.0.
Fix: Upgrade to v1.3.0 or later to ensure teardown failures are correctly reported.
gotcha Subtest reports in pytest might have incorrectly overwritten the parent test result in versions prior to v1.4.0.
Fix: Upgrade to v1.4.0 or later to ensure correct reporting of subtest results without overwriting the parent test's outcome.
gotcha The `BUILDKITE_ANALYTICS_TOKEN` environment variable is mandatory for the collector to upload test results. Without it, tests will run but results won't be sent to Buildkite.
Fix: Always set the `BUILDKITE_ANALYTICS_TOKEN` securely in your CI environment or locally during development.
breaking The format for `span detail` objects in tracing was fixed in v1.2.0 to align with Buildkite's documentation. If you were manually crafting span data, its format might need adjustment.
Fix: Refer to the Buildkite documentation for the correct `span detail` format and update any custom tracing implementations.

Install

pip install buildkite-test-collector Install with pip

Imports

pytest_plugin
```
The library acts as a pytest plugin; no explicit import is typically required for basic functionality.
```
Functionality is automatically enabled when `buildkite-test-collector` is installed alongside `pytest`.

Quickstart

To quickly start collecting test data, first install the library and pytest. Then, set the `BUILDKITE_ANALYTICS_TOKEN` environment variable with your Buildkite Test Engine API token and run your pytest suite as usual. The collector will automatically send test results to Buildkite. For local testing, ensure `BUILDKITE_ANALYTICS_TOKEN` is set, or mock it.

import os

# test_example.py
def test_passing_example():
    assert True

def test_failing_example():
    assert False

# To run:
# 1. Set the Buildkite Analytics Token:
#    export BUILDKITE_ANALYTICS_TOKEN='your-api-token'
# 2. Run pytest:
#    pytest
# You can use os.environ.get('BUILDKITE_ANALYTICS_TOKEN', '') for local testing
# but in CI, it must be properly set as an environment variable.

view raw JSON →