Colcon Parallel Executor

0.4.0 · active · verified Fri Apr 17

colcon-parallel-executor is an extension for the colcon build tool, primarily used within the ROS ecosystem, to enable efficient parallel processing of packages during build, test, and install operations. It aims to maximize throughput by building independent packages concurrently. The current version is 0.4.0. Releases are typically tied to updates in the colcon ecosystem or bug fixes.

Common errors

```
colcon: error: unrecognized arguments: --executor parallel
```
cause The `colcon-parallel-executor` package is not installed in the Python environment where `colcon` is being run, or `colcon` cannot find its entry point.

fix Ensure the package is installed: `pip install colcon-parallel-executor`. If using virtual environments, activate the correct environment before running `colcon`.
```
Build process appears to hang indefinitely or shows no progress during parallel execution.
```
cause This can indicate a deadlock between parallel build jobs, resource exhaustion (e.g., too many processes for available RAM/CPU), or a single build step that is inherently sequential and very long-running, giving the appearance of a hang.

fix Try reducing the number of parallel workers (`colcon build --executor parallel --parallel-workers 2`). Monitor system resources (CPU, RAM). Review the logs of the last package that started building to identify potential issues or long-running tasks. Ensure package dependencies are correctly defined.
```
RuntimeError: 'NoneType' object has no attribute 'add_job'
```
cause This specific error (or similar `AttributeError`s related to internal executor logic) can sometimes occur due to an incompatibility between the installed `colcon-core` version and `colcon-parallel-executor`, indicating an API change was not handled.

fix Upgrade both `colcon-core` and `colcon-parallel-executor` to their latest compatible versions: `pip install --upgrade colcon-core colcon-parallel-executor`. If the issue persists, report it to the `colcon-parallel-executor` GitHub repository.

Warnings

deprecated The `colcon-parallel-executor` package supersedes and replaces the older `colcon-concurrent-executor`. If you were previously using `colcon-concurrent-executor`, you should switch to `colcon-parallel-executor` for continued support and improvements.
Fix: Uninstall `colcon-concurrent-executor` and install `colcon-parallel-executor`. Update any scripts to use `--executor parallel` instead of `--executor concurrent`.
gotcha Recent versions of `colcon-core` (e.g., >=0.10.0) often use the `parallel` executor as the default for many build verbs. Explicitly specifying `--executor parallel` might be redundant but harmless in such cases. However, for older `colcon-core` versions, it is necessary to explicitly enable it.
Fix: Always explicitly specify `--executor parallel` if you need to guarantee parallel execution across different `colcon-core` versions or environments. Otherwise, confirm your `colcon-core` default behavior.
gotcha Using a parallel executor can expose race conditions, deadlocks, or resource contention issues if package dependencies are not correctly defined or if underlying build processes are not parallelization-safe (e.g., writing to shared non-thread-safe resources).
Fix: Ensure `package.xml` or equivalent dependency declarations are accurate. If experiencing hangs or unexpected failures, try reducing parallelism (e.g., `--parallel-workers 1`) to debug, or consult package maintainers for parallelization best practices.

Install

pip install colcon-parallel-executor Install latest version

Quickstart

To use the parallel executor, simply pass `--executor parallel` to your colcon command. This will instruct colcon to use this extension to build packages in parallel where possible based on their dependencies. Ensure `colcon-parallel-executor` is installed in the same environment as `colcon`.

colcon build --executor parallel

view raw JSON →