execnet: Distributed Python Deployment and Communication

2.1.2 · maintenance · verified Sat Mar 28

execnet is a Python library that provides carefully tested means to ad-hoc interact with Python interpreters across different versions, platforms, and network barriers. It offers a minimal and fast API for distributing tasks to local or remote processes, writing hybrid multi-process applications, and administering multiple hosts. The project is currently in maintenance-only mode, with a focus on bug fixes, and is not recommended for new projects. The current stable version is 2.1.2, with releases driven by bug fixes and Python version compatibility updates.

Warnings

deprecated execnet is in maintenance-only mode and should not be used for new projects. Modern Python's capabilities for inter-interpreter communication often make `execnet` unnecessary. It is primarily maintained because it serves as the backend for `pytest-xdist`.
Fix: For new projects, consider alternatives like multiprocessing, concurrent.futures, or dedicated message queues (e.g., Celery, RabbitMQ) depending on your use case.
breaking Version 2.0.0 dropped support for Python versions older than 3.7. Ensure your environment meets the minimum Python requirement of >=3.8.
Fix: Upgrade your Python interpreter to 3.8 or newer. If you need to support older Python versions, you must use an older `execnet` release (e.g., <2.0.0 for Python <3.7).
breaking Version 1.5.0 removed support for Python 2.6 and 3.3. Earlier versions of Python 3.x (e.g., 3.3) are no longer supported.
Fix: Ensure your Python environment is 2.7 (for older series) or 3.4+ (for 1.5.x to <2.0.0 series). For current versions, refer to the Python >=3.8 requirement.
gotcha When using `python=` in a gateway specification, especially with `popen` or `ssh` gateways, pathnames containing spaces can lead to potential regressions or unexpected behavior due to the internal use of `shlex.split`.
Fix: Avoid spaces in Python interpreter pathnames if possible. If unavoidable, thoroughly test your gateway specifications with quoted or escaped paths, or consider using a wrapper script on the remote end to call the desired interpreter.
gotcha The `execnet.dumps` and `execnet.loads` API for cross-interpreter compatible serialization of Python builtin types was introduced in version 1.1. Using serialization mechanisms across different `execnet` versions or without these explicit functions might lead to incompatibility issues.
Fix: Always use `execnet.dumps` and `execnet.loads` for reliable cross-interpreter serialization. Ensure all communicating `execnet` instances are running compatible versions, ideally the same version, especially regarding serialization.

Install

pip install execnet Install stable version

Imports

makegateway
```
import execnet
gateway = execnet.makegateway()
```
While `from execnet import makegateway` might work, the official examples consistently use `execnet.makegateway()`, implying a preference for accessing it through the main `execnet` module object.
remote_exec
```
channel = gateway.remote_exec(source_code)
```
Accessed directly from a `Gateway` object returned by `execnet.makegateway()`.

Quickstart

This quickstart demonstrates creating a local subprocess gateway, executing code on it, and establishing bidirectional communication using a channel. The remote code sends a message, which is received locally, and then a message is sent back to the remote before closing the connection.

import execnet

gateway = execnet.makegateway()
channel = gateway.remote_exec("channel.send('hello from remote!')")
message = channel.receive()
print(f"Received from remote: {message}")
channel.send('hello from local!')
channel.waitclose()
gateway.exit()

view raw JSON →