execnet: Distributed Python Deployment and Communication

raw JSON →
2.1.2 verified Tue May 12 auth: no python install: verified quickstart: stale maintenance

execnet is a Python library that provides carefully tested means to ad-hoc interact with Python interpreters across different versions, platforms, and network barriers. It offers a minimal and fast API for distributing tasks to local or remote processes, writing hybrid multi-process applications, and administering multiple hosts. The project is currently in maintenance-only mode, with a focus on bug fixes, and is not recommended for new projects. The current stable version is 2.1.2, with releases driven by bug fixes and Python version compatibility updates.

pip install execnet
error ModuleNotFoundError: No module named 'execnet'
cause The 'execnet' package is not installed in the Python environment where the code is being run.
fix
Install the 'execnet' library using pip: pip install execnet
error ModuleNotFoundError: No module named 'execnet.rsync'
cause This specific sub-module, often used by 'pytest-xdist', cannot be found. This can occur if 'execnet' is not fully or correctly installed, or if there are issues with bundling applications (e.g., with PyInstaller) that prevent sub-modules from being properly included.
fix
Ensure 'execnet' is correctly installed. If bundling with PyInstaller, explicitly include 'execnet.rsync' as a hidden import: pyinstaller --hidden-import=execnet.rsync your_script.py
error execnet.gateway_base.RemoteError: Traceback (most recent call last): ... TypeError: __init__() got an unexpected keyword argument 'execmodel'.
cause This error typically arises from an incompatibility in 'execnet's internal communication, often when connecting to a remote Python interpreter with an older or different 'execnet' version that does not recognize or expect the 'execmodel' keyword argument during gateway bootstrapping.
fix
Ensure that the 'execnet' version is consistent across both the local and remote Python environments. If explicitly setting execmodel, try removing it or checking for compatibility with the target Python interpreter and 'execnet' version.
error Error 111: Connection Refused
cause The 'execnet' gateway could not establish a connection to the specified remote host because the connection was actively refused. This often indicates that an SSH server is not running on the remote machine, or firewall rules are blocking the connection.
fix
Verify that an SSH server is running on the remote machine and is configured to accept connections. Check firewall rules on both the local and remote machines to ensure the necessary ports are open. Confirm that the remote host's IP address/hostname is correct and reachable.
deprecated execnet is in maintenance-only mode and should not be used for new projects. Modern Python's capabilities for inter-interpreter communication often make `execnet` unnecessary. It is primarily maintained because it serves as the backend for `pytest-xdist`.
fix For new projects, consider alternatives like multiprocessing, concurrent.futures, or dedicated message queues (e.g., Celery, RabbitMQ) depending on your use case.
breaking Version 2.0.0 dropped support for Python versions older than 3.7. Ensure your environment meets the minimum Python requirement of >=3.8.
fix Upgrade your Python interpreter to 3.8 or newer. If you need to support older Python versions, you must use an older `execnet` release (e.g., <2.0.0 for Python <3.7).
breaking Version 1.5.0 removed support for Python 2.6 and 3.3. Earlier versions of Python 3.x (e.g., 3.3) are no longer supported.
fix Ensure your Python environment is 2.7 (for older series) or 3.4+ (for 1.5.x to <2.0.0 series). For current versions, refer to the Python >=3.8 requirement.
gotcha When using `python=` in a gateway specification, especially with `popen` or `ssh` gateways, pathnames containing spaces can lead to potential regressions or unexpected behavior due to the internal use of `shlex.split`.
fix Avoid spaces in Python interpreter pathnames if possible. If unavoidable, thoroughly test your gateway specifications with quoted or escaped paths, or consider using a wrapper script on the remote end to call the desired interpreter.
gotcha The `execnet.dumps` and `execnet.loads` API for cross-interpreter compatible serialization of Python builtin types was introduced in version 1.1. Using serialization mechanisms across different `execnet` versions or without these explicit functions might lead to incompatibility issues.
fix Always use `execnet.dumps` and `execnet.loads` for reliable cross-interpreter serialization. Ensure all communicating `execnet` instances are running compatible versions, ideally the same version, especially regarding serialization.
gotcha Attempting to send data over an `execnet` channel that has already been closed results in an `OSError: cannot send to <Channel id=... closed>`. This often indicates that the remote peer closed its end of the channel prematurely, possibly due to an unhandled exception, explicit closure, or a communication breakdown.
fix Investigate the remote process for unhandled exceptions, explicit channel closures, or issues preventing it from processing messages. Ensure the remote end properly manages its channel lifecycle and does not close it before all expected communication from the local side is complete. Implement robust error handling on both ends to manage channel state gracefully.
python os / libc status wheel install import disk
3.10 alpine (musl) - - 0.04s 18.1M
3.10 slim (glibc) - - 0.03s 19M
3.11 alpine (musl) - - 0.08s 20.0M
3.11 slim (glibc) - - 0.06s 20M
3.12 alpine (musl) - - 0.09s 11.9M
3.12 slim (glibc) - - 0.07s 12M
3.13 alpine (musl) - - 0.06s 11.5M
3.13 slim (glibc) - - 0.06s 12M
3.9 alpine (musl) - - 0.04s 17.6M
3.9 slim (glibc) - - 0.03s 18M

This quickstart demonstrates creating a local subprocess gateway, executing code on it, and establishing bidirectional communication using a channel. The remote code sends a message, which is received locally, and then a message is sent back to the remote before closing the connection.

import execnet

gateway = execnet.makegateway()
channel = gateway.remote_exec("channel.send('hello from remote!')")
message = channel.receive()
print(f"Received from remote: {message}")
channel.send('hello from local!')
channel.waitclose()
gateway.exit()