ppft: Parallel Python Framework

1.7.8 · active · verified Sun Mar 29

ppft is a friendly fork of Parallel Python (pp), designed to provide distributed and parallel Python capabilities on Symmetric Multi-Processing (SMP) systems and clusters. It offers an easier installation process and enhanced serialization through the optional `dill` library. Currently at version 1.7.8, ppft maintains an active development and release cadence, with recent updates focused on Python 3 compatibility and dropping support for older Python versions.

Warnings

breaking Python 2.x support was dropped in early versions, and Python 3.x minimum requirements have increased incrementally. Version 1.7.8 requires Python >=3.9, and upcoming versions (e.g., 1.7.9.dev0 documentation) indicate a requirement of Python >=3.10. Ensure your Python environment meets the specific version's requirements.
Fix: Upgrade your Python environment to 3.9 or higher. For future compatibility, target Python 3.10+.
gotcha ppft installs itself as the 'pp' module. If the original 'Parallel Python' (pp) library is already installed in your environment, 'import pp' may point to the original library instead of the ppft fork. This can lead to unexpected behavior or missing features.
Fix: Before installing ppft, uninstall any existing 'pp' installations: `pip uninstall pp`.
gotcha For enhanced serialization of complex Python objects (e.g., lambdas, nested functions, objects with custom serialization), it is highly recommended to install ppft with the optional 'dill' dependency. Without it, some objects might fail to serialize correctly for parallel execution.
Fix: Install ppft using `pip install ppft[dill]` to include the `dill` library for improved serialization.

Install

pip install ppft Basic Installation
pip install ppft[dill] With Enhanced Serialization

Imports

Server
```
import pp
job_server = pp.Server()
```
ppft installs and is accessed via the `pp` module, maintaining compatibility with the original Parallel Python library. Directly importing from `ppft` is not the intended mechanism for core functionalities.
pp
```
import pp
```
The primary way to interact with ppft is by importing the `pp` module.

Quickstart

This quickstart demonstrates how to create a local `pp.Server`, submit a simple function for parallel execution, and collect the results. The server automatically manages worker processes.

import pp
import math

def my_function(a, b):
    return math.sqrt(a**2 + b**2)

# Create a job server
# The number of workers can be specified, e.g., pp.Server(4)
job_server = pp.Server()

# Submit jobs
jobs = []
for i in range(10):
    jobs.append(job_server.submit(my_function, (i, i+1), ())) # func, args, modules

# Retrieve results
results = [job() for job in jobs]
print(f"Calculated results: {results}")

# Destroy the job server
job_server.destroy()

view raw JSON →