NIXL Python API (CUDA 12)

1.0.0 · active · verified Tue Apr 14

NIXL (NVIDIA Inference Xfer Library) is an open-source Python API designed to accelerate point-to-point communications in AI inference frameworks. It provides a unified abstraction layer over various memory types (CPU, GPU) and storage (file, block, object store) through a modular plugin architecture. The `nixl-cu12` package specifically targets CUDA 12 environments. NIXL is actively maintained with frequent releases, with version 1.0.0 being the current stable release.

Warnings

breaking NIXL 1.0.0 introduces significant breaking changes, including the removal of the legacy Multi-Object UCX backend and the complete transition to Device API V2. The previous Device API V1 implementation has been removed.
Fix: Review and update your code to utilize the new Device API V2. For UCX backend configurations, adapt to the updated request handling mechanisms. Refer to the official NIXL 1.0.0 release notes and documentation for detailed migration guides.
gotcha Installing both `nixl-cu12` and `nixl-cu13` (or their respective meta-package variants) in the same environment can lead to unexpected behavior. If both are present, `nixl-cu13` will take precedence.
Fix: Ensure only the package corresponding to your target CUDA version is installed (`nixl-cu12` for CUDA 12, `nixl-cu13` for CUDA 13). If using the `nixl` meta-package, specify the correct extra, e.g., `pip install "nixl[cu12]"`.
gotcha NIXL is currently only supported on Linux environments (tested on Ubuntu and Fedora). It does not support macOS or Windows.
Fix: Run NIXL in a Linux host, container, or virtual machine environment.
gotcha Errors such as `NIXL_ERR_BACKEND` often indicate issues with the underlying backend configuration or its availability. This can stem from missing dependencies (like UCX or specific CUDA libraries), incorrect paths, or misconfigurations during agent initialization.
Fix: Verify that all required backend libraries (e.g., UCX) are correctly installed and accessible. Check NIXL agent initialization parameters and ensure they match your system's hardware and software configuration. Review NIXL's backend guide for specific setup instructions.

Install

pip install nixl-cu12 Install for CUDA 12
pip install "nixl[cu12]" # for CUDA 12 # For backwards compatibility, 'pip install nixl' automatically installs nixl[cu12] Install via meta-package

Imports

nixl_agent
```
from nixl import nixl_agent
```
The primary interaction is typically by instantiating `nixl_agent` from the top-level `nixl` package after importing it.

Quickstart

This quickstart demonstrates how to import the `nixl` library and initialize a NIXL agent. The agent is the core component for managing data transfers within the NIXL framework.

import nixl

# Initialize a NIXL agent
try:
    agent = nixl.nixl_agent('my_inference_agent')
    print(f"NIXL agent '{agent.name}' initialized successfully.")
    # Further NIXL operations would follow here, e.g., memory registration, transfer requests.
except Exception as e:
    print(f"Error initializing NIXL agent: {e}")

view raw JSON →