{"id":5789,"library":"nixl-cu12","title":"NIXL Python API (CUDA 12)","description":"NIXL (NVIDIA Inference Xfer Library) is an open-source Python API designed to accelerate point-to-point communications in AI inference frameworks. It provides a unified abstraction layer over various memory types (CPU, GPU) and storage (file, block, object store) through a modular plugin architecture. The `nixl-cu12` package specifically targets CUDA 12 environments. NIXL is actively maintained with frequent releases, with version 1.0.0 being the current stable release.","status":"active","version":"1.0.0","language":"en","source_language":"en","source_url":"https://github.com/ai-dynamo/nixl","tags":["AI inference","data transfer","GPU","CUDA","high-performance","NVIDIA"],"install":[{"cmd":"pip install nixl-cu12","lang":"bash","label":"Install for CUDA 12"},{"cmd":"pip install \"nixl[cu12]\" # for CUDA 12\n# For backwards compatibility, 'pip install nixl' automatically installs nixl[cu12]","lang":"bash","label":"Install via meta-package"}],"dependencies":[{"reason":"Often used in conjunction with NIXL for AI inference frameworks.","package":"PyTorch","optional":false},{"reason":"Recommended prerequisite for source builds and managing Python virtual environments.","package":"uv","optional":true},{"reason":"Prerequisite for source builds.","package":"tomlkit","optional":true},{"reason":"High-performance communication library backend for NIXL.","package":"UCX","optional":false}],"imports":[{"note":"The primary interaction is typically by instantiating `nixl_agent` from the top-level `nixl` package after importing it.","wrong":"import nixl_agent # Direct import of nixl_agent not typically done directly from the module's top level without explicit 'from nixl import'","symbol":"nixl_agent","correct":"from nixl import nixl_agent"}],"quickstart":{"code":"import nixl\n\n# Initialize a NIXL agent\ntry:\n    agent = nixl.nixl_agent('my_inference_agent')\n    print(f\"NIXL agent '{agent.name}' initialized successfully.\")\n    # Further NIXL operations would follow here, e.g., memory registration, transfer requests.\nexcept Exception as e:\n    print(f\"Error initializing NIXL agent: {e}\")","lang":"python","description":"This quickstart demonstrates how to import the `nixl` library and initialize a NIXL agent. The agent is the core component for managing data transfers within the NIXL framework."},"warnings":[{"fix":"Review and update your code to utilize the new Device API V2. For UCX backend configurations, adapt to the updated request handling mechanisms. Refer to the official NIXL 1.0.0 release notes and documentation for detailed migration guides.","message":"NIXL 1.0.0 introduces significant breaking changes, including the removal of the legacy Multi-Object UCX backend and the complete transition to Device API V2. The previous Device API V1 implementation has been removed.","severity":"breaking","affected_versions":"1.0.0 and greater"},{"fix":"Ensure only the package corresponding to your target CUDA version is installed (`nixl-cu12` for CUDA 12, `nixl-cu13` for CUDA 13). If using the `nixl` meta-package, specify the correct extra, e.g., `pip install \"nixl[cu12]\"`.","message":"Installing both `nixl-cu12` and `nixl-cu13` (or their respective meta-package variants) in the same environment can lead to unexpected behavior. If both are present, `nixl-cu13` will take precedence.","severity":"gotcha","affected_versions":"All versions supporting both CUDA 12 and CUDA 13"},{"fix":"Run NIXL in a Linux host, container, or virtual machine environment.","message":"NIXL is currently only supported on Linux environments (tested on Ubuntu and Fedora). It does not support macOS or Windows.","severity":"gotcha","affected_versions":"All versions"},{"fix":"Verify that all required backend libraries (e.g., UCX) are correctly installed and accessible. Check NIXL agent initialization parameters and ensure they match your system's hardware and software configuration. Review NIXL's backend guide for specific setup instructions.","message":"Errors such as `NIXL_ERR_BACKEND` often indicate issues with the underlying backend configuration or its availability. This can stem from missing dependencies (like UCX or specific CUDA libraries), incorrect paths, or misconfigurations during agent initialization.","severity":"gotcha","affected_versions":"All versions"}],"env_vars":null,"last_verified":"2026-04-14T00:00:00.000Z","next_check":"2026-07-13T00:00:00.000Z"}