NVIDIA cuFile Python Wrapper

Common errors

• ImportError: No module named 'cufile'

  cause The 'cufile-python' package is not installed or not accessible in your current Python environment.
  fix Run `pip install cufile-python` to install the package.

• RuntimeError: cuFile driver initialization failed

  cause The underlying NVIDIA cuFile C library could not be initialized by the Python wrapper. This almost always indicates that the cuFile system library is not installed, cannot be found in the system's library path, or there's an issue with the GPU setup or drivers.
  fix Verify your NVIDIA GPU drivers and CUDA Toolkit are correctly installed. Ensure the cuFile library (e.g., `libcufile.so`) is present and discoverable by the system's linker (e.g., by checking `LD_LIBRARY_PATH` or system linker configurations). Consult NVIDIA cuFile documentation for specific installation and environment setup instructions.

• TypeError: expected a bytes-like object, not 'str'

  cause You are passing a standard Python `str` object to a `cufile.lib` function (e.g., `open`, `read`, `write`) that expects a `bytes` object for arguments like file paths or data buffers.
  fix Encode your string to bytes before passing it to the function. For example, change `lib.cuFileDriver.open(filename, os.O_RDWR)` to `lib.cuFileDriver.open(filename.encode('utf-8'), os.O_RDWR)`.

Warnings

• gotcha This library is a Python wrapper for the NVIDIA cuFile C library. It absolutely requires an NVIDIA GPU, a compatible CUDA Toolkit installation, and the cuFile library components to be correctly installed and configured on the system. Installing `cufile-python` via pip alone is insufficient for functionality.
  Fix: Verify that your system meets all NVIDIA hardware and software prerequisites for cuFile (GPU, NVIDIA drivers, CUDA Toolkit, cuFile library) and that the cuFile library is discoverable by the Python wrapper (e.g., via `LD_LIBRARY_PATH` on Linux).
• deprecated The cufile-python project appears to be unmaintained. The last commit and PyPI release were in October 2021. Users should be aware that there is no active development, bug fixes, or official support, which introduces risks for long-term projects or compatibility with newer CUDA/Python versions.
  Fix: Evaluate the long-term implications before integrating this library into critical projects. Consider if its current functionality is sufficient without requiring future updates or support, or explore alternative GPU-accelerated I/O solutions.
• gotcha Many low-level functions within cufile-python that accept file paths or data buffers expect them to be `bytes` objects, not standard Python `str`. Providing `str` directly will result in a `TypeError`.
  Fix: Always encode string paths and data to bytes (e.g., `filename.encode('utf-8')`) before passing them to cuFile functions like `cuFileDriver.open` or `cuFileDriver.read`.

Install

• pip install cufile-python Install cufile-python

Imports

• lib
  wrong:

  import cufile

  correct:

  import cufile.lib as lib

  The primary API is exposed through the 'lib' submodule, which contains the cuFileDriver class and related functions. Importing 'cufile' directly does not expose the functional API.
• cuFileDriver
  wrong:

  from cufile import cuFileDriver

  correct:

  from cufile.lib import cuFileDriver

  The `cuFileDriver` class, which provides the main cuFile functionalities, is located within the `lib` submodule of `cufile`.

Quickstart

This quickstart demonstrates the essential steps for interacting with cufile-python: initializing the driver, opening a file with a cuFile descriptor, and deinitializing. It highlights the use of `cufile.lib.cuFileDriver` and the requirement for `bytes` filenames. Actual GPU-direct I/O operations involving data transfer are more complex and would typically follow this setup.

import cufile.lib as lib
import os

# --- Prerequisites: NVIDIA GPU, CUDA Toolkit, and cuFile library must be installed ---
# This example assumes cuFile is correctly set up and discoverable in the system's library path.

# Ensure a dummy file exists for I/O operations
filename = "/tmp/test_cufile.txt"
with open(filename, "w") as f:
    f.write("Hello, cuFile GPU-Direct Storage!")

try:
    # Initialize cuFile library
    lib.cuFileDriver.init()
    print("cuFile driver initialized successfully.")

    # Open file with cuFile. Note: Filename must be encoded to bytes.
    # os.O_RDWR is a standard file access flag.
    fd = lib.cuFileDriver.open(filename.encode('utf-8'), os.O_RDWR)
    print(f"Opened file '{filename}' with cuFile descriptor: {fd}")

    # In a real application, you would now perform GPU-direct I/O operations 
    # using this file descriptor and GPU memory buffers.
    # For this quickstart, we just demonstrate the basic open/close workflow.

    # Close file
    lib.cuFileDriver.close(fd)
    print(f"Closed cuFile descriptor: {fd}")

except RuntimeError as e:
    print(f"Error during cuFile operation: {e}\n")
    print("HINT: Ensure NVIDIA cuFile library is correctly installed, and your GPU is configured.")
except Exception as e:
    print(f"An unexpected error occurred: {e}")
finally:
    # Deinitialize cuFile library
    try:
        lib.cuFileDriver.fini()
        print("cuFile driver deinitialized.")
    except Exception as e:
        # Fini might fail if init failed or was never called successfully
        print(f"Warning: Error during cuFile deinitialization: {e}")

    # Clean up the dummy file
    if os.path.exists(filename):
        os.remove(filename)
        print(f"Cleaned up temporary file: {filename}")