CuPy (CUDA 13.x)

Common errors

• ModuleNotFoundError: No module named 'cupy'

  cause CuPy was either not installed in the active Python environment, or environment variables (like PATH) were not reloaded after installation, particularly when installing a CUDA Toolkit.
  fix Ensure you are in the correct virtual environment. If CuPy was just installed, restart your Python script, IDE, or terminal to refresh environment variables. Verify installation with `pip freeze | grep cupy`.

• TypeError: Argument 'x' has incorrect type (expected cupy.core.core.ndarray, got numpy.ndarray)

  cause Attempting to pass a NumPy array (CPU-resident) directly to a CuPy function that expects a CuPy array (GPU-resident).
  fix Convert the NumPy array to a CuPy array using `cp.asarray()` or `cp.array()` before passing it to CuPy functions. Example: `gpu_array = cp.asarray(numpy_array)`.

• cupy.cuda.compiler.CompileException: nvrtc: error: failed to load builtins; catastrophic error: cannot open source file "cuda_fp16.h"

  cause CuPy's CUDA compiler (NVRTC) cannot find necessary CUDA header files, often due to an incorrect or incomplete CUDA Toolkit installation, or an environment variable (`CUDA_PATH`, `LD_LIBRARY_PATH`) not being set correctly.
  fix Verify your CUDA Toolkit installation. Ensure `CUDA_PATH` or `LD_LIBRARY_PATH` are set if CUDA is in a non-standard location. If using PyPI `[ctk]` installation, make sure the `nvidia-cuda-runtime-cuXX` package is correctly installed to provide headers. You might need to explicitly install `cuda-cudart-dev-12-X` (for CUDA 12) or similar `cuda-cudart-dev-13-X` for CUDA 13.

• TypeError: Implicit conversion to a NumPy array is not allowed. Please use `.get()` to construct a NumPy array explicitly.

  cause Attempting to implicitly convert a CuPy array to a NumPy array in contexts where explicit conversion is required, such as direct interaction with NumPy-only functions or printing large arrays.
  fix Explicitly convert the CuPy array to a NumPy array using `cupy.asnumpy()` or the `.get()` method. For example: `cpu_array = gpu_array.get()` or `cpu_array = cp.asnumpy(gpu_array)`.

Warnings

• breaking CuPy v14 aligns its behavior with NumPy 2 semantics, which includes changes to type promotion rules and casting behavior. Code relying on older NumPy 1.x type promotion might behave differently.
  Fix: Review and adapt code for NumPy 2 compatibility. Refer to NumPy 2 and CuPy v14 release notes for detailed changes.
• breaking CuPy v14 has completely removed all cuDNN-related functionality. Direct usage of `cupy.cuda.cudnn` will fail.
  Fix: Migrate any cuDNN-dependent code to use `cuDNN Frontend` directly or other libraries that wrap cuDNN functionality.
• breaking Support for CUDA 11 and Python 3.9 has been dropped in CuPy v14. Users on these older environments must upgrade.
  Fix: Upgrade to CUDA Toolkit 12.x or 13.x and Python 3.10 or newer.
• gotcha Installing `cupy-cuda13x` requires a compatible NVIDIA CUDA Toolkit 13.x installation or driver. Mismatches in CUDA versions between the installed CuPy wheel and the system's CUDA Toolkit can lead to `ImportError` or runtime compilation errors.
  Fix: Ensure your system's CUDA Toolkit version (specifically the driver) matches the `cupy-cudaXXx` package you install. For easier setup without a full system CUDA Toolkit, use `pip install 'cupy-cuda13x[ctk]'` to install PyPI-distributed CUDA components.
• gotcha Initial execution of CuPy functions can be slower than subsequent calls due to just-in-time compilation and caching of CUDA kernels.
  Fix: This is expected behavior and typically not an issue for repeated operations. For performance-critical loops, ensure initialization or a 'warm-up' run occurs outside the timed section.
• gotcha GPU operations in CuPy are asynchronous by default. For accurate timing of GPU execution in benchmarks or to ensure operations complete before host interaction, explicit synchronization is necessary.
  Fix: Call `cp.cuda.Stream.null.synchronize()` after the GPU computation and before measuring time or accessing results on the CPU.

Install

• pip install cupy-cuda13x Basic Installation (requires system CUDA Toolkit)
• pip install 'cupy-cuda13x[ctk]' Installation with CUDA Toolkit Python packages (driver only needed)

Imports

• cupy

  import cupy as cp

  Standard convention for importing CuPy.
• cupyx.scipy
  wrong:

  import cupy.scipy

  correct:

  import cupyx.scipy as cpxs

  SciPy-compatible functions are located under the `cupyx.scipy` submodule, not directly under `cupy.scipy`.

Quickstart

This quickstart demonstrates basic CuPy array creation, arithmetic operations on the GPU, transferring data between GPU and CPU, and performing a NumPy-like aggregation. It includes a check for GPU availability and the use of `cp.cuda.Stream.null.synchronize()` for explicit GPU synchronization, which is important for accurate performance measurement.

import cupy as cp
import numpy as np

# Check if a GPU is available
if cp.cuda.is_available():
    print(f"CuPy is available. Current device: {cp.cuda.Device().id}")

    # Create a CuPy array on the GPU
    x_gpu = cp.arange(10, dtype=cp.float32).reshape(2, 5)
    print(f"GPU array:\n{x_gpu}")
    print(f"Type of GPU array: {type(x_gpu)}")

    # Perform a computation on the GPU
    y_gpu = x_gpu * 2 + 1
    print(f"Result of computation on GPU:\n{y_gpu}")

    # Transfer the result back to CPU NumPy array
    y_cpu = cp.asnumpy(y_gpu)
    print(f"CPU array (from GPU):\n{y_cpu}")
    print(f"Type of CPU array: {type(y_cpu)}")

    # Demonstrate a simple NumPy-like operation
    sum_gpu = x_gpu.sum(axis=1)
    print(f"Sum along axis 1 on GPU: {sum_gpu}")
    print(f"Type of sum on GPU: {type(sum_gpu)}")

    # Ensure all GPU operations complete before proceeding (useful for timing)
    cp.cuda.Stream.null.synchronize()
else:
    print("No NVIDIA GPU found or CuPy is not properly installed for CUDA.")
    print("Falling back to NumPy for demonstration.")
    x_cpu = np.arange(10, dtype=np.float32).reshape(2, 5)
    print(f"CPU array:\n{x_cpu}")