RMM - RAPIDS Memory Manager (CUDA 12)

Common errors

• ModuleNotFoundError: No module named 'rmm._lib'

  cause Attempting to import from the internal `rmm._lib` module which was removed in RMM v25.02.00.
  fix Remove `from rmm import _lib` or any direct `_lib` imports. Use public RMM APIs like `rmm.device_buffer.DeviceBuffer` or `rmm.mr.CudaMemoryResource` instead.

• RuntimeError: RMM memory resource not initialized

  cause Attempting to allocate RMM-managed memory (e.g., with `DeviceBuffer` or `cupy`) without first setting a global RMM memory resource.
  fix Initialize and set a default RMM memory resource using `rmm.mr.set_current_device_resource(rmm.mr.PoolMemoryResource())` (or another desired resource) before performing any allocations.

• CUDA_ERROR_OUT_OF_MEMORY: out of memory

  cause The GPU ran out of memory, potentially due to large allocations, fragmentation, or an insufficient memory pool size if using `PoolMemoryResource`.
  fix Reduce memory usage, optimize data structures, or increase the `initial_pool_size` and `maximum_pool_size` of your `PoolMemoryResource`. Consider using `ManagedMemoryResource` for automatic memory oversubscription (if supported by your GPU and driver) or ensuring proper deallocation.

• RuntimeError: RMM cannot be used with CUDA 11.x. Please upgrade to CUDA 12.x or install an RMM version compatible with CUDA 11.x.

  cause Using `librmm-cu12` (or RMM v25.08.00+) with an older CUDA Toolkit version (e.g., 11.x).
  fix Upgrade your CUDA Toolkit to 12.x and ensure `cupy-cuda12x` and `cuda-python` are correctly installed. Alternatively, downgrade RMM to a version compatible with your CUDA 11.x setup (e.g., `librmm-cu11`).

Warnings

• breaking Starting with RMM v25.08.00, `librmm-cu12` (and other RMM packages) explicitly require CUDA 12.0 or newer. Previous RMM versions might have supported CUDA 11.x.
  Fix: Ensure your system has CUDA Toolkit 12.x installed and that your `cuda-python` and `cupy-cuda12x` installations match this version. If using older CUDA, you must use an older RMM package (e.g., `librmm-cu11`).
• breaking The internal `rmm._lib` module was removed in v25.02.00. Direct imports from this path will result in `ModuleNotFoundError`.
  Fix: Refactor your code to use the public APIs available directly under `import rmm` or `from rmm.mr import ...`. Most functionality previously in `_lib` is now exposed through these public interfaces.
• breaking RMM v26.02.00 removed host-only memory resources and deprecated certain legacy memory resource interfaces, standardizing on the CCCL interface. Code using `HostMemoryResource` or older `memory_resource` patterns will break.
  Fix: Update memory resource instantiation and usage to align with the new interfaces. Primarily, device-backed memory resources (e.g., `CudaMemoryResource`, `PoolMemoryResource`) should be used. Review RMM documentation for the latest `rmm.mr` API.
• deprecated Direct access to RMM's internal logger (`rmm.logger`) was deprecated in v24.12.00 and fully removed/refactored in v25.02.00/v25.04.00 to use `rapids-logger`.
  Fix: Migrate logging configurations to use the `rapids-logger` library directly or rely on `rmm`'s exposed logging configuration methods if available. Avoid attempting to access `rmm.logger` directly.

Install

• pip install librmm-cu12 Install for CUDA 12.x

Imports

• DeviceBuffer

  from rmm.device_buffer import DeviceBuffer

• CudaMemoryResource

  from rmm.mr import CudaMemoryResource

• PoolMemoryResource

  from rmm.mr import PoolMemoryResource

• set_current_device_resource

  from rmm.mr import set_current_device_resource

• rmm

  import rmm

• rmm._lib
  wrong:

  from rmm import _lib

  correct:

  import rmm

  The internal `rmm._lib` module was removed in v25.02.00. Use public APIs from `rmm` or `rmm.mr` instead.

Quickstart

This quickstart demonstrates how to configure RMM with a `PoolMemoryResource` for efficient GPU memory management and how to allocate memory directly using `DeviceBuffer` or implicitly through libraries like CuPy. Ensure you have CuPy installed (`pip install cupy-cuda12x`) to run the CuPy example.

import rmm
from rmm.mr import PoolMemoryResource, set_current_device_resource
from rmm.device_buffer import DeviceBuffer
import cupy as cp

# 1. Configure RMM with a memory resource (e.g., a memory pool)
pool_size_bytes = 2 * 1024**3 # 2 GB
max_pool_size_bytes = 4 * 1024**3 # 4 GB

# Create a PoolMemoryResource
# Note: As of v26.02.00, host-only memory resources were removed.
# This example uses a device-backed pool.
mr = PoolMemoryResource(initial_pool_size=pool_size_bytes, maximum_pool_size=max_pool_size_bytes)
set_current_device_resource(mr)

print(f"Current RMM memory resource set to: {rmm.mr.get_current_device_resource()}")

# 2. Allocate device memory directly with RMM
db = DeviceBuffer(size=1024, dtype='uint8')
print(f"Allocated DeviceBuffer of size {db.size} bytes: {db}")

# 3. Use CuPy with RMM integration (CuPy will automatically use RMM)
a = cp.arange(10**6, dtype=cp.float32)
b = a * 2
print(f"CuPy array created using RMM: {a.shape}, dtype={a.dtype}")

# Clean up (optional, as RMM resources are typically global and managed)
del db
del a, b
# Note: The PoolMemoryResource itself will be deallocated when 'mr' goes out of scope
# or when the program exits, releasing its managed memory.