{"id":8273,"library":"librmm-cu12","title":"RMM - RAPIDS Memory Manager (CUDA 12)","description":"RMM (RAPIDS Memory Manager) provides a C++ library and Python bindings for managing GPU device memory. It offers various memory resources, including pooling allocators, to improve performance and reduce fragmentation for CUDA-enabled applications. The `librmm-cu12` package is specifically built for CUDA 12.x environments. It follows the RAPIDS release cadence, typically releasing new versions monthly.","status":"active","version":"26.4.0","language":"en","source_language":"en","source_url":"https://github.com/rapidsai/rmm","tags":["CUDA","GPU","memory management","RAPIDS","device memory"],"install":[{"cmd":"pip install librmm-cu12","lang":"bash","label":"Install for CUDA 12.x"}],"dependencies":[{"reason":"Required for certain RMM functionalities and compatibility in some environments (e.g., v25.10.00 notes).","package":"numba-cuda","optional":true},{"reason":"Underpins RMM's CUDA interactions; ensure compatible versions are installed.","package":"cuda-python","optional":false}],"imports":[{"symbol":"DeviceBuffer","correct":"from rmm.device_buffer import DeviceBuffer"},{"symbol":"CudaMemoryResource","correct":"from rmm.mr import CudaMemoryResource"},{"symbol":"PoolMemoryResource","correct":"from rmm.mr import PoolMemoryResource"},{"symbol":"set_current_device_resource","correct":"from rmm.mr import set_current_device_resource"},{"symbol":"rmm","correct":"import rmm"},{"note":"The internal `rmm._lib` module was removed in v25.02.00. Use public APIs from `rmm` or `rmm.mr` instead.","wrong":"from rmm import _lib","symbol":"rmm._lib","correct":"import rmm"}],"quickstart":{"code":"import rmm\nfrom rmm.mr import PoolMemoryResource, set_current_device_resource\nfrom rmm.device_buffer import DeviceBuffer\nimport cupy as cp\n\n# 1. Configure RMM with a memory resource (e.g., a memory pool)\npool_size_bytes = 2 * 1024**3 # 2 GB\nmax_pool_size_bytes = 4 * 1024**3 # 4 GB\n\n# Create a PoolMemoryResource\n# Note: As of v26.02.00, host-only memory resources were removed.\n# This example uses a device-backed pool.\nmr = PoolMemoryResource(initial_pool_size=pool_size_bytes, maximum_pool_size=max_pool_size_bytes)\nset_current_device_resource(mr)\n\nprint(f\"Current RMM memory resource set to: {rmm.mr.get_current_device_resource()}\")\n\n# 2. Allocate device memory directly with RMM\ndb = DeviceBuffer(size=1024, dtype='uint8')\nprint(f\"Allocated DeviceBuffer of size {db.size} bytes: {db}\")\n\n# 3. Use CuPy with RMM integration (CuPy will automatically use RMM)\na = cp.arange(10**6, dtype=cp.float32)\nb = a * 2\nprint(f\"CuPy array created using RMM: {a.shape}, dtype={a.dtype}\")\n\n# Clean up (optional, as RMM resources are typically global and managed)\ndel db\ndel a, b\n# Note: The PoolMemoryResource itself will be deallocated when 'mr' goes out of scope\n# or when the program exits, releasing its managed memory.","lang":"python","description":"This quickstart demonstrates how to configure RMM with a `PoolMemoryResource` for efficient GPU memory management and how to allocate memory directly using `DeviceBuffer` or implicitly through libraries like CuPy. Ensure you have CuPy installed (`pip install cupy-cuda12x`) to run the CuPy example."},"warnings":[{"fix":"Ensure your system has CUDA Toolkit 12.x installed and that your `cuda-python` and `cupy-cuda12x` installations match this version. If using older CUDA, you must use an older RMM package (e.g., `librmm-cu11`).","message":"Starting with RMM v25.08.00, `librmm-cu12` (and other RMM packages) explicitly require CUDA 12.0 or newer. Previous RMM versions might have supported CUDA 11.x.","severity":"breaking","affected_versions":">=25.08.00"},{"fix":"Refactor your code to use the public APIs available directly under `import rmm` or `from rmm.mr import ...`. Most functionality previously in `_lib` is now exposed through these public interfaces.","message":"The internal `rmm._lib` module was removed in v25.02.00. Direct imports from this path will result in `ModuleNotFoundError`.","severity":"breaking","affected_versions":">=25.02.00"},{"fix":"Update memory resource instantiation and usage to align with the new interfaces. Primarily, device-backed memory resources (e.g., `CudaMemoryResource`, `PoolMemoryResource`) should be used. Review RMM documentation for the latest `rmm.mr` API.","message":"RMM v26.02.00 removed host-only memory resources and deprecated certain legacy memory resource interfaces, standardizing on the CCCL interface. Code using `HostMemoryResource` or older `memory_resource` patterns will break.","severity":"breaking","affected_versions":">=26.02.00"},{"fix":"Migrate logging configurations to use the `rapids-logger` library directly or rely on `rmm`'s exposed logging configuration methods if available. Avoid attempting to access `rmm.logger` directly.","message":"Direct access to RMM's internal logger (`rmm.logger`) was deprecated in v24.12.00 and fully removed/refactored in v25.02.00/v25.04.00 to use `rapids-logger`.","severity":"deprecated","affected_versions":">=24.12.00"}],"env_vars":null,"last_verified":"2026-04-16T00:00:00.000Z","next_check":"2026-07-15T00:00:00.000Z","problems":[{"fix":"Remove `from rmm import _lib` or any direct `_lib` imports. Use public RMM APIs like `rmm.device_buffer.DeviceBuffer` or `rmm.mr.CudaMemoryResource` instead.","cause":"Attempting to import from the internal `rmm._lib` module which was removed in RMM v25.02.00.","error":"ModuleNotFoundError: No module named 'rmm._lib'"},{"fix":"Initialize and set a default RMM memory resource using `rmm.mr.set_current_device_resource(rmm.mr.PoolMemoryResource())` (or another desired resource) before performing any allocations.","cause":"Attempting to allocate RMM-managed memory (e.g., with `DeviceBuffer` or `cupy`) without first setting a global RMM memory resource.","error":"RuntimeError: RMM memory resource not initialized"},{"fix":"Reduce memory usage, optimize data structures, or increase the `initial_pool_size` and `maximum_pool_size` of your `PoolMemoryResource`. Consider using `ManagedMemoryResource` for automatic memory oversubscription (if supported by your GPU and driver) or ensuring proper deallocation.","cause":"The GPU ran out of memory, potentially due to large allocations, fragmentation, or an insufficient memory pool size if using `PoolMemoryResource`.","error":"CUDA_ERROR_OUT_OF_MEMORY: out of memory"},{"fix":"Upgrade your CUDA Toolkit to 12.x and ensure `cupy-cuda12x` and `cuda-python` are correctly installed. Alternatively, downgrade RMM to a version compatible with your CUDA 11.x setup (e.g., `librmm-cu11`).","cause":"Using `librmm-cu12` (or RMM v25.08.00+) with an older CUDA Toolkit version (e.g., 11.x).","error":"RuntimeError: RMM cannot be used with CUDA 11.x. Please upgrade to CUDA 12.x or install an RMM version compatible with CUDA 11.x."}]}