MMGP: GPU Memory Management

3.7.6 · active · verified Fri Apr 17

mmgp (Memory Management for the GPU Poor) is a low-level Python library for direct GPU memory management, primarily designed for use with `tinygrad`. It provides functions for allocating and freeing raw memory on the GPU. As of version `3.7.6`, it offers granular control over GPU resources, allowing for efficient memory handling in performance-critical applications. Its release cadence is irregular, often aligning with `tinygrad` development.

Common errors

```
RuntimeError: Out of GPU memory
```
cause The application attempted to allocate more GPU memory than is currently available on the device.

fix Reduce the size of your memory allocations, ensure all unused memory is explicitly `free`d, or switch to a GPU with more VRAM. Profile your application's memory usage.
```
AssertionError: pointer not in allocated_buffers
```
cause This error typically occurs when `mmgp.free()` is called with a pointer that was not previously allocated by `mmgp.alloc()` or has already been freed.

fix Ensure that every `alloc()` call has a corresponding `free()` call exactly once. Do not attempt to free memory that was not allocated by `mmgp` or that has already been deallocated.

Warnings

breaking mmgp is tightly coupled with tinygrad's internal memory management and helper utilities. Significant updates or breaking changes in tinygrad (especially its memory model) may necessitate corresponding updates or code changes in mmgp.
Fix: Always check compatibility with the specific tinygrad version you are using. Refer to tinygrad's release notes and mmgp's `setup.py` or dependencies for compatible versions.
gotcha mmgp provides low-level, manual GPU memory management. Users are fully responsible for correctly allocating and deallocating memory. Failure to `free` allocated pointers will lead to GPU memory leaks, and attempting to `free` an invalid or already freed pointer can lead to crashes or undefined behavior.
Fix: Implement robust memory management practices. Use context managers if possible (though `mmgp` doesn't provide them directly for `alloc/free`), or ensure every `alloc` has a corresponding `free` exactly once. Debug memory issues carefully.
gotcha Due to its direct interaction with GPU drivers and hardware, mmgp can be sensitive to specific GPU models, driver versions, and operating system configurations. This may lead to unexpected behavior or errors on certain setups.
Fix: Test your application thoroughly across target hardware and driver versions. Consult the tinygrad community or issues for known compatibility problems with specific GPUs/drivers.

Install

pip install mmgp Install mmgp

Imports

alloc, free, zeros
```
from mmgp import alloc, free, zeros
```
These are the primary module-level functions for direct memory management.

Quickstart

This quickstart demonstrates basic GPU memory allocation and deallocation using `mmgp.alloc`, `mmgp.zeros`, and `mmgp.free`. It allocates memory, prints the returned pointer, and then frees it.

import mmgp

# Allocate 1MB of GPU memory
sz = 1024 * 1024 # 1MB
ptr = mmgp.alloc(sz)
print(f"Allocated {sz} bytes at GPU pointer: {hex(ptr)}")

# Allocate 512KB of zero-initialized GPU memory
ptr_zeros = mmgp.zeros(512 * 1024)
print(f"Allocated 512KB zero-initialized at GPU pointer: {hex(ptr_zeros)}")

# Free the allocated memory
mmgp.free(ptr)
print(f"Freed {sz} bytes from GPU pointer: {hex(ptr)}")

mmgp.free(ptr_zeros)
print(f"Freed 512KB from GPU pointer: {hex(ptr_zeros)}")

view raw JSON →