cymem

2.0.13 · active · verified Sun Mar 29

cymem is a Python library that provides efficient memory-management helpers for Cython. It simplifies tying C-level memory allocations (via `calloc`/`free`) to the lifecycle of Python objects, automatically freeing memory when the owning Python object is garbage collected. The core component is `cymem.Pool`, a thin wrapper around `calloc`. Currently at version 2.0.13, it maintains a regular release cadence, often aligning with new Python version support and performance enhancements like free-threading.

Warnings

gotcha When `cymem.Pool` is used with CPython 3.13+ free-threaded builds (PEP 703), operations like `alloc()`, `free()`, and `realloc()` are thread-safe. However, reading the internal state (e.g., `addresses` dict) without explicit critical sections is not thread-safe. Users are also responsible for synchronizing access to the *contents* of the allocated memory across threads.
Fix: Ensure critical sections are used for internal state access, and apply fine-grained locks or other synchronization primitives when sharing access to memory contents across threads. Do not rely on coarse-grained locks on the `Pool` instance itself for memory content synchronization.
breaking Incompatibility between `cymem`'s compiled C++ files and the installed Cython version can lead to errors like `ValueError: cymem.cymem.Pool has the wrong size, try recompiling.` This often happens when Cython's internal structures change.
Fix: Ensure your installed Cython version is compatible with the `cymem` version. If encountering this error, try recompiling your Cython project (if you're compiling `cymem` from source or an older wheel) or updating `cymem` to a version with wheels pre-built against a compatible Cython release. It's recommended to update `pip`, `setuptools`, and `wheel` before installing to ensure the latest compatible binary wheels are used.
gotcha `cymem.Pool` simplifies memory deallocation by tying it to the Python object's lifecycle. However, users must still ensure that no raw C pointers obtained from the pool outlive the Python object that owns the `Pool` instance. If the `Pool` object is garbage collected, all its managed memory is freed, invalidating any lingering pointers.
Fix: Carefully manage the lifetime of your `cdef` Python objects that hold `cymem.Pool` instances. Ensure that any C pointers derived from the pool are only accessed while the owning Python object and its associated `Pool` are still alive and in scope.

Install

pip install cymem Install with pip

Imports

Pool
```
from cymem.cymem cimport Pool
```
Used in Cython (.pyx) files to import the memory pool class. This is a cimport, not a regular Python import.

Quickstart

This example demonstrates how to allocate C-level memory using `cymem.Pool` within a Cython `.pyx` file. The `Pool` object handles the deallocation of all memory allocated through it when the `Pool` instance itself is garbage collected, simplifying memory management in Cython extensions.

from cymem.cymem cimport Pool
from libc.stdlib cimport sizeof

def main():
    cdef Pool mem = Pool()
    cdef int* data1 = <int*>mem.alloc(10, sizeof(int))
    cdef float* data2 = <float*>mem.alloc(12, sizeof(float))

    # Use data1 and data2
    data1[0] = 100
    data2[0] = 3.14

    print(f"Data1 at index 0: {data1[0]}")
    print(f"Data2 at index 0: {data2[0]}")

    # Memory is automatically freed when 'mem' (the Pool object) is garbage collected.
    # No explicit free() calls are needed for memory allocated via Pool.

# To run this, you would typically compile it with Cython:
# cython -3 --inplace your_module.pyx
# Then import and call main() from Python:
# import your_module
# your_module.main()

view raw JSON →