cymem

raw JSON →
2.0.13 verified Tue May 12 auth: no python install: stale

cymem is a Python library that provides efficient memory-management helpers for Cython. It simplifies tying C-level memory allocations (via `calloc`/`free`) to the lifecycle of Python objects, automatically freeing memory when the owning Python object is garbage collected. The core component is `cymem.Pool`, a thin wrapper around `calloc`. Currently at version 2.0.13, it maintains a regular release cadence, often aligning with new Python version support and performance enhancements like free-threading.

pip install cymem
error ModuleNotFoundError: No module named 'cymem'
cause The `cymem` package is not installed in the current Python environment or the Python interpreter cannot find it.
fix
Ensure cymem is installed using pip install cymem. If using a virtual environment, activate it first. If still failing, check pip list to confirm its presence and verify your Python path.
error ValueError: cymem.cymem.Pool has the wrong size, try recompiling.
cause This error occurs when `cymem` (or a library depending on it) was compiled against a different version of Python, Cython, or operating system architecture than the one currently being used, leading to a binary incompatibility.
fix
Reinstall cymem and its dependent libraries (like spaCy or thinc) from scratch, often by uninstalling and then reinstalling with pip install --no-cache-dir --force-reinstall cymem and similarly for any dependent packages.
error AttributeError: module 'cymem.cymem' has no attribute 'PyMalloc'
cause This typically indicates a version mismatch between `cymem` and another library that depends on it (e.g., `spaCy` or `thinc`), where the dependent library expects an older or newer API from `cymem` that is not present.
fix
Uninstall and then reinstall cymem and its main dependent packages (e.g., pip uninstall thinc cymem && pip install spacy for spaCy-related issues) to ensure compatible versions are installed.
error error C2039: 'tp_print': is not a member of '_typeobject'
cause This is a C compilation error that arises when `cymem` (or its Cython-generated C source) is being compiled with an older version of Cython against a newer Python interpreter (specifically Python 3.9+ which removed `PyTypeObject.tp_print`).
fix
Upgrade your Cython package to a newer version that supports your Python interpreter version using pip install --upgrade Cython. Ensure your compiler (e.g., MSVC on Windows) is correctly set up and compatible with your Python version.
gotcha When `cymem.Pool` is used with CPython 3.13+ free-threaded builds (PEP 703), operations like `alloc()`, `free()`, and `realloc()` are thread-safe. However, reading the internal state (e.g., `addresses` dict) without explicit critical sections is not thread-safe. Users are also responsible for synchronizing access to the *contents* of the allocated memory across threads.
fix Ensure critical sections are used for internal state access, and apply fine-grained locks or other synchronization primitives when sharing access to memory contents across threads. Do not rely on coarse-grained locks on the `Pool` instance itself for memory content synchronization.
breaking Incompatibility between `cymem`'s compiled C++ files and the installed Cython version can lead to errors like `ValueError: cymem.cymem.Pool has the wrong size, try recompiling.` This often happens when Cython's internal structures change.
fix Ensure your installed Cython version is compatible with the `cymem` version. If encountering this error, try recompiling your Cython project (if you're compiling `cymem` from source or an older wheel) or updating `cymem` to a version with wheels pre-built against a compatible Cython release. It's recommended to update `pip`, `setuptools`, and `wheel` before installing to ensure the latest compatible binary wheels are used.
gotcha `cymem.Pool` simplifies memory deallocation by tying it to the Python object's lifecycle. However, users must still ensure that no raw C pointers obtained from the pool outlive the Python object that owns the `Pool` instance. If the `Pool` object is garbage collected, all its managed memory is freed, invalidating any lingering pointers.
fix Carefully manage the lifetime of your `cdef` Python objects that hold `cymem.Pool` instances. Ensure that any C pointers derived from the pool are only accessed while the owning Python object and its associated `Pool` are still alive and in scope.
gotcha Attempting to use Cython-specific `cimport` syntax (e.g., `from cymem.cymem cimport Pool`) directly in a `.py` file executed by a standard Python interpreter will result in a `SyntaxError`. Cython's `cimport` is exclusively for defining types and functions within Cython source files (`.pyx`) that will be compiled, not for Python scripts.
fix If you are developing a Cython extension, ensure your source file is named `.pyx` and is compiled with Cython. If you intend to use `cymem` from Python, you typically import a *compiled* module that uses `cymem` internally, or `cymem`'s public Python API if available, using standard Python `import` statements. Do not use `cimport` in `.py` files.
breaking Attempting to use Cython's `cimport` statement directly in a standard Python (`.py`) file will result in a `SyntaxError`. `cimport` is a Cython-specific keyword valid only in Cython (`.pyx`) source files, which must be compiled into a Python-loadable extension before they can be imported into Python.
fix Use `cimport` exclusively within Cython (`.pyx`) files. To use `cymem.Pool` from a Python (`.py`) file, import it like a regular Python object (e.g., `from cymem.cymem import Pool`) after the `cymem` library (or your own Cython code that `cimports` `cymem`) has been properly compiled and installed.
python os / libc status wheel install import disk
3.10 alpine (musl) wheel - - 18.4M
3.10 alpine (musl) - - - -
3.10 slim (glibc) wheel 1.5s - 19M
3.10 slim (glibc) - - - -
3.11 alpine (musl) wheel - - 20.2M
3.11 alpine (musl) - - - -
3.11 slim (glibc) wheel 1.6s - 21M
3.11 slim (glibc) - - - -
3.12 alpine (musl) wheel - - 12.2M
3.12 alpine (musl) - - - -
3.12 slim (glibc) wheel 1.4s - 13M
3.12 slim (glibc) - - - -
3.13 alpine (musl) wheel - - 11.9M
3.13 alpine (musl) - - - -
3.13 slim (glibc) wheel 1.5s - 13M
3.13 slim (glibc) - - - -
3.9 alpine (musl) wheel - - 17.9M
3.9 alpine (musl) - - - -
3.9 slim (glibc) wheel 1.8s - 19M
3.9 slim (glibc) - - - -

This example demonstrates how to allocate C-level memory using `cymem.Pool` within a Cython `.pyx` file. The `Pool` object handles the deallocation of all memory allocated through it when the `Pool` instance itself is garbage collected, simplifying memory management in Cython extensions.

from cymem.cymem cimport Pool
from libc.stdlib cimport sizeof

def main():
    cdef Pool mem = Pool()
    cdef int* data1 = <int*>mem.alloc(10, sizeof(int))
    cdef float* data2 = <float*>mem.alloc(12, sizeof(float))

    # Use data1 and data2
    data1[0] = 100
    data2[0] = 3.14

    print(f"Data1 at index 0: {data1[0]}")
    print(f"Data2 at index 0: {data2[0]}")

    # Memory is automatically freed when 'mem' (the Pool object) is garbage collected.
    # No explicit free() calls are needed for memory allocated via Pool.

# To run this, you would typically compile it with Cython:
# cython -3 --inplace your_module.pyx
# Then import and call main() from Python:
# import your_module
# your_module.main()