mprofile: A Low-Overhead Memory Profiler

0.0.15 · active · verified Thu Apr 16

mprofile is a low-overhead sampling memory profiler for Python, derived from heapprof, with an interface similar to tracemalloc. It attempts to give results comparable to tracemalloc, but uses statistical sampling to lower memory and CPU overhead. The sampling algorithm is the one used by tcmalloc and Golang heap profilers. The current version is 0.0.15, and it is compatible with Python >= 3.4.

Common errors

```
DeprecationWarning: Using or importing the ABCs from 'collections' instead of from 'collections.abc' is deprecated
```
cause Running mprofile on Python 3.8+ with a version of mprofile older than 0.0.13.

fix Upgrade mprofile to version 0.0.13 or newer: `pip install --upgrade mprofile`.
```
Segmentation fault (core dumped)
```
cause This error specifically occurred on Ubuntu 20.04 with mprofile versions older than 0.0.12.

fix Upgrade mprofile to version 0.0.12 or newer: `pip install --upgrade mprofile`.
```
ModuleNotFoundError: No module named 'mprofile'
```
cause The mprofile package is not installed in the current Python environment.

fix Install the package using pip: `pip install mprofile`.

Warnings

gotcha mprofile is compatible with Python >= 3.4. Using it with earlier versions of Python (< 3.4) requires building CPython from source and manually applying `pytracemalloc` patches, which is a complex setup.
Fix: Use Python 3.4 or newer, or follow the complex manual build process for older Python versions.
gotcha mprofile uses statistical sampling for memory profiling, not comprehensive tracing like `tracemalloc`. This design choice prioritizes low overhead, meaning the reported memory usage is an estimate and may not be an exact, byte-for-byte measurement. Interpret results with this in mind.
Fix: Understand that mprofile provides sampled estimates, which are generally sufficient for identifying memory hotspots but may not be suitable for precise memory accounting.
deprecated Versions of mprofile prior to 0.0.13 caused a `DeprecationWarning` in Python 3.8+ due to changes in the `collections.abc` module.
Fix: Upgrade to mprofile version 0.0.13 or newer (`pip install --upgrade mprofile`).
gotcha Versions of mprofile prior to 0.0.12 were known to cause segmentation faults when used on Ubuntu 20.04.
Fix: Upgrade to mprofile version 0.0.12 or newer (`pip install --upgrade mprofile`).

Install

pip install mprofile Install mprofile via pip

Imports

mprofile
```
import mprofile
```

Quickstart

This quickstart demonstrates how to initialize `mprofile`, simulate some memory allocation, take a snapshot of the current memory usage, and print the top memory-consuming lines. The `sample_rate` is set to the recommended 128KB for a balance between overhead and precision.

import mprofile
import os

# Start profiling with a recommended sample rate (128KB) for low overhead
mprofile.start(sample_rate=128 * 1024)

# Simulate some memory allocation
data = []
for i in range(100_000):
    data.append(os.urandom(100)) # Allocate 100 bytes per iteration

# Take a snapshot of memory usage
snapshot = mprofile.take_snapshot()

# Print top 5 memory consuming lines and their statistics
print("Top 5 memory consuming lines:")
for stat in snapshot.statistics(key_type='lineno')[:5]:
    print(stat)

# Stop profiling
mprofile.stop()

view raw JSON →