Memory Profiler

Warnings

• deprecated The `memory-profiler` package is no longer actively maintained by its original developers. While functional, new features or active issue resolution are unlikely.
  Fix: For new projects, consider exploring more actively maintained memory profiling alternatives (e.g., `memray`, `fil-profiler`). For existing projects, be aware that support might be limited.
• gotcha When using the `mprof run` command-line utility for time-based memory usage with timestamps, importing `@profile` directly in your Python script (`from memory_profiler import profile`) will prevent timestamps from being recorded.
  Fix: Comment out the `from memory_profiler import profile` statement when running with `mprof run` to ensure timestamps are captured, leaving only the `@profile` decorator on functions.
• gotcha Using `memory-profiler` introduces overhead and consumes additional memory itself. It is not recommended for production environments due to its intrusive nature.
  Fix: Use `memory-profiler` exclusively for development and debugging. For production memory monitoring, consider less intrusive system-level tools or Python's built-in `tracemalloc` for lower overhead.
• gotcha By default, `memory-profiler` only tracks the memory usage of the parent process. Memory consumed by forked child processes (e.g., when using `multiprocessing`) is not automatically included in the report.
  Fix: Use the `mprof run --include-children` or `--multiprocess` flags to track child processes. When using the `memory_usage` API directly, the return value will include child memory in a nested list that needs to be manually processed.
• gotcha Applying the `@profile` decorator can sometimes subtly change the behavior of the decorated function, particularly when interacting with complex data structures or libraries like Pandas, potentially leading to unexpected errors or different outputs than when run un-profiled.
  Fix: If encountering unusual behavior when profiling, test the function without the `@profile` decorator. If the issue resolves, consider using the `memory_usage` function for profiling specific code blocks, or explore non-intrusive profilers like `fil-profiler`.

Install

• pip install -U memory-profiler Install latest version

Imports

• profile

  from memory_profiler import profile

  When using `mprof run` to collect time-based memory usage with timestamps, do NOT import `profile` within your script. Only apply the `@profile` decorator to functions you want to profile. Importing `profile` will prevent timestamps from being recorded by `mprof`.
• memory_usage

  from memory_profiler import memory_usage

Quickstart

To perform a line-by-line memory usage analysis, decorate the function you want to profile with `@profile`. Then, run your script using the `python -m memory_profiler` command. The output will be printed to standard output, showing memory usage and increments for each line within the decorated function.

import time
from memory_profiler import profile

# Save this as 'my_script.py'

@profile
def create_large_lists():
    a = [0] * (10 ** 6)  # Allocates ~8MB
    time.sleep(0.1)
    b = [1] * (2 * 10 ** 7) # Allocates ~160MB
    time.sleep(0.1)
    c = [2] * (5 * 10 ** 6)  # Allocates ~40MB
    del b # Frees ~160MB
    return a, c

if __name__ == '__main__':
    print("Starting memory intensive task...")
    lists = create_large_lists()
    print("Task completed.")
    # The profiling output will be printed to stdout when run via python -m memory_profiler