Memory profiling — tracemalloc, memray, and what holds memory

Article.

Time profiling answers “what is slow”. Memory profiling answers a question people get wrong in both directions: the service’s memory keeps growing; is it a leak, and if so, what is holding it? Half the “leaks” reported are not leaks at all, and the other half are a reference someone forgot. Every number below is from Python 3.14.5.

The mechanism: when Python frees memory, and when the OS sees it

Three layers, and a different answer at each:

1. Reference counting frees an object the moment its last reference goes. Most memory is released this way, immediately.
2. The cycle collector frees objects that refer to each other and nothing else. They wait for a collection, which runs by allocation thresholds, not time.
3. The allocator (pymalloc, for objects up to 512 bytes) carves 1 MiB arenas into 16 KiB pools. A freed object goes back to its pool for reuse; an arena goes back to the operating system only when every object in it is free.

So process memory (RSS, what the dashboard shows) and Python’s live memory are different numbers, and the third layer is why.

Freed is not returned

A million small dicts, then 98% of them freed, with every fiftieth kept:

objs = [{"id": i, "name": f"user-{i}"} for i in range(1_000_000)]
keep = objs[::50]                 # 2% survive, scattered through every arena
del objs
gc.collect()

RSS: start 18 MB, after building 1M dicts 303 MB, after freeing 98% 296 MB
after freeing the last 2% too: 21 MB

Freeing 98% of the objects returned almost nothing, because the survivors were spread across nearly every arena and an arena with one live object stays. When the last 2% went, the arenas emptied and RSS fell to 21 MB. This is fragmentation, not a leak: memory is reused by the next allocations, it just does not show up as a falling graph. A leak is memory that grows under steady load and never plateaus.

Finding what grows: tracemalloc

tracemalloc is in the standard library and traces Python-level allocations by the line that made them. The leak-hunting pattern is two snapshots and a diff:

tracemalloc.start()
before = tracemalloc.take_snapshot()
serve_traffic_for_a_while()
after = tracemalloc.take_snapshot()
for stat in after.compare_to(before, "lineno")[:5]:
    print(stat)

Run against an unbounded cache:

mem.py:21: size=20.6 MiB (+20.6 MiB), count=40002 (+40002), average=539 B
mem.py:26: size=907 KiB (+907 KiB), count=20000 (+20000), average=46 B

Line 21 is _cache[key] = bytearray(1000): the cache is the growth, and the keys on line 26 are the smaller part. In a service, take snapshots minutes apart under steady traffic; what keeps growing between them is the leak. Tracing adds memory and CPU to every allocation, more with deeper stored tracebacks (tracemalloc.start(nframes)), so enable it on one instance or behind a flag, not fleet-wide.

Finding who holds it

The diff tells you which line allocated the growing objects. What keeps them alive is a reference chain from a root (a module, a running frame, a thread), and objgraph draws it:

import objgraph

objgraph.show_growth(limit=5)          # call twice; prints types that grew
leaked = objgraph.by_type("Session")[-1]
objgraph.show_backrefs([leaked], max_depth=6, filename="session.png")

The graph usually ends at something mundane: a module-level list, a class attribute used as a registry, a closure on a long-lived object. Without Graphviz, gc.get_referrers(obj) gives the same chain one hop at a time.

What tracemalloc cannot see

It traces allocations made through Python’s allocators. NumPy reports its buffers to it; most other C extensions allocate natively and are invisible. When RSS grows and tracemalloc shows nothing growing, the memory is native: an extension, a driver, or the allocator’s own retention.

The usual holders

A stored exception keeps its frames alive

errors = []
def handler():
    payload = bytearray(50 * 2**20)        # a 50 MB request body
    raise ValueError("bad input")

try:
    handler()
except ValueError as e:
    errors.append(e)                       # "keep the last errors for the status page"

3 stored exceptions: traced memory 151 MB
tracebacks dropped:  traced memory 1 MB

An exception’s __traceback__ references every frame it passed through, and each frame references its locals, so storing three exceptions kept three request bodies alive. Store the message and the type, or a formatted traceback string, never the exception object in anything long-lived.

A cycle waits for the collector

cycle unreachable, gc disabled: traced 21 MB
gc.collect() found 1 objects: traced 1 MB

An object that refers to itself (here a node with a 20 MB buffer and self.me = self) is unreachable but not freed until the cycle collector runs. Services that tune or disable the collector to save CPU (some do, to avoid pause spikes) turn every such cycle into memory that is only reclaimed eventually, or never.

And the rest

• Unbounded caches. A module-level dict or an @cache on a function whose arguments never repeat. Use @lru_cache(maxsize=N) or a TTL cache.
• Closures and callbacks. A callback registered on a long-lived object keeps everything its closure captured.
• Accumulating handlers. Logging handlers, signal subscriptions or listeners added per request and never removed.
• Per-request state on a module global. A list that “collects metrics for this request” and is never reset.

The tools in 2026

memray (Bloomberg, 1.20.0 in August 2026) records every allocation including native ones, and memray flamegraph shows where peak memory came from. memory_profiler, the line-by-line tool many older guides use, has had no release since 2022 and states it is no longer maintained.

memray run --native -o run.bin service.py
memray flamegraph run.bin            # writes an HTML flame graph of peak usage

The trade not taken: restart on a schedule

The common production answer to slow growth is a restart: gunicorn’s --max-requests, a nightly rollout, a container memory limit that kills and replaces the pod. It works, and as a bound it is reasonable, which is why max_tasks_per_child exists for process pools too.

As the only answer it hides the leak until traffic grows enough that the restart interval is shorter than a burst, and then pods die under load, which is the worst time. Take a tracemalloc diff or a memray capture from one instance first; restart limits are the safety net, not the fix.

Related

• Profiling Python — cProfile, pstats, timeit - CPU profiling, the other half
• Debugging a Live Production Incident - inspecting a live process
• weakref — References That Don't Prevent GC - references that do not keep objects alive
• Python memory model — allocation, refcounting, GC - reference counting and object layout