The allocator: heap, arenas, pools, blocks

Garbage collection decides when memory is reclaimed; the allocator decides where objects live and when memory goes back to the operating system. CPython does not call C malloc for every small object, which would be slow and fragment the heap. It ships its own small-object allocator, pymalloc (Objects/obmalloc.c).

Its design explains the question interviewers actually ask: why a Python process does not shrink after you free a lot of objects. Numbers below are from Python 3.14.5 on 64-bit Windows.

When objects are freed is in Python memory model — allocation, refcounting, GC.

Four layers

A request tries its type’s free list first; failing that, pymalloc if it is small; otherwise malloc. PEP 445 lets each layer’s allocator be replaced or wrapped, which is how tracemalloc and the debug hooks work. The free-threaded build uses mimalloc instead of pymalloc, because pymalloc assumes the GIL.

The hierarchy

Before 3.10 arenas were 256 KiB with 4 KiB pools, and much older material also quotes 8-byte alignment; 64-bit builds have aligned blocks to 16 bytes since 3.8. Every request is rounded up to the next multiple of 16, giving 32 size classes:

$ python -c "import sys; sys._debugmallocstats()"
Small block threshold = 512, in 32 size classes.
class   size   num pools   blocks in use  avail blocks
    0     16           1              31           990
    1     32           2             733           287
    ...
2 arenas * 1048576 bytes/arena     =            2,097,152

Because every block in a pool has the same size, allocation never searches for a hole that fits.

Allocating and freeing a small object

Each size class keeps a list of partially used pools. Allocation takes a block from one; freeing pushes it onto that pool’s free list. The freed block stores the pointer to the next free block in its own memory, so both operations are O(1) with no side bookkeeping.

a = object()          # 16 bytes: one block in a 16-byte class pool
addr = id(a)
del a                 # the block goes back on its pool's free list
b = object()
id(b) == addr         # often True: the freed block is reused (not guaranteed)

A pool is used (partly full), full, or empty; an empty pool can be reassigned to any size class.

Why memory does not go back to the OS

An arena is returned to the operating system only when all 64 of its pools are empty. One live 16-byte object keeps a whole 1 MiB arena. CPython allocates from the most heavily used arenas first, so that lightly used ones can drain and be released, but long-lived survivors scattered across arenas defeat it:

objs = [object() for _ in range(5_000_000)]    # RSS 21 MB -> 141 MB
keep = objs[::1000]                            # one survivor per 1000
del objs; gc.collect()                         # RSS 101 MB, not ~21

Freeing 99.9% of the objects gave back less than a third of the memory, because a survivor in nearly every pool kept nearly every arena. The process’s size is set by its peak usage and by how survivors are spread, not by what is live now. Objects above 512 bytes go through malloc, and whether that memory returns to the OS is the C library’s decision (glibc often keeps it).

Per-type free lists

The hottest types skip even pymalloc. Floats, tuples (per length), lists, dicts and some internal objects keep lists of recently freed instances to reuse, and the set of types and limits changes between versions (3.11 to 3.13 reworked several). Small ints from -5 to 256 are not a free list at all: they are preallocated and immortal since 3.12.

Inspecting it

import sys, tracemalloc

sys._debugmallocstats()          # arenas, pools and blocks per size class
tracemalloc.start()
...                              # run the workload
for stat in tracemalloc.take_snapshot().statistics("lineno")[:5]:
    print(stat)                  # the top allocation sites by size

PYTHONMALLOC=debug turns on the debug hooks (buffer overruns, use-after-free in C extensions), PYTHONMALLOC=malloc bypasses pymalloc so tools like Valgrind see every allocation, and PYTHONMALLOCSTATS=1 prints the statistics at exit (Memory profiling — tracemalloc, memray, and what holds memory).

Where it breaks in production

• A worker that never shrinks. A request that built a huge object graph raises RSS to its peak, and fragmentation keeps it there. The fixes are architectural: stream instead of materialising, do the big job in a subprocess that exits, or recycle workers (max_requests in gunicorn, --max-tasks-per-child in Celery).
• Memory “leaks” that are fragmentation. RSS stays flat-high while tracemalloc shows live allocations falling. Compare the two before hunting a leak.
• Native extensions. NumPy arrays and most C-extension buffers come from malloc, not pymalloc; tracemalloc sees them only if the extension uses the Python allocator APIs.

Related

• Python memory model — allocation, refcounting, GC - reference counting and the cycle collector
• Stack and heap in Python - frames and references
• __slots__ — when it saves memory and what it breaks - smaller instances
• dict internals and list internals - how containers size themselves
• Memory profiling — tracemalloc, memray, and what holds memory - tracemalloc and finding leaks
• Python memory in production - memory in production: measuring, leaks, plateaus, fork, containers