Memory Management and Garbage Collection

Advanced⏱75 minutes📖16 min readLesson 6 of 10

Memory Management and Garbage Collection

Understand reference counting, the generational collector and the 3.14 revert, measure real object costs, and find leaks with tracemalloc, weak references and memray

A worker process grows by forty megabytes per hour and is restarted every night to stay inside its container limit. The team adds memory, then restarts every six hours, then upgrades the container again. The actual bug is a dictionary that maps request ids to response objects and never removes an entry. No garbage collector can free those objects, because the dictionary still references them.

Memory problems in Python are rarely about the collector. They are about ownership: something still holds a reference, or something was copied when a view would do, or the process is not leaking at all but has a high-water mark that never comes down. The collector handles cycles, and cycles are a small part of the story.

This lesson builds the full picture. It starts with reference counting and the moment an object is freed, then the generational collector and the unusual history of the incremental collector that shipped in 3.14.0 to 3.14.4 and was reverted in 3.14.5. It measures the real cost of common representations, shows how accidental cycles appear, covers weak references and caches, and ends with the tools that find a leak in a running service.

📌Important

Python 3.14.0 through 3.14.4 shipped an incremental garbage collector. It was reverted in 3.14.5 because of reports of significant memory pressure in production, and Python 3.15 uses the generational collector from 3.13. Advice that describes the incremental collector as the current behaviour is out of date.¹²

Key terms

Reference count

A counter on every object that records how many references point to it.

Cyclic garbage

Objects that reference each other and would never reach zero references without a cycle detector.

Generation

A group of objects the collector tracks together, based on how many collections they have survived.

Young generation

Generation 0, where new objects start and where most objects die.

Full collection

A collection that scans every generation, including long-lived objects.

High-water mark

The maximum resident memory a process has reached, which the operating system may not return after the peak.

Fragmentation

Free memory that exists in pieces too small to satisfy the next allocation.

Weak reference

A reference that does not increase the reference count and becomes dead when the object is freed.

Shallow size

The memory of one object without the objects it references.

Deep size

The memory of an object plus everything reachable from it, counted once.

Resident set size

The physical memory a process currently occupies, reported by the operating system.

Immortal object

An object whose reference count never reaches zero, such as None and small integers since 3.12.

1. Where the Memory Lives

CPython allocates every object on the heap. The variable in a function is a name bound to a pointer, and the object itself carries a header with a type pointer and a reference count. Small objects come from arenas managed by pymalloc, which is why the process can hold memory after the objects are gone.

💡 Tip: Use Ctrl + Mouse Wheel to zoom, or click the fullscreen icon

Description

A flowchart of an assignment from a local name to a heap object, showing the stack frame holding a pointer, the object header with type and reference count, and the payload.

Two consequences follow. First, assignment is cheap because it copies a pointer and increments a counter. Second, deleting a name does not necessarily free memory, because another reference may exist and because the allocator keeps the arena for reuse.

python

import sys

value = [1, 2, 3]
alias = value

print(sys.getrefcount(value))
del alias
print(sys.getrefcount(value))

sys.getrefcount returns one more than the number of real references, because the argument itself is a temporary reference. That off-by-one is documented and is the reason the numbers look odd in examples.³

2. Reference Counting

Every assignment, argument pass, container insertion and attribute store increments a count. Every rebinding, deletion, scope exit and container removal decrements it. When the count reaches zero, CPython deallocates the object immediately, which is why a file or socket can close as soon as the last reference disappears.

python

import sys

class Resource:
    def __init__(self, name):
        self.name = name

    def __del__(self):
        print("freeing", self.name)

resource = Resource("db")
print(sys.getrefcount(resource))
copy = resource
print(sys.getrefcount(resource))
del copy
print(sys.getrefcount(resource))
del resource

The final del prints freeing db before the next statement. A __del__ method is not a destructor contract in the C++ sense, and it must not be used for critical cleanup. Use a context manager, try/finally, or weakref.finalize instead, because the interpreter may not call __del__ at shutdown and because objects in cycles need the collector.

Reference counting is deterministic and local, and it has two costs. Every assignment touches a counter, which is why free-threaded builds need atomic or biased counters, and cycles never reach zero without help.

💡 Tip: Use Ctrl + Mouse Wheel to zoom, or click the fullscreen icon

Description

A sequence diagram of reference counting, where an assignment increments the count, a second name increments it again, deleting one name decrements it, and deleting the last name triggers deallocation.

3. Cycles and the Generational Collector

Two objects that point at each other keep each other alive. Their reference counts never reach zero, so reference counting alone would leak them. The cyclic garbage collector finds these groups by tracking container objects and occasionally walking their references.⁴

python

import gc
import sys

class Node:
    pass

first = Node()
second = Node()
first.peer = second
second.peer = first

print(sys.getrefcount(first))
del first, second
print("collected", gc.collect())

The collector sorts objects into three generations. New objects start in generation 0. Objects that survive a collection move to generation 1, and objects that survive again move to generation 2. A generation 0 collection is cheap because it only scans young objects, and a full collection is expensive because it scans everything.

💡 Tip: Use Ctrl + Mouse Wheel to zoom, or click the fullscreen icon

Description

A state diagram of object generations, where new objects enter generation zero, survivors move to generation one and then two, and each collection scans the young generation plus a portion of the older ones.

The thresholds control how often each generation is scanned. gc.get_threshold returns three numbers, and gc.set_threshold changes them. The defaults changed across versions, so read the value instead of hard-coding it.

python

import gc

print(gc.get_threshold())
print(gc.get_count())
print(gc.get_stats())

On the measured interpreters, CPython 3.11 returned (700, 10, 10) and CPython 3.15.0rc2 returned (2000, 10, 10). Code that assumes 700 is measuring a different collector from the one that runs in production.

gc.collect() with no argument runs a full collection and returns the number of unreachable objects it found. gc.collect(0) scans only the young generation, and gc.collect(1) scans generations 0 and 1. A manual full collection in a request path is almost always a mistake, because it scans the whole heap and adds latency to one request.

gc.freeze() moves every tracked object into a permanent generation that is never scanned again. Called before forking worker processes, it prevents the collector from touching pages that would otherwise stay shared, which reduces copy-on-write memory in prefork servers.

python

import gc

gc.collect()
gc.freeze()
print(gc.get_freeze_count())

4. The 3.14 Collector That Was Reverted

Python 3.14.0 shipped an incremental collector with two generations, young and old. Instead of collecting whole generations, it collected the young generation and an increment of the old generation, which reduced maximum pause times for large heaps. The meaning of the threshold tuple changed with it, and gc.collect(1) performed an increment rather than a generation 1 collection. The release notes for 3.14 preserve both descriptions.¹

Production feedback changed the decision. Teams reported significant memory pressure, and the collector was reverted to the generational design from 3.13 in 3.14.5. Python 3.15 uses that generational collector, and the 3.15 release notes state the revert explicitly.²

The practical lessons are worth keeping:

A collector tuned for pause time can raise memory use, because delayed collections mean more unreachable objects stay resident.
Threshold values are not portable across versions. Read them at startup.
A behaviour change in a patch release can still be large enough to revert, so pin the interpreter version in production and read the changelog.

python

import gc
import sys

print(sys.version.split()[0])
print(gc.get_threshold())
print([stat.get("duration") for stat in gc.get_stats()])

On 3.15 the per-generation statistics gained duration and candidates, so a service can log how long each collection took and how many objects it considered.⁴ That is cheaper than guessing from a latency graph.

⚠️Warning

Do not disable the collector with gc.disable() to "improve performance" unless you have measured the pause cost and you have a plan for cycles. A long-running process with cycles and no collector grows without bound.

The video below walks through the collector and the reference-counting model from the CPython source, which makes the generation diagram concrete.

mCodingWatch on YouTube

Description

A video explanation of reference counting and the cyclic garbage collector in CPython, including generations, thresholds and the cost of a full collection.

5. Measuring Memory

Four numbers describe memory, and using the wrong one produces the wrong conclusion.

Tool	What it answers	Limitation
`sys.getsizeof`	Shallow size of one object	Ignores referenced objects
`tracemalloc`	Python allocations with tracebacks	Ignores native allocations
`resource.getrusage`	Peak resident set size	One number, no attribution
`memray`	Python and native allocations over time	Needs a run or an attach

sys.getsizeof is the shallow size. A list of a million integers reports about eight megabytes, because the list itself is an array of pointers, and the integer objects are separate.³

python

import sys

print(sys.getsizeof(0))
print(sys.getsizeof(1.5))
print(sys.getsizeof(""))
print(sys.getsizeof([]))
print(sys.getsizeof({}))
print(sys.getsizeof(list(range(10))))

On 64-bit CPython 3.14 the measured values were 28 bytes for an integer, 24 for a float, 41 for an empty string, 56 for an empty list and 64 for an empty dictionary. The exact numbers depend on the build and the pointer size, so measure on the target platform.

tracemalloc tracks Python allocations and records where they happened. Start it early in the process, take a snapshot at a stable point, take another after the suspected leak, and compare. The diff shows which line allocated the memory that is still alive.⁵

python

import tracemalloc

tracemalloc.start()

first = tracemalloc.take_snapshot()

cache = {}
for index in range(100_000):
    cache[index] = str(index)

second = tracemalloc.take_snapshot()
for stat in second.compare_to(first, "lineno")[:5]:
    print(stat)

current, peak = tracemalloc.get_traced_memory()
print(round(current / 1e6, 2), round(peak / 1e6, 2))
tracemalloc.stop()

resource.getrusage(resource.RUSAGE_SELF).ru_maxrss reports the peak resident set on Linux in kilobytes, and on macOS in bytes.⁶ It is the number the container limit enforces, and it never decreases, which is why a process can look like it leaks while it is only holding its high-water mark.

python

import resource

usage = resource.getrusage(resource.RUSAGE_SELF)
print(round(usage.ru_maxrss / 1024, 1), "MB peak")

memray records both Python and native allocations and writes a file that can be viewed as a flame graph, a summary or a table. It can attach to a running process, which makes it the tool for a service that only leaks in production.⁷

bash

memray run -o output.bin script.py
memray flamegraph output.bin
memray summary output.bin
memray attach 12345

6. Object Size and Layout

The biggest memory wins are representation changes, not collector tuning. A container of boxed Python objects costs a pointer plus one object per element, while a packed array costs only the element.

python

import array
import tracemalloc

def measure(build):
    tracemalloc.start()
    value = build()
    current, peak = tracemalloc.get_traced_memory()
    tracemalloc.stop()
    return round(current / 1e6, 2), value

print(measure(lambda: list(range(1_000_000)))[0])
print(measure(lambda: [0] * 1_000_000)[0])
print(measure(lambda: array.array("i", range(1_000_000)))[0])

The measured values for one million elements were 40.0 megabytes for list(range(...)), 8.0 megabytes for [0] * 1_000_000, and 4.1 megabytes for array.array("i", ...). The second case is small because every slot points at the same cached zero, which is a useful reminder that a list stores pointers.

Memory for one million values by representation

Description

Bar chart of megabytes for one million values on CPython 3.14. A list of distinct integers takes 40 megabytes, a list of zeros takes 8 megabytes, an array of 32 bit integers takes 4.09 megabytes, a NumPy int32 array takes 4 megabytes and a bytearray takes 1 megabyte.

array.array stores machine numbers in a contiguous buffer and supports the buffer protocol, so a memoryview or a NumPy array can read it without a copy.⁸ A bytearray is the same idea for bytes, and a memoryview is a typed window over any buffer.

python

import array

values = array.array("i", range(10))
view = memoryview(values)
print(view[0], view.nbytes)
print(view[0:4].tolist())

An instance with __slots__ avoids the per-instance dictionary. On the measured build a plain instance was 48 bytes plus a 296-byte dictionary, while a slotted instance was 48 bytes total. With a million live instances that is roughly 300 megabytes saved.

python

import sys

class WithDict:
    pass

class WithSlots:
    __slots__ = ("x", "y")

plain = WithDict()
plain.x = 1
plain.y = 2

slotted = WithSlots()
slotted.x = 1
slotted.y = 2

print(sys.getsizeof(plain), sys.getsizeof(plain.__dict__))
print(sys.getsizeof(slotted))

dataclass(slots=True) generates the same layout without hand-written slots. The previous lesson on classes covers the rules, and the memory rule is simple: one dict per instance is expensive when there are many instances.

7. Copies, Views and Accidental Retention

Slicing a list copies the pointers, not the objects, so the copy costs eight bytes per slot. Slicing a NumPy array creates a view that shares the buffer, which is cheap and surprising when a write through the view changes the original.

python

import copy

rows = [[1, 2], [3, 4]]
shallow = copy.copy(rows)
deep = copy.deepcopy(rows)

shallow[0].append(9)
print(rows)
print(deep)

A shallow copy shares the inner lists, so appending through the copy changes the original. deepcopy recursively copies and uses a memo dictionary to preserve shared structure and cycles.⁹ Deep copying a large object graph is expensive and often a sign that the design shares mutable state it should not.

Common retention bugs:

A module-level dictionary used as a cache with no eviction policy.
A closure that captures a large object and outlives the request.
An exception traceback stored in a log record, which keeps every frame alive.
A bound method stored in a list, which keeps its instance alive.
A generator that is never exhausted and holds its frame.

python

import traceback

saved = []

def fail():
    big = [0] * 1_000_000
    raise ValueError("boom")

try:
    fail()
except ValueError as error:
    saved.append(error.__traceback__)

The traceback holds the frame, the frame holds big, and the list holds the traceback. The million-element list stays alive long after the exception was handled. Log the message and the stack text, then drop the exception object.

8. Reference Cycles You Create by Accident

Cycles are easy to create without noticing. A parent that stores children and a child that stores the parent is the classic case. A closure that captures a function which captures the closure is another. A cache keyed by an object that also references the cache is a third.

python

class Parent:
    def __init__(self):
        self.children = []

class Child:
    def __init__(self, parent):
        self.parent = parent
        parent.children.append(self)

parent = Parent()
child = Child(parent)
print(child.parent is parent, parent.children[0] is child)

The pair is unreachable from module scope after the names are deleted, and the collector frees it. The trouble starts when one of the objects is still reachable from a global registry, because then the collector correctly keeps the whole group alive.

Weak references break the cycle without a collector pass. A weakref.ref does not increment the count, and it returns None after the target is freed.¹⁰

python

import weakref

class Child:
    def __init__(self, parent):
        self.parent = weakref.ref(parent)

class Parent:
    def __init__(self):
        self.children = []

parent = Parent()
child = Child(parent)
parent.children.append(child)
print(child.parent() is parent)
del parent
print(child.parent())

weakref.WeakValueDictionary and WeakSet build caches that do not keep their values alive. A WeakKeyDictionary attaches metadata to an object without changing its lifetime.

python

import weakref

class Session:
    pass

cache = weakref.WeakValueDictionary()
session = Session()
cache["current"] = session
print(len(cache))
del session
print(len(cache))

Not every object supports weak references. Instances of classes with __slots__ need a __weakref__ slot, and built-in types such as int and str do not support them at all. The documentation lists the exact rules.¹⁰

💡 Tip: Use Ctrl + Mouse Wheel to zoom, or click the fullscreen icon

Description

A class diagram of a strong parent to child reference and a weak child to parent reference, showing that the weak reference does not keep the parent alive when the last strong reference disappears.

weakref.finalize registers a callback that runs when an object is collected, which is the correct replacement for most __del__ methods. The callback must not capture the object, or it creates the cycle it was meant to avoid.

python

import weakref

class Connection:
    pass

def close(name):
    print("closing", name)

connection = Connection()
finalizer = weakref.finalize(connection, close, "db")
del connection
print(finalizer.alive)

Fill in the Blanks

How to answer

Fill-in-the-blank exercise with 1 blanks. Type the missing words, then check your answer.

Complete the cache that does not keep its values alive.

import weakref cache = weakref.blank() cache["session"] = session

Word Bank, drag words into the blanks above:

WeakValueDictionary

9. Free-Threading and Reference Counts

The default CPython build uses a single reference count per object. The free-threaded build cannot update one shared counter from several threads without atomics on every assignment, so it uses biased reference counting: an object owned by one thread uses a plain counter, and shared objects use atomic operations. Some object types use deferred reference counting instead.¹¹

The tradeoff is memory and complexity. Biased counters and deferred processing add fields and bookkeeping, and the Quansight recap documents both the progress and the remaining performance and memory costs across the ecosystem.¹² A service that moves to a free-threaded build should measure resident memory as carefully as it measures throughput.

Immortal objects reduce counter traffic for values that never die. Since 3.12, objects such as None, True, False and small integers carry a fixed reference count that is never modified.¹³ That is why sys.getrefcount(None) returns a large constant instead of a small number.

python

import sys

print(sys.getrefcount(None))
print(sys.getrefcount(1))
print(sys.getrefcount(()))

On the measured 3.14 build all three returned 3221225472, the immortal sentinel value. A normal list returned 2, which is one real reference plus the temporary argument.

10. Diagnosing a Leak in Production

Leaks in a service have a signature. Resident memory grows with traffic and does not fall after a deploy, while tracemalloc shows a growing Python allocation count. If the Python number is flat and the resident number grows, the leak is in native code or in fragmentation.

Start with a baseline snapshot after warm-up, then take snapshots on a schedule. Compare each snapshot to the baseline and group by line or by traceback. The allocation that grows fastest is the candidate.

python

import tracemalloc

tracemalloc.start(25)

baseline = tracemalloc.take_snapshot()

def handle_request(request_id):
    return {"id": request_id, "payload": "x" * 1000}

registry = {}
for request_id in range(10_000):
    registry[request_id] = handle_request(request_id)

current = tracemalloc.take_snapshot()
for stat in current.compare_to(baseline, "traceback")[:3]:
    print(stat)

If the leak is native, memray is the tool that sees it. Run the service under memray run, or attach to a live process with memray attach, then read the flame graph from the root down to the allocating frame.⁷

💡 Tip: Use Ctrl + Mouse Wheel to zoom, or click the fullscreen icon

Description

A mind map of memory diagnosis, branching into Python leaks measured with tracemalloc, native leaks measured with memray, high water marks measured with resource, cycles inspected with gc, and layout problems fixed with slots and arrays.

A useful decision rule: if tracemalloc grows, fix ownership. If only resident memory grows, look for native buffers, fragmentation, or a cache outside the Python heap. If both are flat and the container still restarts the process, the limit is set below the legitimate working set.

11. Failure Modes

Symptom	Likely cause	Fix
Memory grows with traffic	Unbounded cache or registry	Add a size limit and an eviction policy
Memory spikes then stays high	High-water mark held by the allocator	Reuse buffers, or accept the peak and size the limit
Objects with `__del__` never freed	Cycle that keeps the group alive	Use a context manager or `weakref.finalize`
`gc.collect()` is slow	Large heap and full collection	Reduce cycles, call `gc.freeze()` before fork
Free-threaded build uses more memory	Biased and deferred reference counting	Measure the build, do not assume parity
`getsizeof` says the list is small	It reports shallow size	Use `tracemalloc` or a deep-size tool
Resident memory falls after restart only	Native leak or fragmentation	Profile with memray
Process killed by the OOM killer	Limit below the real working set	Raise the limit or stream the data

Ordering Question

How to answer

Ordering exercise with 6 items. Move them into the correct order, then check your answer.

Order the steps of investigating a slow memory leak.

Record resident memory and the container limit over time

Take a tracemalloc baseline after warm-up

Compare later snapshots by traceback and find the growing allocation

Check whether the growth is Python memory or native memory

Attach memray if the Python heap is flat

Fix the owner of the reference and add a regression test

12. Case Studies

Case study: memray at Bloomberg

Bloomberg built memray to track allocations in Python programs, including allocations made by native extensions, and to present them as flame graphs and summary tables.⁷ It can attach to a running process, which matters for services where the leak appears only under production traffic.

What to copy: record allocations over time, not just a single snapshot, and read the flame graph from the root to the allocating frame. What to avoid: running the tracer in production without a sampling or duration limit, because the recording file grows with every allocation.

Case study: tracemalloc in the standard library

tracemalloc answers a narrower question than memray: which Python line allocated memory that is still alive. Its snapshot comparison is the cheapest way to confirm a suspected leak in a test or a staging run.⁵

What to copy: start tracing before the workload, keep a baseline snapshot, and compare by traceback rather than by file. What to avoid: leaving tracemalloc enabled forever with a large traceback depth, because it retains frames and adds overhead.

Case study: reference counting in the free-threaded build

The free-threaded build replaces the single shared counter with biased reference counting and defers some work, and Quansight's recap documents the memory and performance effects across real packages.¹² The result is that a free-threaded process can use more memory for the same objects, even when it runs faster.

What to copy: measure resident memory, allocation rate and pause behaviour alongside throughput when evaluating the build. What to avoid: assuming that removing the GIL is free, because the counter machinery has a cost that shows up in memory and in single-threaded speed.

13. 2026 Updates (as of November 2026)

Python 3.15.2 is the current stable release, and 3.15 uses the generational collector from 3.13 after the 3.14 experiment was reverted.²
The 3.14 release notes keep both descriptions: the incremental collector in 3.14.0 to 3.14.4 and the revert in 3.14.5 and later.¹
gc.get_stats() gained duration and candidates in 3.15, so collection cost can be logged per generation.⁴
Immortal objects keep their fixed reference count since 3.12, which reduces counter traffic for None, booleans and small integers.¹³
The free-threaded build still uses biased and deferred reference counting, and its memory profile differs from the default build.¹¹¹²
memray remains the practical tool for native allocation tracking, and tracemalloc remains the standard library option for Python allocations.⁷⁵

📚 Did you know?

sys.getrefcount(None) returns 3221225472 on a current build because None is immortal and its counter is never updated.¹³
gc.freeze() before forking moves objects to a permanent generation, which keeps their pages shared between workers and reduces copy-on-write memory.⁴
A list of a million zeros uses 8 megabytes while a list of a million distinct integers uses about 40, because the zeros are one shared object.³