Threading, Multiprocessing, and Concurrency

Advanced⏱90 minutes📖15 min readLesson 1 of 10

Threading, Multiprocessing, and Concurrency

Choose between threads, processes, subinterpreters and free-threaded builds, then prove the choice with a measurement instead of a guess

A nightly report takes forty minutes. The code calls a slow HTTP endpoint eight times in a loop, then spends another twenty minutes computing a score for every row. A developer wraps the whole function in a thread pool. The HTTP part gets much faster and the score part gets slightly slower, because the two halves have different bottlenecks and the same tool cannot fix both.

Concurrency advice fails when it is delivered as a slogan. "Threads share memory" and "processes avoid the GIL" are true, and neither sentence tells you what to do with a twenty-line function whose first half waits on the network and whose second half burns the CPU. The decision depends on what the work spends its time doing, how much data crosses the boundary, and how much failure you can tolerate.

This lesson builds a decision you can defend. It covers the GIL and the free-threaded build, threads with locks and queues, processes and pickling, subinterpreters on 3.14, and the measurements that separate a real speedup from a hopeful one. Every performance claim in this lesson is tied to a measurement on a named interpreter, and the code that cannot run in a browser is marked as a local listing.

ℹ️Note

Snippets that create processes, subinterpreters or free-threaded builds are shown as plain listings. The in-browser runner is a single WebAssembly interpreter with no process model, so it cannot execute them. Run those listings in a local terminal with Python 3.14 or newer.

Key terms

GIL

The global interpreter lock, a mutex that allows one thread at a time to run Python bytecode in the default CPython build.

Free-threaded build

A CPython build without the GIL, selectable at build time and identified by sysconfig.get_config_var("Py_GIL_DISABLED").

Concurrency

Making progress on several tasks during the same period, without requiring them to run at the same instant.

Parallelism

Running several tasks at the same instant on different cores or interpreters.

Race condition

A bug whose result depends on the order in which threads interleave their reads and writes.

Deadlock

A state in which two or more threads each hold a lock the other needs and none can continue.

Pickling

Serializing an object into bytes so another process can rebuild it. Process pools require picklable callables and arguments.

Start method

The rule a process pool uses to create workers: fork, spawn or forkserver.

Subinterpreter

An isolated Python interpreter inside one process, with its own modules and global state.

Executor

An object that runs submitted callables on a pool and returns Future objects.

Amdahl's law

The limit on total speedup when only part of a workload can run in parallel.

Oversubscription

Starting more worker threads or processes than the machine has cores, which adds switching cost without adding throughput.

1. What the GIL Actually Does

In the default CPython build, a thread must hold the global interpreter lock before it can execute Python bytecode. Only one thread holds it at a time, so two threads cannot run Python instructions in parallel on two cores.¹ The lock exists to protect interpreter internals, including reference counts, from concurrent mutation. It is not a lock on your data, and it does not make your own objects thread-safe.

The lock is released around blocking operations such as file reads, socket reads, time.sleep and many C extension calls. That is why a thread waiting on the network does not block the others. It is also why a CPU-bound pure Python loop gets no speedup from threads: every iteration needs the lock, so the threads take turns and pay switching costs on top.

python

import sys
import sysconfig

print("version:", sys.version.split()[0])
print("free-threading build:", "free-threading" in sys.version)
print("GIL disabled at build:", sysconfig.get_config_var("Py_GIL_DISABLED"))
print("GIL enabled now:", sys._is_gil_enabled())

On a normal 3.14 or 3.15 build the three checks report False, 0 or None, and True. On a free-threaded build the build check reports 1, and the runtime check reports False unless the GIL was re-enabled with PYTHON_GIL=1 or -X gil=1.²

The distinction between build support and runtime state matters. A free-threaded binary can run with the GIL enabled, which is useful for testing and for extensions that have not been audited. It can also have the GIL re-enabled automatically when it imports a C extension that does not declare support for the free-threaded ABI.

💡Tip

Print the build check at startup in any service that claims to use free-threading. A container image can be rebuilt from a different base and silently return to the default build.

2. Choosing a Concurrency Model

Four models cover almost every Python workload, and they fail in different ways. The table is the short version. The diagram shows the same decision as a flow.

Model	Best for	Cost	Main failure
Threads	I/O waits, blocking libraries, small shared state	Memory per thread, lock discipline	Races, deadlocks, GIL-bound CPU loops
Processes	Pure Python CPU work, isolation, crash containment	Startup, pickling, memory per process	Unpicklable work, fork plus threads
Subinterpreters	Isolated CPU work with less memory than processes	Extension support, data copying	Modules that assume one interpreter
Free-threaded threads	CPU work in a build without the GIL	Newer ecosystem, different performance profile	Extensions that re-enable the GIL

💡 Tip: Use Ctrl + Mouse Wheel to zoom, or click the fullscreen icon

Description

A decision flowchart that starts from the bottleneck, routes I/O waits to threads or asyncio, pure Python CPU work to processes or subinterpreters, and mixed work to a free-threaded build, with a measurement step after every branch.

The first question is not "which tool is fastest" but "where does the time go". A profiler answers that in minutes, and the answer often removes the need for concurrency at all. Caching a response, batching requests or replacing a quadratic loop with a dictionary lookup can beat any pool.

Matching Question

How to answer

Matching exercise with 5 pairs. Match each item on the left with its answer on the right.

Match each workload to the model that fits it best.

Downloading 200 URLs and parsing each response

Drop here

Computing a hash for 5 million rows in pure Python

Drop here

Running a pandas groupby on a large frame

Drop here

Running untrusted plugin code with separate globals

Drop here

A CPU-bound loop on a free-threaded build

Drop here

Drag answers from here:

Thread pool or asyncio

Process pool

Native code that already releases the GIL

Subinterpreters

A thread pool without the GIL

3. Threads: Shared Memory and Locks

A threading.Thread runs a callable in a new thread of the same process. All threads see the same objects, so communication is cheap and accidental sharing is easy.¹ The classic demonstration is a counter that four threads increment without a lock.

plaintext

import threading

counter = 0

def bump() -> None:
    global counter
    for _ in range(50_000):
        counter += 1
        # Yield the GIL so the interleaving is visible in a short run.
        time.sleep(0)

threads = [threading.Thread(target=bump) for _ in range(4)]
for thread in threads:
    thread.start()
for thread in threads:
    thread.join()

print(counter)  # often below 200000

counter += 1 is three operations: read the value, add one, store the result. A thread switch between the read and the store loses the other thread's update. The GIL makes each bytecode instruction atomic, not the whole statement.

The fix is a lock around the read, modify and write sequence. The lock does not make the counter faster. It makes the result correct, which is the only property that matters here.

plaintext

import threading

counter = 0
lock = threading.Lock()

def bump() -> None:
    global counter
    for _ in range(50_000):
        with lock:
            counter += 1

threads = [threading.Thread(target=bump) for _ in range(4)]
for thread in threads:
    thread.start()
for thread in threads:
    thread.join()

print(counter)  # 200000

A lock that is held while waiting on I/O serializes the whole pool and removes the benefit of threads. Keep the critical section tiny, and never call user code or a network client while holding a lock.

threading.Event coordinates a one-time signal, Condition waits for a predicate, Semaphore limits concurrent access, and Barrier makes threads wait for each other. threading.local gives each thread its own attribute namespace, which is useful for request ids and database connections. For new code that needs per-task context, contextvars is the better tool because it also works with asyncio.

python

import threading
import time

ready = threading.Event()

def worker() -> None:
    print("worker waiting")
    ready.wait()
    print("worker running")

thread = threading.Thread(target=worker)
thread.start()
time.sleep(0.1)
print("main signalling")
ready.set()
thread.join()

The output order is fixed by the event, not by timing luck: the worker prints its waiting line, the main thread signals, and only then does the worker continue.

4. Futures and Pools

Creating one thread per task does not scale. A pool keeps a fixed number of worker threads and a queue of work. ThreadPoolExecutor wraps that pattern in an executor that returns Future objects.³

python

from concurrent.futures import ThreadPoolExecutor, as_completed
import time

def fetch(name: str) -> str:
    time.sleep(0.2)
    return f"{name} done"

with ThreadPoolExecutor(max_workers=4) as pool:
    futures = [pool.submit(fetch, f"job-{index}") for index in range(8)]
    for future in as_completed(futures):
        print(future.result())

The executor starts four workers and queues the rest. as_completed yields futures in completion order, which is what you want for progress reporting. Executor.map preserves input order and is shorter when order matters.

💡 Tip: Use Ctrl + Mouse Wheel to zoom, or click the fullscreen icon

Description

A sequence diagram of a submitted task moving from the client to the executor queue, then to a worker thread, then into a Future that the client reads before shutdown.

An exception inside a worker is stored on the future and re-raised by result(). If nobody calls result(), the exception disappears, which is one of the most common reasons a pool "does nothing". Always consume every future, even when the result is not needed.

python

from concurrent.futures import ThreadPoolExecutor

def parse(value: str) -> int:
    return int(value)

with ThreadPoolExecutor(max_workers=2) as pool:
    futures = [pool.submit(parse, value) for value in ["1", "two", "3"]]
    for future in futures:
        try:
            print(future.result())
        except ValueError as error:
            print("failed:", error)

The pool keeps running after one task fails. That is usually what you want, but it also means a silent failure can leave a partial result set. Count successes and failures explicitly when the output feeds a report.

The video below demonstrates the multiprocessing side of the same pool API and shows why process pools are the tool for CPU work.

mCodingWatch on YouTube

Description

A video lesson that shows why threads do not speed up CPU-bound Python, then uses process pools to spread work across cores and explains the pickling and start method traps along the way.

5. Processes: Isolation and Pickling

ProcessPoolExecutor runs each task in a separate Python process. Each process has its own interpreter, its own GIL and its own memory, so CPU-bound Python code can use several cores.⁴ The price is that arguments and return values must cross a process boundary, which means they must be picklable.

plaintext

from concurrent.futures import ProcessPoolExecutor

def score(rows: list[int]) -> int:
    return sum(value * value for value in rows)

if __name__ == "__main__":
    chunks = [list(range(index, index + 1000)) for index in range(0, 8000, 1000)]
    with ProcessPoolExecutor(max_workers=4) as pool:
        print(sum(pool.map(score, chunks)))

The if __name__ == "__main__" guard is not decoration. With the spawn and forkserver start methods, a worker imports the main module to rebuild the function references. Without the guard, the import runs the pool creation again and the program forks itself into a loop.

Lambdas, local functions, open file handles, locks and most live objects cannot be pickled. A lambda fails with a PicklingError, and a class defined inside a function fails because the worker cannot import it by name. Move the callable to module level and pass plain data.

plaintext

from concurrent.futures import ProcessPoolExecutor

if __name__ == "__main__":
    with ProcessPoolExecutor(max_workers=2) as pool:
        # AttributeError: Can't pickle local object
        future = pool.submit(lambda value: value + 1, 1)
        print(future.result())

Python 3.14 changed the default start method on Unix platforms other than macOS from fork to forkserver. Windows and macOS keep spawn.⁵ The change reduces the chance of inheriting locks and threads from the parent, which is the classic source of hangs after a fork. Code that depends on inherited global state must now pass that state explicitly or request a fork context.

plaintext

import multiprocessing as mp
from concurrent.futures import ProcessPoolExecutor

if __name__ == "__main__":
    context = mp.get_context("fork")
    with ProcessPoolExecutor(max_workers=2, mp_context=context) as pool:
        print(pool.submit(abs, -3).result())

forkserver starts a small server process once and forks workers from it. The workers inherit a clean state, not the state of your main process. That is safer, and it also means a module-level cache populated before the pool starts is not visible inside the workers.

💡 Tip: Use Ctrl + Mouse Wheel to zoom, or click the fullscreen icon

Description

A state diagram of a worker process moving from created to bootstrapped by spawn or forkserver, then running a pickled call, then either returning a value or failing with an exception that the parent re-raises.

Process pools also gained two controls in 3.14. terminate_workers() stops workers with SIGTERM, and kill_workers() uses SIGKILL when a worker ignores the first signal.⁵ Both are useful during shutdown, and both leave the pool unusable, so they belong at the end of a failure path rather than inside normal work.

plaintext

from concurrent.futures import ProcessPoolExecutor

if __name__ == "__main__":
    pool = ProcessPoolExecutor(max_workers=2)
    futures = [pool.submit(abs, value) for value in (-1, -2)]
    print([future.result() for future in futures])
    pool.terminate_workers()

The 3.14 release also added buffersize to Executor.map. It limits how many submitted tasks can wait for a result, which keeps a fast producer from filling memory with pending work.

plaintext

from concurrent.futures import ProcessPoolExecutor

if __name__ == "__main__":
    with ProcessPoolExecutor(max_workers=4) as pool:
        for result in pool.map(abs, range(-100, 0), buffersize=8):
            print(result)

Fill in the Blanks

How to answer

Fill-in-the-blank exercise with 1 blanks. Type the missing words, then check your answer.

Complete the process pool so workers rebuild the callable safely on every platform.

if __name__ == "__main__": with ProcessPoolExecutor(max_workers=4) as pool: results = list(pool.blank(score, chunks))

Word Bank, drag words into the blanks above:

map

6. Subinterpreters: Isolation Without a Second Process

A subinterpreter is a separate Python interpreter inside one process. It has its own modules, its own builtins and its own global state, but it shares the address space and the process. Python 3.14 exposes them through concurrent.interpreters and through InterpreterPoolExecutor.⁶

plaintext

from concurrent.futures import InterpreterPoolExecutor

def square(value: int) -> int:
    return value * value

if __name__ == "__main__":
    with InterpreterPoolExecutor(max_workers=3) as pool:
        print(list(pool.map(square, range(6))))

The API looks like the other executors, and the isolation is stronger than a thread pool: a module imported in one interpreter is not shared with another. That isolation is also the catch. Each interpreter imports its own copy of the modules it uses, so a large dependency tree costs memory per worker. Extension modules must be built for multiple interpreters, and not every package on PyPI is.⁷

Data still crosses an interpreter boundary by being pickled or copied, so a subinterpreter is not a free shared-memory channel. Use it when you want isolation and cheaper startup than a process, not when you want to share a large mutable structure.

💡 Tip: Use Ctrl + Mouse Wheel to zoom, or click the fullscreen icon

Description

A class diagram of Executor with three subclasses for threads, processes and interpreters, showing submit, map and shutdown plus the extra terminate and kill methods on the process pool.

7. Free-Threaded Builds in Practice

A free-threaded build removes the GIL, so two threads can run Python bytecode at the same time.⁸ The build has existed since Python 3.13 as an experiment and has been officially supported since 3.14, following the criteria in PEP 779.⁹ It remains opt-in: the default python.org build still has the GIL.

Detecting the build is a three-line check, and the runtime state is separate from the build state.² The build variable comes from the interpreter configuration and the runtime flag comes from the sys module.¹⁰¹¹

python

import sys
import sysconfig

build_supports_free_threading = sysconfig.get_config_var("Py_GIL_DISABLED") == 1
gil_is_enabled = sys._is_gil_enabled()

print(sys.version)
print(build_supports_free_threading, gil_is_enabled)

The free-threaded build changes the performance profile of existing code. It uses biased reference counting and deferred reference counting for some object types, which reduces contention but adds bookkeeping. Quansight's one-year recap documents both the progress and the remaining work in the ecosystem, including extension support and performance regressions in some workloads.¹²

For extension authors, Python 3.15 adds abi3t, a stable ABI for free-threaded builds, so one wheel can target several free-threaded versions. The extension must use newer C APIs and an export hook instead of PyInit_, and build tools such as meson-python, scikit-build-core and Maturin can select it.¹³ Extensions outside that ABI still need a version-specific cp315t wheel, and the 3.15 release notes list the C API changes required for the new target.¹⁴

💡 Tip: Use Ctrl + Mouse Wheel to zoom, or click the fullscreen icon

Description

A timeline of free-threaded CPython from the acceptance of PEP 703 in 2023, through the opt-in build in Python 3.13 and official support in 3.14, to the abi3t stable ABI in Python 3.15.

Do not treat the free-threaded build as a drop-in upgrade. Measure the same workload on both builds, check that every C extension declares support, and watch memory as well as time. The build is a different runtime, not a flag that makes existing code faster.

⚠️Warning

A free-threaded process can re-enable the GIL automatically when it imports an extension that does not support free-threading. Check sys._is_gil_enabled() after imports, not only at startup.

8. Measuring the Choice

A concurrency change is an experiment. Pick a representative workload, run it with one worker and with several, and record wall clock time, CPU time and peak memory. The numbers below come from a local CPython 3.14.4 on an eight-core Linux machine, with eight tasks, a 0.2 second sleep per I/O task and a 3 million iteration pure Python loop per CPU task.

plaintext

import time
from concurrent.futures import ProcessPoolExecutor, ThreadPoolExecutor

TASKS = 8
WORK = 3_000_000

def cpu(n: int) -> int:
    return sum(i * i for i in range(n))

def io_task(_: int) -> int:
    time.sleep(0.2)
    return 1

def run(fn, args, pool):
    start = time.perf_counter()
    with pool as executor:
        list(executor.map(fn, args))
    return time.perf_counter() - start

if __name__ == "__main__":
    print("cpu threads", run(cpu, [WORK] * TASKS, ThreadPoolExecutor(TASKS)))
    print("cpu processes", run(cpu, [WORK] * TASKS, ProcessPoolExecutor(TASKS)))
    print("io threads", run(io_task, range(TASKS), ThreadPoolExecutor(TASKS)))
    print("io processes", run(io_task, range(TASKS), ProcessPoolExecutor(TASKS)))

Workload	Sequential	Threads	Processes
CPU-bound, 8 tasks	1.70 s	3.15 s	0.78 s
I/O-bound, 8 tasks	1.60 s	0.21 s	0.27 s

The CPU row is the important one. Threads are slower than sequential because they contend for the GIL and add switching overhead. Processes are about twice as fast as sequential on this small workload, and the gap grows as the work per task grows, until process startup and pickling dominate.

Wall clock time for eight tasks by concurrency model

Description

Bar chart of wall clock seconds for eight tasks on CPython 3.14.4 on an eight core Linux machine. For CPU bound work, sequential takes 1.70 seconds, threads take 3.15 seconds and processes take 0.78 seconds. For I/O bound work, sequential takes 1.60 seconds, threads take 0.21 seconds and processes take 0.27 seconds.

Amdahl's law explains the ceiling. If 80 percent of a workload can run in parallel and 20 percent must stay serial, the best possible speedup is five times, no matter how many workers you add.³ Measure the serial fraction before buying more cores.

python

def amdahl(parallel_fraction: float, workers: int) -> float:
    serial = 1 - parallel_fraction
    return 1 / (serial + parallel_fraction / workers)

for workers in (1, 2, 4, 8, 16):
    print(workers, round(amdahl(0.8, workers), 2))

The output approaches 5.0 and never reaches it. A pool with 16 workers buys almost nothing over 8 when the serial fraction is 0.2, and it costs memory and scheduling overhead.

9. Failure Modes and Debugging

Concurrency bugs are hard to reproduce because they depend on timing. The table maps the common symptom to the tool that finds it.

Symptom	Likely cause	Tool
Pool never finishes	Deadlock between locks, or a worker waiting on the parent	`faulthandler.dump_traceback_later`
Results are lower than expected	Lost updates from unsynchronized read and write	Code review plus a lock
`PicklingError`	Lambda, local class or open handle crossing a process boundary	Move the callable to module level
Program forks forever	Missing `if __name__ == "__main__"` guard	Add the guard
CPU at 100 percent with no progress	Busy wait, or oversubscription	`py-spy top` and core count
Memory grows per task	Unbounded queue or a future kept in a list	Bounded queue and `buffersize`

faulthandler writes the traceback of every thread after a timeout, which turns a hang into evidence.¹⁵ py-spy dump attaches to a running process without restarting it and prints the stack of every thread, including native frames.¹⁶

plaintext

import faulthandler

faulthandler.dump_traceback_later(10, exit=True)
# start the pool here

The timeout is a diagnostic, not a fix. If the dump shows two threads each waiting for a lock the other holds, the fix is a lock ordering rule: every thread acquires locks in the same global order, and no thread holds two locks when it can avoid it.

plaintext

import threading

first = threading.Lock()
second = threading.Lock()

def path_a() -> None:
    with first:
        with second:
            print("a")

def path_b() -> None:
    with first:          # same order as path_a
        with second:
            print("b")

Ordering Question

How to answer

Ordering exercise with 6 items. Move them into the correct order, then check your answer.

Order the steps of a concurrency investigation.

Profile the single-threaded version and identify the bottleneck

Estimate the serial fraction and the best possible speedup

Choose a model from the bottleneck, not from fashion

Run the same workload with one worker and with several

Record wall clock, CPU time and peak memory

Keep the change only if the gain justifies the added failure modes

10. What Changed in 2026

Change	Version	Why it matters
Free-threading is officially supported	3.14	PEP 779 set the criteria; the build is still opt-in
`forkserver` is the default on Unix outside macOS	3.14	Safer workers, but inherited global state no longer works
`InterpreterPoolExecutor` and `concurrent.interpreters`	3.14	Isolated parallel work with less memory than processes
`terminate_workers()` and `kill_workers()`	3.14	Explicit shutdown for stuck pools
`Executor.map(buffersize=...)`	3.14	Bounded in-flight work
`abi3t` stable ABI for free-threaded builds	3.15	One wheel for several free-threaded versions
macOS installers ship free-threading support by default	3.15	Easier local testing on macOS

The ecosystem moves more slowly than the interpreter. Cython documents free-threading support with cython.critical_section and keeps the existing with gil and with nogil semantics, but extension coverage still varies by project. Check the wheel tags and the project's release notes before promising a free-threaded deployment.

11. Case Studies

Case study: Quansight's free-threaded recap

Quansight published a one-year recap of the free-threaded build after 3.13. It documents performance work, extension porting and the remaining gaps, and it is useful precisely because it separates "the interpreter works" from "the ecosystem works".¹² The lesson for a team is to pilot the build on a real dependency set, not on a microbenchmark.

What to copy: measure both builds with the same workload and record memory as well as time. What to avoid: assuming that a pure Python benchmark predicts a service that imports twenty C extensions.

Case study: PyTorch DataLoader workers

PyTorch's DataLoader uses worker processes to overlap data loading with model training. The documentation exposes num_workers, prefetch_factor, persistent_workers and worker initialization hooks.¹⁷ Each knob trades memory and startup cost for throughput, and the right values depend on the storage system and the transform cost.

What to copy: the explicit prefetch and persistence controls, which prevent a worker from being recreated for every batch. What to avoid: raising num_workers without measuring, because too many workers contend for the same disk and add memory without adding throughput.

Case study: joblib and scikit-learn

joblib.Parallel gives scikit-learn a single API for process and thread backends, selected with prefer="processes" or prefer="threads".¹⁸ The library chooses processes by default for CPU work and threads when the backend releases the GIL. The choice is visible in the API instead of hidden in a global setting, which is a good design to copy.

What to copy: make the backend a parameter, document the default and log the effective choice. What to avoid: a hidden global that silently changes the backend in production.