CPU-bound versus I/O-bound

Article.

“Threads for I/O, processes for CPU” is the answer everyone gives, and it is right for the wrong reason unless you can say what the GIL actually locks — and what changed in Python 3.14.

The mechanism, briefly

A thread must hold the GIL to run Python bytecode. It lets go around blocking I/O (socket and file reads, time.sleep) and inside C routines written to release it, and otherwise hands it over every 5 ms at most (The GIL, and free-threaded Python). So on the default build:

Measured on CPython 3.14.5, 4 threads on a 12-thread Windows machine: one pure-Python task took 324 ms, four on four threads took 1,259 ms, exactly the serial time.

Classify by measurement, not by intuition

Whether a task is CPU-bound is a measurement: CPU time against wall time.

import time

def cpu_ratio(fn):
    cpu, wall = time.process_time(), time.perf_counter()
    fn()
    return (time.process_time() - cpu) / (time.perf_counter() - wall)

cpu_ratio(lambda: sum(i * i for i in range(3_000_000)))   # 0.97: CPU-bound
cpu_ratio(lambda: time.sleep(0.5))                        # 0.00: waiting

Near 1.0 per busy thread means the CPU is the bottleneck; near 0 means the task waits. In production the same signal comes from py-spy top (with --idle to include waiting threads), from CPU time vs request time in your metrics, or from a process pinned at 100% of one core while latency grows.

“I/O” that is really CPU work

Much of what a web service calls I/O happens after the bytes arrive, under the GIL. Whether threads help depends on whether the C routine releases it:

Parsing a large JSON body, validating it with Pydantic or rendering a big template is CPU work on the event loop or the thread, even though it sits “in the I/O path”. Hashing and compression of large buffers parallelise on threads because those modules drop the GIL around the work.

A third class: contention

Some workloads are bound by neither: they wait on a lock, on the GIL itself (many CPU threads handing it around), or on memory bandwidth. The signature is throughput that stays flat as you add workers while CPU is not saturated per worker. py-spy top --gil shows GIL contention; lock contention shows up as threads parked in acquire.

The exception people miss

“CPU-bound means no threads” is a rule about Python bytecode. NumPy, pandas, Pillow and most compiled numeric libraries release the GIL inside their C routines:

# This genuinely parallelises across cores despite the GIL.
with ThreadPoolExecutor(4) as ex:
    ex.map(lambda a: np.linalg.svd(a), matrices)

The bytecode holds the GIL only long enough to enter the C function. If your CPU work is inside a library like that, threads are correct and processes are needless overhead — worth checking before reaching for multiprocessing.

What changed in 3.14

Free-threading is officially supported as of Python 3.14 (PEP 779), no longer experimental. It is a separate build — python3.14t — with the GIL compiled out, so threads execute bytecode genuinely in parallel.

That does not make it the default answer:

• It is opt-in. Standard CPython still has the GIL, so “we’ll just use threads” assumes a build most deployments are not running.
• Single-threaded code pays. Removing the GIL requires other synchronisation, which costs some single-thread performance — substantially reduced since the 3.13 experimental builds, but not free.
• C extensions must support it. A module built without free-threading support will re-enable the GIL at import, silently undoing the reason you chose the build.

The honest interview position: know it is official, know it is a separate build, and do not present it as the default. For most services in 2026, the GIL is still the operative constraint — and being precise about that is worth more than enthusiasm.

3.14 also brought subinterpreters into the stdlib (PEP 734), giving each interpreter its own GIL in one process. Cheaper than processes, more isolated than threads, and awkward to share data across — a real fourth option worth naming.

The models, with their costs

Measured on Windows, where processes start with spawn and re-import the main module; fork and forkserver on Linux are far cheaper to start. Holding 10,000 idle connections: 10,000 threads added 341 MB of resident memory, 10,000 asyncio tasks 8 MB. Per-unit costs, isolation and failure modes are compared in Threads and processes.

asyncio is the answer for high-concurrency I/O: one thread, one loop, tasks that cost microseconds and kilobytes, so tens of thousands of open connections are ordinary. Its constraint is that it is cooperative — a task that does not await blocks everything.

Threads are the answer for I/O against libraries with no async support, and for CPU work inside GIL-releasing C extensions. Thousands are fine; tens of thousands cost hundreds of megabytes and make GIL hand-offs and scheduling the bottleneck.

Processes are the answer for genuine Python CPU work on the default build. Real parallelism, at the price of start-up, memory per process and moving data across the boundary. Subinterpreters sit between threads and processes, and free-threaded threads are the direction of travel where dependencies allow (The GIL, and free-threaded Python).

Where it breaks in production

One blocking call stops the whole loop

The most common async bug, and the mechanism makes it obvious: an event loop is a single thread, so anything that does not yield blocks every task on it.

@app.get("/report")
async def report():
    rows = db.execute(HEAVY_QUERY)     # sync driver: blocks the loop
    return render(rows)

Under load this looks like latency rising with concurrency while CPU sits idle, and it is routinely misdiagnosed as a slow dependency. The fix is an async driver, or asyncio.to_thread to move it off the loop — see run_in_executor and asyncio.to_thread.

Pickling costs more than the work

Every argument and return value crossing a process boundary is serialised.

def total(xs):                 # module level: a lambda cannot be pickled at all
    return sum(xs)

data = list(range(5_000_000))
total(data)                                    # 30 ms inline
with ProcessPoolExecutor(1) as ex:
    ex.submit(total, data).result()            # 1,160 ms: 25 MB pickled across

Sending a large object to a worker and getting a number back cost forty times the work itself. (Pass a lambda instead and it fails before any data moves, with PicklingError.) Send references, not payloads — a file path, an object key, a row range — and let the worker load what it needs.

Thread pools sized by guesswork

ThreadPoolExecutor() with no argument uses min(32, (os.process_cpu_count() or 1) + 4) workers: 16 on a 12-thread machine. That is the wrong basis for I/O work, where threads wait rather than compute; size the pool from concurrency and latency (Little’s law in Concurrency and parallelism).

In a container, process_cpu_count() follows CPU affinity, not the CPU quota, so set PYTHON_CPU_COUNT (or -X cpu_count, 3.13) to override it: with 2, the default pool drops to 6. Every blocking call from async code also shares the loop’s default executor unless you supply your own.

fork plus threads

On Linux, multiprocessing historically forked, and forking a process that holds locks in threads produces a child with locks held by threads that do not exist. The symptom is a deadlock that reproduces once a week. Python 3.14 defaults to forkserver on Unix other than macOS, which removes the sharpest edge.

Code that relied on fork (inheriting module state in the children) must opt back in with mp_context=multiprocessing.get_context("fork"); since 3.12, os.fork() in a process with running threads emits a DeprecationWarning, which is the signal to look for.

Choosing

Is it I/O-bound?
├─ yes ─ async libraries?      yes -> asyncio
│                              no  -> threads
└─ no ── in GIL-releasing C?   yes -> threads
         free-threaded build,
         deps ready, audited?  yes -> threads (3.14t)
         shared-nothing work,
         pure-Python deps?     yes -> subinterpreters
         otherwise                  -> processes

And before any of it: is it actually slow, and is it actually the bottleneck? Concurrency is often reached for where the fix is an index, a cache, or not doing the work per request.

The trade not taken

The model nobody argues for is one process doing one thing at a time, run more times. Four single-threaded workers behind a load balancer parallelise as well as four threads, with no shared state, no GIL question, no pickling, and a crash that takes out one request instead of all of them.

That is what a WSGI or ASGI deployment already is, and it means a great many “we need concurrency” conversations are really about a worker count in a config file. Reach for in-process concurrency when work must share memory or when connection counts make a process-per-connection model absurd — which is exactly where asyncio earns its place.

Related

• The GIL, and free-threaded Python - the GIL in detail
• TaskGroup and structured concurrency - running async work correctly
• When to use processes - the process side