concurrent.futures

One API, three executors. ThreadPoolExecutor, ProcessPoolExecutor and, since 3.14, InterpreterPoolExecutor take the same calls — submit, map, shutdown — and hand back the same Future, so the choice between them is one line. The interview questions are about what that line hides: the GIL, pickling, and what a Future can and cannot do once it is running.

submit and Future

from concurrent.futures import ThreadPoolExecutor

def fetch(url):
    return http.get(url).json()

with ThreadPoolExecutor(max_workers=10) as pool:
    fut = pool.submit(fetch, "https://example.com")   # returns at once
    fut.running(), fut.done()                          # state, non-blocking
    data = fut.result(timeout=5)                       # blocks; re-raises

A Future is a handle to work that may not have started. result() blocks until it has finished and re-raises whatever the callable raised, so the exception surfaces where you read the result, not where it happened. exception() returns it instead of raising. The with block calls shutdown(wait=True), so nothing escapes the block still running.

add_done_callback(fn) runs fn(future) when the work finishes — in whichever thread completes the future (a pool worker, or the process pool’s manager thread), or immediately in the caller if it is already done. Anything the callback touches needs to be thread-safe, which is why callbacks are usually a queue put and nothing more.

map versus as_completed

from concurrent.futures import ThreadPoolExecutor, as_completed

with ThreadPoolExecutor(max_workers=10) as pool:
    for page in pool.map(fetch, urls):          # results in input order
        store(page)

    futs = {pool.submit(fetch, u): u for u in urls}
    for fut in as_completed(futs):              # results in finish order
        url = futs[fut]
        try:
            store(fut.result())
        except Exception as e:                  # one failure, not the batch
            log.warning("%s failed: %s", url, e)

map is the cleaner call and has three edges. It submits every item up front — a million URLs means a million queued futures, with no backpressure. It yields results in order, so one slow item stalls everything behind it. And an exception is raised from the iterator when its turn comes, ending the loop: the futures still queued are cancelled and the ones already running finish into the void.

as_completed fixes all three at the cost of bookkeeping; feeding map in slices fixes the memory one, and Python 3.14 adds buffersize= to bound the queue directly.

Threads, processes or interpreters

InterpreterPoolExecutor sits between them: one process, one interpreter per worker thread, each with its own GIL, with the callable, arguments and results pickled as for processes. On Windows a process pool is capped at 61 workers.

Measured: the same CPU work in each

Four tasks of pure-Python arithmetic, one taking 0.12 s, four workers, on CPython 3.14.5:

The choice is the workload. Threads for anything that waits on I/O, because the GIL is released while waiting; processes for CPU work that does not release it. Native extensions — NumPy, hashlib, compression — release the GIL themselves, so threads often win there too.

Python 3.14 adds a third executor, InterpreterPoolExecutor — one sub-interpreter per worker, each with its own GIL, in one process. The GIL, the free-threaded build and sub-interpreters are in The GIL, and free-threaded Python.

What a process pool demands

from concurrent.futures import ProcessPoolExecutor
from itertools import batched

def parse(chunk):                       # module-level: it has to be importable
    return [heavy(line) for line in chunk]

if __name__ == "__main__":              # spawn re-imports this module
    chunks = batched(lines, 10_000)     # amortise the per-task pickling
    with ProcessPoolExecutor(initializer=load_model) as pool:
        for rows in pool.map(parse, chunks):
            ...

Everything crosses a pipe pickled — the function by qualified name, the arguments and results by value. So lambdas, nested functions and open connections cannot be submitted; large arrays cost more to ship than to compute on; and per-task overhead means small tasks must be batched.

How much batching matters: pool.map(tiny, range(100_000)) on four processes took 48.4 s with the default chunksize=1, and 0.06 s with chunksize=1000. Every item was a round trip through a pipe; batched, a few hundred round trips carry all of them. chunksize is ignored by thread pools, where there is no pipe.

initializer runs once per worker for the expensive setup — a model, a client — that must not be re-created per task. max_tasks_per_child (3.11) replaces each worker after N tasks, which contains memory leaks in native code. When a worker dies (OOM, segfault) the pool raises BrokenProcessPool and is unusable: catch it and build a new one.

Since 3.14, terminate_workers() and kill_workers() stop a pool’s processes from outside, the one way to end work that is already running. Detail in ProcessPoolExecutor: what processes add.

Cancellation, timeouts and shutdown

with ThreadPoolExecutor() as pool:
    fut = pool.submit(slow)
    try:
        fut.result(timeout=5)
    except TimeoutError:
        fut.cancel()                 # True only if it never started
    pool.shutdown(wait=False, cancel_futures=True)   # drop the queue (3.9+)

cancel() only works on a future still in the queue; a running thread or process cannot be stopped from outside, and a timeout on result() stops your waiting, not the work. That is the honest answer to “how do you time out a task in a thread pool”: you do not — the function has to check a flag or use a socket timeout, or you run it in a process you can kill.

For fail-fast over a batch, wait returns as soon as anything fails:

futs = [pool.submit(work, i) for i in items]
done, pending = wait(futs, return_when=FIRST_EXCEPTION)   # returns at the first failure
for f in pending:
    f.cancel()              # only the ones still queued; running ones finish

From asyncio

result = await asyncio.to_thread(sync_fn, arg)             # default pool
result = await loop.run_in_executor(process_pool, cpu_fn)  # your pool

An asyncio Future and a concurrent.futures.Future are different classes; run_in_executor wraps the second in the first, and asyncio.wrap_future(f) does the same for a future you got elsewhere, so a coroutine can await it. Which pool, how to size it, and contextvars propagation are in run_in_executor and asyncio.to_thread.

Where it breaks in production

• The exception nobody reads. A future whose result() is never called swallows its exception silently. Always consume results, or attach a callback that logs.
• map over a huge input. Everything is submitted at once; memory grows with the input, not the pool.
• Blocking callbacks. add_done_callback runs in the worker; a callback that does I/O or takes a lock stalls the pool.
• A pool per request. Creating ThreadPoolExecutor() inside a handler spawns threads on every call. Make one at startup and share it.
• Process pool at import time. With spawn, the child re-imports the module and creates another pool. Guard with if __name__ == "__main__".
• Shutdown that never returns. shutdown(wait=True) waits for running work; a task stuck on a socket without a timeout holds the process open.

Related

• ProcessPoolExecutor: what processes add - pickling, spawn vs fork, sizing.
• run_in_executor and asyncio.to_thread - offloading from an event loop.
• CPU-bound versus I/O-bound - classifying the workload first.
• The GIL, and free-threaded Python - why threads do not parallelise Python bytecode.