asyncio and threads

Article.

Both give you concurrency, neither gives you parallelism for Python bytecode, and the choice between them is almost never about speed. It is about where the switch happens — and everything else, the memory numbers, the locking, the way failures behave, follows from that one difference.

Which model a given workload wants is CPU-bound versus I/O-bound. This note is the two of them side by side.

The mechanism: who decides to switch

# threads: the interpreter can switch between ANY two bytecodes
def worker():
    total = 0
    for i in items:          # a switch can happen here,
        total += weigh(i)    # or here, or mid-expression
    return total

# asyncio: control moves only where you wrote `await`
async def worker():
    total = 0
    for i in items:
        total += await weigh(i)   # ← the only switch point
    return total

Threads are preemptive: the scheduler takes the thread away whether it is ready or not. Coroutines are cooperative: a task holds the thread until it yields, and nothing can take it.

“Any two bytecodes” is the right safety rule, but on the GIL build CPython actually hands over only at a blocking call, or after another thread has waited the 5 ms switch interval, at the next safe point in the loop. That is why a race can pass every test on the GIL build and fail constantly on the free-threaded one (The GIL, and free-threaded Python).

That is the entire difference. The rest of this note is consequences.

The consequences

Memory, and therefore scale

A thread carries an OS stack: megabytes of reserved address space, of which it touches tens of kilobytes. A coroutine is an object on the heap of about 200 bytes plus what its frame holds. Measured on CPython 3.14.5 (Windows):

Ten thousand threads will run, but they cost hundreds of megabytes, take seconds just to start, and make scheduling the bottleneck. Ten thousand coroutines is a moderately busy web server. This is why asyncio exists: the win is memory and start-up first, switch latency second.

One blocking call

async def handler():
    rows = db.query(sql)         # synchronous driver: blocks the loop
    return render(rows)

Under threads, that call blocks one thread and the other requests continue. Under asyncio it blocks every task in the process, because nothing can preempt it, and the symptom is that latency rises across endpoints that have nothing to do with this one.

It is the most common asyncio production incident, and the fix is run_in_executor and asyncio.to_thread — push the blocking call to a thread and await that.

Where races live

Both models have race conditions. They differ in where.

# Threads: this can be interrupted mid-operation.
counter += 1                 # read, add, write - three steps

# asyncio: this cannot. Nothing runs between the read and the write
# unless there is an `await` between them.
counter += 1

With threads any two bytecodes can be interrupted, so shared mutable state needs a lock by default. With asyncio the switch points are await statements, which are visible in the source — so you can read a function and see whether it is atomic.

That does not mean asyncio is race-free. An await in the middle of a read-modify-write is exactly as broken as a thread interrupt, and it is easier to miss because it looks like ordinary sequential code. asyncio.Lock exists for that case; see Synchronization primitives.

Failure behaviour

An exception in a thread kills that thread and goes to threading.excepthook (3.8), which prints it; the program carries on missing a worker. An exception in a coroutine propagates through await like an ordinary call stack, which is better; a task that fails and is never awaited keeps its exception until it is garbage-collected, when the loop’s exception handler logs “Task exception was never retrieved”, which is worse.

Structured concurrency fixes the second half by making failures propagate to the group — see TaskGroup and structured concurrency.

Cancellation and timeouts

A task can be cancelled at any await: task.cancel() or asyncio.timeout() raises CancelledError there, and finally blocks run. A thread cannot be stopped from outside at all. Future.cancel() only removes work that has not started, and a timeout around a thread only stops waiting:

async with asyncio.timeout(0.2):
    await asyncio.to_thread(blocking_call)   # TimeoutError after 0.2 s...
# ...while blocking_call keeps running in its thread, holding a pool slot

So timeouts and cancellation favour asyncio strongly: a slow dependency behind async I/O can be abandoned cleanly, while one behind a thread pool keeps consuming threads until it returns.

Debugging a stuck process

Threads are inspected with OS-level tools: py-spy dump --pid prints every thread’s stack, and faulthandler.dump_traceback() does it from inside. Tasks are invisible to those tools, which only see the loop thread. Since 3.14, python -m asyncio ps <pid> and pstree <pid> list a running process’s tasks and who awaits whom, and asyncio.print_call_graph() prints it from inside.

On the free-threaded build

On 3.14t threads also run CPU work in parallel, while an event loop still runs on one thread. That shifts the CPU-bound case towards threads; it does not change the I/O case, where asyncio’s memory and cancellation advantages remain.

The part people underrate: colour

Choosing asyncio is a choice about your whole dependency tree, not about one function.

def get_user(uid): ...          # callable from anywhere
async def get_user(uid): ...    # callable only from async code

An async function can only be awaited from another async function, so the requirement propagates up to the entry point and outward through every caller. A synchronous library — a database driver, an SDK, a parser — cannot be used directly, and you need an async equivalent or a thread to wrap it in.

Threads have no such constraint, which is why they remain the pragmatic answer for integrating with code you do not control: any library works in a thread pool, unchanged.

Bridging the two

• Sync code inside async: asyncio.to_thread(fn, *args) runs it in the default executor and awaits the result (run_in_executor and asyncio.to_thread).
• Async code inside sync: asyncio.run(coro) per call creates and closes a whole event loop each time, which is slow and breaks loop-bound clients. Keep one loop: asyncio.Runner for repeated calls from one thread, or a background loop thread fed with asyncio.run_coroutine_threadsafe(). Django’s asgiref packages both directions as sync_to_async and async_to_sync.

Where it breaks in production

An await inside a read-modify-write

The asyncio race looks like sequential code, which is why it ships:

async def deposit(amount):
    global balance
    current = balance
    await audit_log.write(amount)     # a switch point between read and write
    balance = current + amount

100 deposits of 10, await between read and write: 10
same, under asyncio.Lock:                         1000

Every task read the balance before any of them wrote it back, so 99 deposits vanished without an error. The fix is to keep the read and the write with no await between them, or hold an asyncio.Lock across the whole update; in a database, it is an atomic UPDATE ... SET balance = balance + %s.

asyncio.run per call, from sync code

Calling async code from a sync codebase with asyncio.run on each call creates and closes a loop every time. Anything bound to the first loop breaks on the second:

queue = asyncio.Queue()              # or a client with a connection pool

asyncio.run(use(queue))              # works
asyncio.run(use(queue))              # a new loop

first asyncio.run: x
second asyncio.run: RuntimeError: <Queue at 0x... maxsize=0 tasks=1>
  is bound to a different event loop

Async clients with connection pools (aiohttp, asyncpg, httpx’s async client) bind the same way, and fail as “Event loop is closed” or “attached to a different loop”. Keep one loop for the process: asyncio.Runner for repeated calls from one thread, or a background loop thread fed with run_coroutine_threadsafe.

The same dashboard, two different causes

“Latency is up” means different things in each model. With threads, look at pool queue time: requests waiting for a free thread while the threads themselves are blocked on I/O. With asyncio, look at event-loop lag: how late a scheduled callback runs, which grows when something blocks the loop. Measure the one that matches your model; a service with both has both.

The trade not taken

The tempting project is rewriting a working threaded service in asyncio “for performance”. Per request it is rarely faster: a sync Django or Flask service under gunicorn threads at a few hundred concurrent requests spends its time in the database either way.

What asyncio measurably buys is memory per connection (8 MB against 341 MB at 10,000) and cancellation, which matter at thousands of mostly idle connections: WebSockets, long polling, fan-out to slow APIs.

The rewrite costs the colour: every library, every caller, every test. So the default is threads, and asyncio where connection count or cancellation is the actual constraint, usually in a new service rather than a converted one.

Summary of the trade

Related

• CPU-bound versus I/O-bound - which model for which workload
• run_in_executor and asyncio.to_thread - bridging the two
• Concurrency and parallelism - the underlying distinction