Backend / Async & concurrency / 04_context_switch.md

Context switching: threads vs coroutines

Updated 4 interview angles 2 min read source
On this page3
  1. Measured
  2. What each costs beyond the switch
  3. Interview angle

Context switching: threads vs coroutines

A context switch is the moment one unit of work stops running and another starts. With threads, the operating system does it: preemptively, at any instruction, saving registers and switching stacks in the kernel, and in CPython also handing over the GIL. With asyncio coroutines, the event loop does it in user space, only at an await, by resuming a different coroutine object.

The second is about ten times cheaper, and that difference is the whole argument for async I/O at high concurrency.

Measured

A thread round trip (two threading.Event hand-offs, so two switches) against a task switch (await asyncio.sleep(0) between two tasks), 20,000 iterations:

python 
async def spin():
    for _ in range(N):
        await asyncio.sleep(0)          # yield to the loop: one task switch

await asyncio.gather(spin(), spin())
text 
3.12.6  thread round trip  27.3 us   task switch 2.26 us
3.14.5  thread round trip  20.5 us   task switch 2.04 us

Even halving the thread round trip for one switch, a thread switch costs about 10 us against 2 us for a task switch, before counting the cache and TLB effects of the kernel switch, which the microbenchmark understates.

What each costs beyond the switch

  • Memory: a thread reserves a stack (8 MB virtual by default on Linux, committed as used) plus kernel structures; a coroutine is a Python object of a few kilobytes. Thousands of threads strain a machine; hundreds of thousands of coroutines do not.
  • Correctness: threads are preempted anywhere, so shared state needs locks. A coroutine runs until its next await, so code between awaits is atomic with respect to other tasks, and one coroutine that never awaits freezes everything (time.sleep in a handler is the classic bug).
  • The GIL: threads contend for it on CPU-bound work, which is why threads do not speed up pure-Python CPU work on the default build. Coroutines run in one thread and never contend, but they do not run in parallel either. The free-threaded build changes the thread side of this (Free-threaded Python).

Choosing between them is in asyncio and threads; what the loop does on each switch is in Event loop internals.

Interview angle 4

  • “What does a context switch cost?” - an OS thread switch saves and restores registers and the stack pointer, and typically invalidates cache lines and TLB entries. It’s on the order of microseconds, and the cache effects usually dominate the direct cost.
  • “Why are coroutine switches cheaper?” - they happen in user space with no kernel involvement, no privilege transition, and a much smaller saved state. That’s why a process can hold hundreds of thousands of coroutines but not hundreds of thousands of threads.
  • “What is thrashing?” - more runnable threads than cores means the scheduler spends an increasing share of time switching rather than executing. It’s why an unbounded thread pool performs worse than a sized one.
  • “Preemptive or cooperative?” - OS threads are preempted at any instruction, so shared state needs locks. Coroutines only yield at await, so you get atomicity between yield points for free, and one coroutine that never yields blocks everything.
Previous
Threads and processes