Async Python, end to end

Article.

The Step 5 notes take asyncio apart one mechanism at a time. This one puts it back together, for the question that tests whether the pieces connect:

“Explain what happens from async def to the socket, and what goes wrong in production.” The first half is a map through the other notes; the second half covers what they do not: async generator cleanup, context variables across tasks, eager tasks, looking inside a running loop, and uvloop. Measurements are from CPython 3.14.5 on Windows and 3.14.8 in a Linux container where noted.

The whole stack

async def handler()          defines a coroutine function
handler()                    a coroutine object: nothing runs yet
asyncio.create_task(...)     wraps it in a Task, scheduled on the loop
loop: run ready callbacks    each Task step runs until its next await
await reader.read()          not ready: the Task parks on a Future
selector / IOCP              the loop asks the OS which sockets are ready
socket readable              the Future completes, the Task is ready again

Every rule of asyncio follows from that picture. Code between two awaits runs alone, so it is atomic, and if it is slow, everything stops. A coroutine nobody awaits or wraps in a task never runs. A task nobody holds a reference to can be garbage-collected mid-flight. Cancellation is an exception thrown in at the await where the task is parked.

The map

Async generators: cleanup runs later than you think

async def rows():
    try:
        async for row in cursor:            # e.g. a database cursor
            yield row
    finally:
        await cursor.close()                # releases the connection

Leaving an async for early does not run that finally at once:

plain break:     ['after the loop', 'generator cleanup', 'later']
with aclosing:   ['generator cleanup', 'after the loop']

A synchronous generator is closed when its last reference goes. An async generator cannot be: closing it means running awaits, and that needs the loop. So asyncio schedules the cleanup as a task through the loop’s async-generator hooks, and it runs some time later, or at loop shutdown, or, if the loop is already gone, never. Meanwhile the database connection stays checked out. contextlib.aclosing() closes it at the end of the block:

from contextlib import aclosing

async with aclosing(rows()) as it:
    async for row in it:
        if done(row):
            break                           # cleanup runs here

Context variables and tasks

contextvars is how request ids, tracing spans and auth carry through async code without passing them explicitly. A task gets a copy of the context at create_task:

parent sets request_id = 'req-42', then awaits a child task
child saw 'req-42', set its own value; parent still sees 'req-42'

Values flow down into tasks, never back up. That is the right default (a child cannot corrupt its parent’s request id) and the source of a classic bug: a value set inside a task, or inside a callback run with loop.run_in_executor, is invisible to the caller. asyncio.to_thread copies the context into the thread; run_in_executor does not (run_in_executor and asyncio.to_thread).

Eager tasks

A Task normally waits for the next loop iteration before its first step. If the coroutine can finish without ever suspending (a cache hit, a validation error), that round trip is pure overhead. With the eager task factory (3.12), create_task runs the coroutine immediately until its first real suspension:

loop.set_task_factory(asyncio.eager_task_factory)
# 3.14: per task, asyncio.create_task(coro, eager_start=True)

200,000 tasks that finish without awaiting:
  default task factory:  1.335 s
  eager_task_factory:    0.587 s

The cost is ordering: an eager task’s first step runs inside create_task, before the caller continues, so code that assumed “the task has not started yet” changes behaviour, and an early exception surfaces at a different point.

Seeing inside a running loop

Debug mode

asyncio.run(main(), debug=True) or PYTHONASYNCIODEBUG=1 logs every callback slower than loop.slow_callback_duration (100 ms by default):

Executing <Task finished name='Task-1200028'
          coro=<blocking() ...> took 0.2xx seconds

That line names the coroutine that blocked the loop. Debug mode also tracks where unawaited coroutines were created. It is slow; use it in development and staging.

The mistakes asyncio reports

RuntimeWarning: coroutine 'blocking' was never awaited
Task exception was never retrieved
future: <Task finished ... coro=<boom() done ...> exception=RuntimeError('lost')>

The first is a call without await: nothing ran. The second is a task whose exception nobody read; it is only logged when the task is garbage-collected, so it can appear far from the cause, or never. Treat both as errors in tests (-W error::RuntimeWarning).

python -m asyncio ps and pstree (3.14)

Python 3.14 can read the task tree of a running process from outside, without changing its code (it uses the remote debugging support of PEP 768):

$ python -m asyncio pstree 12
└── (T) Task-1
    └──  main /w/tree.py:15
        └──  TaskGroup._aexit ...
                ├── (T) request-1
                │   └──  handle_request /w/tree.py:9
                │       ├── (T) db-1
                │       │   └──  fetch /w/tree.py:5
                │       │       └──  sleep tasks.py:712
                │       └── (T) cache-1
                └── (T) request-2 ...

ps prints the same as a table: each task, its coroutine stack and the task awaiting it. For a service that hangs with no error, this answers “what is every task waiting on” in one command; in a container it needs the SYS_PTRACE capability. Name tasks (create_task(..., name=...)) so the tree is readable.

uvloop

uvloop replaces the event loop with one built on libuv:

50 clients x 2,000 echo round trips over TCP, Linux container:
  asyncio  34,775 round trips/s
  uvloop   42,035 round trips/s

About 1.2× here, less than the two to four times often quoted: the loop’s own overhead is a small share when each request also does Python work, and asyncio’s loop has itself become faster. It helps most for very many small I/O operations with little work per message. It does not run on Windows.

Where it breaks in production

The failures in the other notes, ranked by how often they cause incidents:

1. Blocking the loop: a sync driver, time.sleep, CPU work, or a large json.loads inside async def; every request waits (18, FastAPI and the event loop).
2. Unbounded fan-out: gather over 10,000 coroutines opens 10,000 connections; use a semaphore inside the task (12).
3. Lost tasks: create_task without keeping a reference, or a failure in a background task nobody awaits (12).
4. Swallowed cancellation: except Exception or except CancelledError: pass that keeps a task running after shutdown or a timeout (13).
5. Races across await: read, await, write back (10).
6. Async generators never closed: connections held until shutdown (above).
7. Primitives bound to another loop: a lock or queue created at import and used in a second asyncio.run, typically in tests (10).

The trade not taken

asyncio buys thousands of concurrent waits on one thread at the price of “coloured” functions: every caller of an async function must be async, and one blocking call stalls everything. Threads keep ordinary code and cost a stack and a context switch each; with the GIL they are fine for I/O, and on the free-threaded build they also run CPU work in parallel (Free-threaded Python).

Most backends end up with both: an async web layer, thread pools for sync libraries, processes for CPU. The skill is knowing which part of a request runs where.

Related

• Coroutines and async def - coroutines
• Event loop internals - the loop
• TaskGroup and structured concurrency - running many tasks
• Cancellation, timeouts and error handling in asyncio - cancellation
• Free-threaded Python - the alternative for CPU work
• Two awaits for one HTTP request - what one await waits for