FastAPI and the event loop

Article.

FastAPI is an ASGI application. It does not own the event loop — Uvicorn does — and your handlers are coroutines scheduled on it. Everything that surprises people about FastAPI performance follows from that one fact, including the failure where every endpoint gets slow at once and nothing in your database looks wrong.

The mechanism: how FastAPI decides where code runs

At route registration, FastAPI inspects the handler with inspect.iscoroutinefunction — once, at import, the same moment it builds the dependency tree.

The check unwraps functools.partial, follows __wrapped__ from decorators, and looks at the __call__ of a callable class, so partial(async_fn), a correctly wrapped async decorator and a class with async def __call__ all run on the loop. A plain def wrapper around an async function does not (see the decorators note, Decorators).

@app.get("/a")
async def a():
    return await db.fetch(...)      # runs ON the event loop

@app.get("/b")
def b():
    return blocking_db.fetch(...)   # handed to a THREAD POOL

A coroutine function is awaited on the loop. A plain function is given to a worker thread so its blocking cannot stall everything else.

Three combinations, one bug

That gives three combinations, and only one of them is a bug:

@app.get("/c")
async def c():
    return requests.get(url).json()   # nothing rescues this

One slow call there freezes the entire worker, including handlers that touch nothing. The signature is p99 rising on every endpoint simultaneously while CPU sits idle and no single query looks slow — which is why it gets misdiagnosed as a network or database problem for days.

Measured: the three cases

One uvicorn worker (FastAPI 0.142.4, Starlette 1.8, uvicorn 0.54, Python 3.14.5), 50 concurrent requests to an endpoint that waits one second:

await asyncio.sleep(1) overlaps perfectly; the blocking rows use time.sleep(1). The def endpoint ran in two waves of 40 threads. asyncio.to_thread ran in four waves of 16. The blocking call in an async def ran the requests one after another.

Two thread pools, not one

The detail that turns “use def instead” into a real capacity question.

AnyIO's limiter: 40 tokens
├── every `def` handler and sync dependency
├── run_in_threadpool(...)
└── anyio.to_thread.run_sync(...)

the loop's default executor: min(32, CPUs + 4)
├── asyncio.to_thread(...)
├── loop.run_in_executor(None, ...)
└── DNS lookups (getaddrinfo)

Starlette runs sync handlers and sync dependencies through AnyIO’s limiter, 40 tokens by default. asyncio.to_thread does not use it: it goes to the event loop’s own ThreadPoolExecutor, sized from the CPU count, which is why it ran in waves of 16 above.

Forty concurrent sync requests saturate the first pool, and request forty-one waits — not for the database, for a thread. Latency climbs with concurrency, CPU is idle, and there is nothing slow to point at.

The ceiling

By Little’s law the ceiling is easy to state: 40 tokens at 0.2 s per sync request is 200 requests a second per worker, before the database pool is even considered (Concurrency and parallelism).

@asynccontextmanager
async def lifespan(app: FastAPI):
    # the limiter belongs to the running loop: at import time this raises
    # NoEventLoopError
    anyio.to_thread.current_default_thread_limiter().total_tokens = 100
    yield

Raise it only when the work is genuinely I/O-bound and the downstream can take the concurrency. More threads against a database with a 20-connection pool just moves the queue.

If you must call blocking code from an async handler, be explicit:

row = await anyio.to_thread.run_sync(blocking_db.fetch, key)

Where the concurrency actually comes from

Async does not make one request faster. It makes a waiting request cheap, so one worker serves many at once.

Two corollaries people get wrong:

• It buys nothing for CPU-bound work. A coroutine computing a hash holds the loop exactly as a blocking call does. That needs processes.
• It does not replace worker processes. uvicorn --workers 4 runs four processes, each with its own loop. Async gives concurrency within a worker; processes give parallelism across cores. You need both on a multi-core box — see CPU-bound versus I/O-bound.

Inside one handler, concurrency is explicit:

async with asyncio.TaskGroup() as tg:
    price = tg.create_task(prices.get(sku))
    stock = tg.create_task(stock_svc.get(sku))
return {"price": price.result(), "stock": stock.result()}

Total latency becomes the slower call rather than the sum, and a failure in either cancels the other — see TaskGroup and structured concurrency.

Where it breaks in production

Blocking work hidden behind an innocent import

The requests call above is easy to spot in review. These are not:

jwt.decode(token, key, algorithms=["RS256"])   # RSA verify: CPU, on the loop
bcrypt.checkpw(password, hashed)               # deliberately slow, by design
Image.open(f).resize((256, 256))               # Pillow, CPU
json.loads(payload)                            # fine until the payload is 40 MB

bcrypt is the sharpest: it is designed to take ~100ms, and 100ms on the loop is 100ms during which the worker serves nobody. A login endpoint under load takes the whole service down. Password hashing belongs in a thread.

Middleware on the hot path

BaseHTTPMiddleware is convenient and sits in front of every request. Anything blocking inside it blocks everything, and it adds measurable overhead per request because of how it wraps the ASGI call.

For cross-cutting work that must be fast, prefer pure ASGI middleware or a dependency — see Middleware.

Blocking during startup

@asynccontextmanager
async def lifespan(app: FastAPI):
    warm_the_cache()          # sync, 30 seconds
    yield

Nothing serves traffic until this returns, so a slow startup becomes failing readiness probes and a rolling deploy that stalls. Do the minimum synchronously and warm asynchronously afterwards.

Use lifespan rather than @app.on_event("startup"), which is deprecated.

Hidden consumers of the thread pool

Not only def endpoints take a token. A sync generator behind a StreamingResponse is iterated in the pool, one chunk at a time; UploadFile methods run in it; sync dependencies with yield enter and exit through it (FastAPI gives each exit its own one-slot limiter, so cleanup cannot deadlock waiting for a token held by the request it is cleaning up). A streaming endpoint with a sync generator holds a token for the whole stream.

The sync driver nobody noticed

An async def handler with SQLAlchemy’s sync engine, or psycopg2 instead of psycopg, or redis instead of redis.asyncio. The code reads as async and blocks anyway.

The check that settles it is not code review — it is a load test with concurrency well above one. If throughput does not rise with concurrency, the loop is blocked somewhere.

Diagnosing it

Add a watchdog before you need one. The loop can tell you it is blocked:

async def loop_watchdog(threshold: float = 0.25):
    while True:
        start = time.perf_counter()
        await asyncio.sleep(0.1)
        lag = time.perf_counter() - start - 0.1
        if lag > threshold:
            log.warning("event loop blocked", extra={"lag_s": round(lag, 3)})

If sleep(0.1) takes 0.9 seconds, something held the loop for 0.8 of them. That single number distinguishes “the loop is blocked” from “the dependency is slow”, which is the distinction that takes teams longest to make.

asyncio debug mode (PYTHONASYNCIODEBUG=1) logs slow callbacks and is worth enabling in staging.

The second number tells the other failure apart: thread-pool saturation. AnyIO reports it directly:

stats = anyio.to_thread.current_default_thread_limiter().statistics()
stats.borrowed_tokens, stats.tasks_waiting     # in use, and queued for a thread

Export both as gauges next to the loop lag. A blocked loop shows lag with an idle pool; a saturated pool shows tasks_waiting above zero with a healthy loop. Django has the same split: an async view that calls the sync ORM raises SynchronousOnlyOperation, and sync_to_async(thread_sensitive=True) runs such calls on a single shared thread, which serialises them (Async Django, Channels, and Async ORM).

The trade not taken

Async is not free. It removes def from your vocabulary, forces async libraries for every dependency, and makes a single mistake catastrophic rather than local — a blocking call in a threaded server slows one request, and on an event loop it slows all of them.

If your service is not I/O-concurrency-bound, sync is a legitimate choice. A FastAPI app with all-def handlers behind enough workers is straightforward, uses ordinary libraries, and fails locally rather than globally. The same is true of Flask or Django with a decent worker count.

Async earns its complexity when you hold many concurrent connections that are mostly waiting — a chat backend, an API gateway, anything streaming, anything fanning out to several services per request. Reaching for it because it is the modern default is how teams acquire the failure mode above without needing the capability.

Related

• Dependencies and injection - Depends, and sync dependencies in the same pool
• Deployment - workers, Gunicorn, and process counts
• CPU-bound versus I/O-bound - the underlying model