Async I/O and background work

Two decisions that get conflated: how to run several I/O calls concurrently inside a request, and when work should leave the request entirely.

Inside the request: concurrency

python

# Sequential - latency is the SUM
user = await users.get(uid)          # 120ms
prefs = await prefs.get(uid)         # 80ms
history = await history.get(uid)     # 200ms   -> 400ms total

# Concurrent - latency is the MAX
async with asyncio.TaskGroup() as tg:
    user_t = tg.create_task(users.get(uid))
    prefs_t = tg.create_task(prefs.get(uid))
    # -> 200ms total
    history_t = tg.create_task(history.get(uid))

Independent calls should never run sequentially. It’s the most common and most easily fixed latency problem in an async codebase, and await in a loop is the usual shape of it.

TaskGroup (3.11+) over gather: on failure it cancels siblings and raises an ExceptionGroup, so you don’t leak tasks that keep running against a service you’ve already decided is broken. See TaskGroup and Structured Concurrency.

Use gather(return_exceptions=True) deliberately when partial success is the goal — see Fallbacks, partial failure and graceful degradation.

Bound the fan-out

python

sem = asyncio.Semaphore(10)

async def fetch_one(item):
    # acquire INSIDE the task, not around creation
    async with sem:
        return await client.get(item.url)

async with asyncio.TaskGroup() as tg:
    tasks = [tg.create_task(fetch_one(i)) for i in items]   # 10_000 items is fine

Creating 10,000 tasks at once is fine — they’re cheap. Making 10,000 simultaneous HTTP calls is not: you exhaust the connection pool, trip the provider’s rate limit, and starve everything else. The semaphore must be acquired inside the coroutine; wrapping task creation instead serialises the whole thing.

Never block the event loop

One synchronous call stalls every coroutine in the process.

python

# Wrong - blocks the loop for the entire duration
result = requests.get(url)               # sync HTTP
data = pd.read_csv(huge_file)            # sync I/O + CPU
hashed = bcrypt.hashpw(pw, salt)         # CPU, ~100ms

# Right
result = await client.get(url)                        # async client
data = await asyncio.to_thread(pd.read_csv, path)     # blocking I/O -> thread
hashed = await loop.run_in_executor(process_pool, bcrypt.hashpw, pw, salt)   # CPU -> process

Work	Where
Async-capable I/O	the loop
Blocking I/O (sync driver, file, legacy SDK)	`asyncio.to_thread`
CPU-bound	process pool, or out of the service entirely

The symptom of a blocked loop is distinctive: p99 latency rises across every endpoint at once, including ones that do nothing. Unrelated endpoints degrading together is the fingerprint. See run_in_executor and asyncio.to_thread.

Leaving the request: background work

Different question. Move work out of the request when:

The user doesn’t need the result to proceed (email, webhook, indexing).
It’s slow enough to blow the latency budget.
It must survive a failure and be retried.
It’s scheduled rather than triggered.

The options, and what each guarantees

Mechanism	Survives restart	Retries	Use for
`asyncio.create_task`	no	no	truly fire-and-forget, loss acceptable
FastAPI `BackgroundTasks`	no	no	short post-response work, loss acceptable
Celery / RQ	yes	yes	real jobs
Kafka consumer	yes	yes	event-driven, high volume, ordered
Temporal	yes	yes, plus compensation	multi-step, long-running processes

The line that matters: the first two are in-process and die with it. Using BackgroundTasks to send a payment confirmation means a deploy mid-request silently loses it. If losing the work is unacceptable, it needs a broker.

python

# Acceptable: metrics, best-effort cache warm
background_tasks.add_task(track_event, user_id, "viewed")

# NOT acceptable: anything the business needs to have happened
await queue.enqueue("send_invoice", order_id=order.id)

The enqueue-after-commit trap

python

# Wrong - the worker may start before the transaction commits, or the row may never exist
await session.commit()
# if this fails, no email, no record of the failure
send_email.delay(order.id)

Two failure modes: enqueue before commit and the worker can read a row that doesn’t exist yet or gets rolled back; enqueue after commit and the enqueue itself can fail, leaving committed state with no job.

The correct fix is the transactional outbox: write the job to an outbox table in the same transaction, and let a relay publish it. Atomic by construction. See Transactional Outbox Pattern.

Workers need the same discipline

Idempotent — at-least-once delivery means every job may run twice.
Bounded — job timeouts, or one stuck job holds a worker forever.
Dead-letter queue after N failures, so a poison message doesn’t block the queue.
Monitored — queue depth and oldest-message age are the two alerts that matter.

Queue depth rising is the earliest signal that consumers can’t keep up, and it precedes user-visible failure by a comfortable margin.

Choosing

text

Does the caller need the result now?
  yes -> in-request, concurrent, bounded, with timeouts
  no  -> Is losing it acceptable?
           yes -> fire-and-forget task
           no  -> Is it multi-step with compensation?
                    yes -> workflow engine (Temporal)
                    no  -> queue (Celery / Kafka), enqueued via outbox

Interview angle 6

“Three independent API calls in one endpoint. How do you run them?” — concurrently with TaskGroup, so latency is the slowest rather than the sum, each with its own timeout under the overall budget. TaskGroup over gather because it cancels siblings on failure instead of leaking tasks.
“You need to call an API for 10,000 items. Approach?” — create the tasks freely but bound concurrency with a semaphore acquired inside each coroutine. Unbounded fan-out exhausts the connection pool and trips rate limits; wrapping task creation in the semaphore accidentally serialises everything.
“Every endpoint’s p99 went up, including ones doing nothing. Why?” — something is blocking the event loop. Find the sync call — a sync HTTP client, file I/O, or CPU work like password hashing — and move it to a thread or process.
“When do you use FastAPI BackgroundTasks versus Celery?” — BackgroundTasks runs in-process and dies with it, so only for work you can afford to lose. Anything the business needs to have happened requires a broker with durability and retries.
“You commit, then enqueue a job, and the enqueue fails. Now what?” — that’s the dual-write problem. Use a transactional outbox: write the job in the same transaction as the state change and let a relay publish it, so the two can’t diverge.
“What do you monitor on a queue?” — depth and oldest-message age. Both rise before users notice, which makes them the earliest actionable signal that consumers can’t keep up.