System Design Interview — The Framework

Article.

The 45-60 minute design round has a known shape, and most candidates know it: requirements, estimate, API, data model, diagram, deep dive, trade-offs. That checklist gets a mid-level pass.

What a senior answer adds is arithmetic that changes the design (how many machines, connections and dollars), a failure walk over every box on the board, and requirements turned into mechanisms (“99.99% means automated multi-AZ failover”, not “high availability”).

The question this note answers: what do you actually say in each phase, with which numbers, so that every box on the diagram is justified? It ends with a worked 45-minute transcript. Building blocks are linked rather than repeated: Scaling building blocks, Resilience, and the finished designs in Worked designs.

The phases and the clock

Say the plan out loud in the first minute: “I’ll spend five minutes on requirements, estimate, then design; stop me if you want to go deep somewhere.” The interviewer will steer; the plan shows you can drive.

The single most common failure is reaching phase 7 with ten minutes left, because phases 1-5 were done at the same depth as the deep dive. The diagram should be on the board by minute 25.

Phase 1: requirements that constrain something

Functional: narrow the prompt to a slice that fits. “Design Instagram” is three systems; “photo upload and the home feed read path” is one.

Non-functional requirements are only useful as numbers, because each number forces a mechanism. This table is what to reach for:

Notes on the rows that interviewers probe:

• Four nines needs more than one AZ. AWS’s EC2 region-level SLA of 99.99% applies only to instances spread over two or more AZs; a single database instance cannot meet it, so the database tier needs automated failover (RDS Multi-AZ, Patroni).
• RPO with async replication is the replication lag at the moment of failure. Patroni bounds it with maximum_lag_on_failover. RPO = 0 means a synchronous standby, and every commit then waits one cross-AZ round trip, about 1 ms in the same region.
• RTO is a sum: failure detection (health checks, often 10-30 s), promotion, then every client reconnecting or a DNS TTL expiring. A 60-second DNS TTL alone spends the 1-minute budget.
• Latency budgets limit sequential hops. At a 200 ms p99, five sequential calls of 40 ms p99 each already spend it, and the tail of a chain is worse than the sum of the parts.

Phase 2: estimates that decide something

Each number should end in a decision. The method and reference figures are in Capacity Estimation — Back-of-Envelope Math and Latency Numbers and Performance Reference; the three calculations below are the ones most answers skip.

Little’s law: concurrency, not just QPS

L = λ x W: requests in flight = arrival rate x mean time in the system.

peak 3,000 req/s x 40 ms mean     = 120 requests in flight
size on p99 as a bound: x 80 ms   = 240
per instance (async, ~50 in flight at target p99) → 5-6 instances
each instance's DB pool 20 x 6    = 120 connections

That last line is the one that matters: 120 application connections against Postgres’s default max_connections = 100 is a failed deploy. The fix is PgBouncer in transaction mode in front of Postgres, with a server pool sized to the database’s cores, not the app’s concurrency. The same law sizes worker pools and Celery concurrency.

Backlog drain time = backlog / (consume rate − produce rate). A queue of 1.8 M messages, consumed at 2,500/s while 2,000/s still arrive, drains in 1.8 M / 500 = 3,600 s, an hour. That is the number to state when someone asks “what happens after the outage”.

Tail at scale

If one request fans out to N backends, the chance that at least one leg is slower than its p99 is 1 − 0.99^N:

N = 1     1%
N = 10   10%
N = 100  63%

So a page that calls 100 shards is slow most of the time even when every shard is healthy (Dean and Barroso, The Tail at Scale, 2013). The fixes are fewer legs, hedged requests after the p95, and partial results; hedging is in Timeouts, retries and backoff.

Cost, as a number

Say a monthly figure for the dominant line. Usually it is one of:

• Egress: 15 GB/s of video is about 39 PB a month; at list internet egress prices of $0.05-0.09/GB that is roughly $2-3.5 M a month, which is why the design has a CDN and why the CDN contract is the real number.
• Storage: 3 TB in Postgres on gp3 is a few hundred dollars; the same data kept 30 days as logs can cost more than the database.
• LLM tokens for AI features: requests x tokens x price per million, which often dominates compute; see ML system design framework.
• Cross-AZ traffic: charged per GB in each direction; chatty services spread over three AZs pay for every call.

Phase 3: API

A few operations, with the properties that matter:

POST /v1/webhooks/endpoints   {url, events[]}
     → 201 {endpoint_id, secret}
POST /v1/events               {type, payload}
     Idempotency-Key: …       → 202 {event_id}
GET  /v1/events?endpoint=…&cursor=…
     → 200 {items[], next_cursor}

Say why: 202 because delivery is asynchronous; an idempotency key because clients retry; cursors because offsets skip and repeat rows on a moving table. Auth and versioning go in one sentence each, unless they are the point of the question.

Phase 4: data model, checked against access patterns

Write the table and its keys, then check each access pattern against them out loud:

CREATE TABLE deliveries (
  id            uuid PRIMARY KEY DEFAULT uuidv7(),  -- PG18: time-ordered
  endpoint_id   uuid NOT NULL,
  event_id      uuid NOT NULL,
  status        text NOT NULL,          -- pending | ok | failed | dead
  attempts      int  NOT NULL DEFAULT 0,
  next_attempt  timestamptz NOT NULL,
  UNIQUE (endpoint_id, event_id)        -- one delivery per pair
);
CREATE INDEX deliveries_due ON deliveries (next_attempt)
  WHERE status IN ('pending', 'failed');

• “Find work that is due”: the partial index on next_attempt.
• “Show a merchant their deliveries”: (endpoint_id, id); uuidv7 sorts by time, so it doubles as the cursor.
• The skew check: is any key hot? One merchant producing 40% of events is a hot partition in any key-partitioned store, and a single noisy tenant in the worker pool. Name it now; it is usually the deep dive.

SQL or not falls out of this: a relational store while one primary handles the writes (a single modern Postgres node handles thousands of write transactions per second and far more reads); a key-value store when the access is single-key at a scale or latency that one primary cannot serve.

Phase 5: the diagram, and one request through it

client → LB → API (stateless) → Postgres (primary + replica)
                   │
                   └→ queue → workers → customer endpoints

Walk one request through it and say which requirement each box serves. Every box must be justified by a requirement or a number from phases 1-2; a box that is not (“and Kafka, for scale”) is a liability in the next phase.

Phase 6: the failure walk

Before the deep dive, go box by box and ask four questions. This phase is the one most candidates skip and the one that sounds most senior.

The arithmetic to say for a cache: a 99% hit rate that falls to 0% sends 100x the reads to the database. If the database was sized for the 1%, it falls over within seconds, so the answer is request coalescing, load shedding, and warming the cache before taking traffic, not “the cache is replicated”.

Breakers and bulkheads: Circuit breakers and bulkheads; fallbacks: Fallbacks, partial failure and graceful degradation.

Then say where the design loses data and how much: async replicas lose the lag; Redis with appendfsync everysec loses up to about a second; a consumer that acknowledges before processing loses the message on a crash.

Phase 7: the deep dive

The interviewer usually picks it; if not, pick the part that the requirements make hard, and say why. Go to mechanism level: data structures, the exact failure sequence, the numbers. One hard part done properly beats four done at the depth of a diagram.

Validating an estimate

If the conversation reaches “how would you know this holds 3,000 req/s”: load-test with an open model, which sends at a fixed arrival rate (k6’s constant-arrival-rate, wrk2 -R). A closed-loop tool (plain wrk, Locust users) waits for each response before sending the next, so when the server slows down the tool slows down too and the p99 looks fine. That is coordinated omission.

Phase 8: wrap-up

Three sentences: what breaks first at 10x and which metric would show it; the dominant cost line; what you deliberately left out. Naming your own design’s weak point is the clearest senior signal in the round.

A worked 45 minutes: webhook delivery

The prompt: “Design a system that delivers webhooks to our merchants’ endpoints.”

0-6, requirements. Events from internal services; deliver each to every subscribed endpoint; retries for 3 days; merchants can replay. NFRs: 50 M events a day, delivery started within 5 s at p99, at-least-once (no loss), 99.95% for accepting events. Out of scope: the merchant UI.

6-10, estimate.

50 M/day / 86,400   ≈ 580/s average, x3 peak ≈ 1,750/s
mean delivery 300 ms → in flight = 1,750 x 0.3 ≈ 525
one big merchant down, 10% of traffic, 10 s timeout:
    175/s x 10 s    = 1,750 in flight for that merchant alone
storage 50 M x 2 KB x 30 days ≈ 3 TB

The third line is the design: one broken endpoint would hold three times the normal total concurrency.

10-14, API and data: the endpoints and deliveries table above.

14-22, design: the API writes the event and its delivery rows in one transaction, then publishes (transactional outbox, so no event is lost between the database and the queue). Workers pull due deliveries, POST with an HMAC signature and a webhook-id header, and record the result.

22-26, failure walk: an endpoint slow (timeout 10 s, then backoff); an endpoint down for days (exponential backoff to 3 days, then dead, with an email); the queue down (the outbox keeps events in Postgres); a worker crash mid-POST (redelivery, so the merchant may see a duplicate, which the webhook-id lets them drop).

The deep dive and the close

26-40, deep dive on isolation, chosen because of the 1,750 in-flight number: a concurrency limit per endpoint (say 20), with deliveries beyond it left in the table rather than holding a worker; endpoints that keep failing move to a slower lane so they cannot starve healthy ones; ordering is per endpoint only where a merchant asks for it, because strict order means one stuck event blocks the rest.

40-45, wrap-up: at 10x, the deliveries_due index scan and the table’s write rate go first; partition by next_attempt day, or move scheduling to a queue with delayed delivery. Cost is small (3 TB and a worker pool); the expensive failure is support load from duplicates, hence the header.

The finished artifacts for other prompts are in Worked designs; this transcript is the conversation that produces one.

Where candidates lose the round

• A diagram at minute 3: no requirements, so no box can be justified.
• Numbers that decide nothing: 12 TB computed, then never used.
• Concurrency never computed: the pool exhausts Postgres in the deep dive, and the candidate finds out from the interviewer.
• Every component “highly available”: no failure walk, no data-loss figure, no RTO.
• Buzzword boxes: Kafka, Cassandra and Kubernetes with no requirement behind them; each becomes a question you have to defend.
• Refusing to commit: “it depends” without then choosing. State the assumption, choose, move on.
• Ignoring the steer: the interviewer asks about the feed and the candidate keeps refining auth.

The trade not taken

Starting from a known architecture (“this is basically Twitter, here is Twitter’s design”) instead of deriving from requirements. It is faster and sometimes right, and interviewers do accept naming a known pattern. It fails when the prompt has been changed on purpose (a different read:write ratio, a hard consistency requirement), and it produces a design you cannot defend box by box.

Derive first; then say which known design it resembles and where it differs.

Related

• Capacity Estimation — Back-of-Envelope Math - the estimation method and when numbers change the design
• Latency Numbers and Performance Reference - reference latencies and throughputs
• Scaling building blocks - load balancing, CDN, replication
• Resilience - timeouts, retry budgets, breakers, fallbacks
• Worked designs - finished designs for common prompts
• ML system design framework - the same round for ML systems