Backend / REST APIs / 01_idempotency.md

Idempotency

Updated 5 interview angles 3 min read source
On this page4
  1. What the methods promise
  2. Making POST retryable
  3. Where else it shows up
  4. Interview angle

Idempotency

An operation is idempotent if performing it twice leaves the same server state as performing it once. It matters because a timeout tells the client nothing — the request may have succeeded, failed, or be still running — and without idempotency the only safe response to that is to give up.

REST’s other constraints are in REST principles; statelessness in Stateful and stateless.

What the methods promise

GET, HEAD, OPTIONS, PUT and DELETE are idempotent by definition; POST is not; PATCH is “neither safe nor idempotent” in RFC 5789, though a given patch can be idempotent. The full safe / idempotent / cacheable grid, as RFC 9110 defines it, is in HTTP semantics and caching.

These are contracts you uphold, not properties you get. A PUT handler that appends rather than replaces has broken the contract, and every proxy, client library and retry policy in the chain is now wrong about your API.

The distinction people miss: idempotent means the state is the same, not the response. DELETE /orders/42 returning 204 then 404 is fully idempotent — the order is gone either way.

PATCH depends entirely on the patch document. A JSON Merge Patch (RFC 7396) such as {"status": "paid"} sets values and is idempotent; a JSON Patch (RFC 6902) with an add to an array appends again on every retry. Which writes are safe to retry, and how If-Match preconditions make the rest safe, is in PUT vs PATCH: which writes are safe to retry.

Making POST retryable

The client generates a key per logical operation and sends it with every attempt of that operation:

http
POST /payments HTTP/1.1
Idempotency-Key: 9f8c2b41-0e7a-4d3f-a1e2-77b0d5c9a3ef
Content-Type: application/json

{"amount": 4200, "currency": "GBP"}

The server stores the key with the result, and a repeat returns the stored response instead of charging again.

The implementation has one trap that costs real money: the key must be claimed and committed before the external charge. If the claim and the charge share one database transaction, a crash after the provider accepted the charge rolls the claim back, and the retry charges again: measured on PostgreSQL 18, two charges instead of one.

The full pattern, with concurrency, crash recovery and retention, is in Idempotency keys.

The key must be stable across retries: generating it inside the retry loop defeats the mechanism, and this is the common client-side bug.

python
key = str(uuid.uuid4())                      # once per logical operation
for attempt in range(5):
    try:
        r = client.post("/payments", json=body, headers={"Idempotency-Key": key})
    except httpx.TransportError:
        r = None                             # network error: retry
    if r is not None and r.status_code not in (409, 429, 500, 502, 503, 504):
        break
    time.sleep(min(2 ** attempt, 30) * random.random())

Where else it shows up

The same requirement appears in every at-least-once delivery system: a message broker redelivers after a consumer crash, so consumers must be idempotent or deduplicate on a message id. See Message queues.

And in outbound calls — a retried POST to a payment provider can charge twice, which is why the retry policy in Writing a client for an external API excludes non-idempotent methods unless the provider honours a key.

Interview angle 5

  • “Which HTTP methods are idempotent?” - GET, HEAD, PUT and DELETE by definition; POST is not, and PATCH depends on the body. Idempotent means repeating the request leaves the same server state, not that the response is identical — DELETE returning 204 then 404 is still idempotent.
  • “How do you make POST safely retryable?” - a client-supplied idempotency key stored with the result. On a repeat, return the original response rather than performing the action again. The key must be stable across retries, and claimed and committed before any external side effect such as a card charge.
  • “What’s the race condition in that?” - SELECT then INSERT lets two concurrent retries both pass the check. Use a unique constraint with ON CONFLICT DO NOTHING so the database decides, and handle the in-flight case explicitly rather than falling through.
  • “Why does this matter more than it looks?” - a timeout is precisely the case where the client doesn’t know whether the operation applied. Without idempotency, the safe choice is not to retry, which means abandoning a request that probably succeeded.
  • “Where else does this apply?” - any at-least-once delivery. A broker redelivers after a consumer crash, so the consumer must be idempotent or deduplicate on a message id; the alternative is duplicated side effects on every restart.