Short Polling vs Long Polling

Article.

Both are the client asking “anything new?”. They differ in when the server answers. A short poll is answered at once and the client asks again on a timer. A long poll is parked until there is something to send, or until a hold timer set below every proxy timeout on the path runs out, and the client asks again immediately. SSE and WebSockets answer the same question by keeping a connection open, so this is one decision with four options.

The defaults in 2026: SSE for server-to-browser push, WebSockets when the client also talks continuously, long polling when the network path breaks streaming or the consumer is a script or a worker, and short polling when the data is shared and cacheable, or a few seconds of lag cost nothing.

How each one works on the wire

Short polling

Every T seconds the client sends an independent request and the server answers with whatever it has, usually nothing. An event waits for the next tick, so delivery latency is spread evenly between 0 and T: T/2 on average, T at worst.

client                         server
  |-- GET /events?since=7 ------->|
  |<- 200 {"events": []} ---------|  nothing new
  |                               |
  |      ...sleep T...            |<- event 8 published
  |                               |
  |-- GET /events?since=7 ------->|
  |<- 200 {"events": [8]} --------|  up to T late

Long polling

The client sends the same kind of request and the server parks it. In an async server the handler awaits a condition, which costs a suspended coroutine and a socket, not a thread. A publish wakes it and the answer leaves at once. If nothing happens before the hold expires, the server answers 204 and the client asks again with the same cursor.

client                         server
  |-- GET /poll?since=7 --------->|
  |                               |  parked, awaiting
  |                               |  a condition
  |                               |<- event 8 published
  |<- 200 {"events": [8]} --------|  one trip later
  |-- GET /poll?since=8 --------->|  re-poll at once
  |                               |
  |      ...25 s, nothing...      |
  |<- 204 No Content -------------|  hold expired
  |-- GET /poll?since=8 --------->|  same cursor

SSE and WebSockets, for contrast

Both hold one connection and push each event down it, so there is no request per event at all. SSE is one HTTP response streamed for hours; the browser reconnects on its own and sends Last-Event-ID. A WebSocket upgrades the connection and carries frames both ways. The depth is in Server-Sent Events (SSE) and The WebSocket protocol.

The decision

All four need a cursor to be correct across gaps. Polls carry it in the URL, SSE carries it in Last-Event-ID, and a WebSocket carries whatever your own protocol defines.

Latency and cost, with numbers

100,000 clients, one event each per minute

Short polling at T = 5 s is 12 requests per client per minute, and with one event a minute 11 of them come back empty. Each carries 0.5-2 KB of headers and cookies, so at 1 KB that is about 20 MB/s (160 Mbit/s) of mostly empty traffic, plus a TLS record, a load-balancer charge, an auth check and usually a cache or database read, 20,000 times a second.

Long polling’s idle ceiling is 100,000 / 25 = 4,000 re-polls a second, plus one request per delivered batch. A poll answered by an event restarts its hold, so the mean cycle is 60 x (1 - e^(-25/60)), about 20.4 s: 4,900 requests a second, roughly 1,700 of them carrying data. The price is 100,000 parked requests, which is cheap or impossible depending on the server.

On phones the cost is battery. Each request wakes the cellular radio, which stays in its high-power state for a tail of ten seconds or more, so a 5-second poll means it never sleeps. Background mobile delivery belongs to APNs and FCM, which share one connection across every app on the device.

What a parked request costs the server

In asyncio a parked long poll is a suspended coroutine awaiting a condition, plus its socket and buffers: tens of kilobytes and no thread. One process holds thousands, and 100,000 parked clients cost a few gigabytes across a fleet.

A thread-per-request server is a different animal. Under gunicorn’s default sync workers each parked poll occupies a whole worker, so N waiting clients need N workers: two parked polls on two workers leave the load balancer’s health check queued behind them, and a hold past gunicorn’s 30-second timeout gets the worker killed mid-request with WORKER TIMEOUT.

Threaded workers (gthread, WSGI thread pools) pin a thread per poll instead, which moves the ceiling without removing it. Long polling is an async-server technique.

A long-poll endpoint in FastAPI

The cursor is since, the id of the last event the client processed, and the retention window is the last 1,000 events. The short-poll endpoint further down lives in the same module.

import asyncio
import time
from collections import deque

from fastapi import FastAPI, Request, Response
from fastapi.responses import JSONResponse

app = FastAPI()
log: deque[dict] = deque(maxlen=1000)  # retention: the last 1,000 events
changed = asyncio.Condition()           # woken on every publish
MAX_HOLD = 25.0                         # below every timeout on the path

def last_id() -> int:
    return log[-1]["id"] if log else 0

def after(since: int) -> list[dict]:
    return [e for e in log if e["id"] > since]

@app.post("/publish")
async def publish(data: str) -> dict:
    async with changed:
        event = {"id": last_id() + 1, "ts": time.time(), "data": data}
        log.append(event)
        changed.notify_all()
    return {"id": event["id"]}

The poll handler

@app.get("/poll")  # long poll: park until there is something after the cursor
async def poll(since: int = 0, hold: float = MAX_HOLD) -> Response:
    if since > last_id() or (log and since < log[0]["id"] - 1):
        return Response(status_code=410)   # cursor outside retention: resync
    async with changed:
        try:
            async with asyncio.timeout(min(hold, MAX_HOLD)):
                await changed.wait_for(lambda: last_id() > since)
        except TimeoutError:
            return Response(status_code=204)   # nothing new: ask again, same cursor
    return JSONResponse({"events": after(since), "next": last_id()})

What the endpoint encodes

1. Await a notification; never sleep and re-query. A loop of await asyncio.sleep(1) plus a query inside the held request is short polling moved to the server: up to a second of added latency and one query per second per parked client.
2. Hold below every timeout on the path, enforced server-side. The client cannot see the path. 25 s clears the 30 s and 60 s defaults tabled below.
3. 204 on timeout, same cursor. Timing out loses nothing.
4. Everything after the cursor, not the next event. wait_for checks the predicate before it waits, so a client that is behind is answered at once and a burst arrives as one batch.
5. 410 for a cursor outside retention: older than the window, or ahead of a log that restarted, as this in-memory one does. The client resyncs.

The client

import asyncio
import random

import httpx


async def follow(c: httpx.AsyncClient, handle, resync, cursor: int = 0) -> None:
    failures = 0
    while True:
        try:
            r = await c.get("/poll", params={"since": cursor}, timeout=35)
            if r.status_code == 410:          # cursor fell out of retention
                cursor = await resync(c)      # reload full state, take its cursor
                continue
            r.raise_for_status()
        except httpx.HTTPError:
            failures += 1                     # full jitter, capped at 30 s
            await asyncio.sleep(random.uniform(0, min(30, 2**failures)))
            continue
        failures = 0
        if r.status_code == 200:              # 204: hold expired, same cursor
            body = r.json()
            await handle(body["events"])      # idempotent: dedupe by event id
            cursor = body["next"]

The client timeout (35 s) must exceed the hold; httpx defaults to 5 seconds, which turns every quiet long poll into a ReadTimeout. Errors back off with jitter, because a loop that re-polls instantly on errors turns one failing endpoint into a request storm from every client at once.

The missed-event window

The bug long polling invites is waiting for “the next event”:

@app.get("/next")  # WRONG: no cursor, waits for whatever is published next
async def next_event() -> dict:
    async with changed:
        await changed.wait()
    return log[-1]

An event published while the client is between requests - handling the last batch, on the network, reconnecting - finds nothing parked to receive it, and is gone. Bursts make it certain: the first event releases the poll, and the ones milliseconds behind it land in the gap. Every client spends part of each cycle between requests, so the loss is structural, not a race to tune away.

The fix has three parts:

1. The client sends the id of the last event it processed, and the server returns everything after it.
2. The server keeps a retention window and answers 410 when a cursor falls out of it, and the client reloads full state.
3. The client dedupes by event id. Delivery is at-least-once: a response lost in transit is redelivered by the retry carrying the old cursor. Ids are monotonic, so order survives reconnects too.

The price is a little latency on bursts: a follower waits for the previous batch to be handled and the re-poll to land.

Gotcha: a cursor is only as good as the order the log becomes visible in. A Postgres bigserial id is allocated at insert but the row appears at commit, so id 41 can commit after a reader has moved past 42, and the cursor skips it. Use a log ordered at append (a single writer, a Redis Stream entry id, a Kafka offset), or re-read a short overlap window and dedupe.

Across processes

An asyncio.Condition lives in one process. Behind uvicorn --workers 4 or three pods, a publish handled by one process wakes nothing parked in the others. The endpoint stays; what moves is where the log and the wake-up live:

1. The log goes to shared storage (a table or a Redis Stream), which enforces the retention window.
2. Each process runs one subscriber (Redis pub/sub or Postgres LISTEN/NOTIFY) that calls notify_all() locally: a doorbell, not a mailbox.
3. A woken or timed-out handler re-reads the log after its cursor.

Pub/sub is at-most-once, and that is fine here: a lost doorbell costs one hold of latency, never an event, because the next poll reads the log. Send the id, not the event; a NOTIFY payload must stay under 8,000 bytes by default. Redis’s XREAD BLOCK <ms> STREAMS key <last-id> is a long poll with a cursor, but one blocking read per parked request is one Redis connection each, against a default maxclients of 10,000.

Making short polling cheap

Short polling costs per request, so make each request cheap and send fewer. This endpoint answers an unchanged feed with a bodiless 304:

@app.get("/events")  # short poll: answer now, 304 if nothing changed
async def events(request: Request, since: int = 0) -> Response:
    etag = f'"{last_id()}"'
    if request.headers.get("if-none-match") == etag:
        return Response(status_code=304, headers={"ETag": etag})
    body = {"events": after(since), "next": last_id()}
    return JSONResponse(body, headers={"ETag": etag})

• 304 via ETag. GitHub’s Events API works this way, and its 304s do not count against the rate limit. With Cache-Control: no-cache and an ETag, a browser revalidates with If-None-Match on its own and hands fetch the cached body on a 304.
• A shared feed at the CDN. When every client sees the same data (scores, a status page, prices), Cache-Control: public, max-age=2 lets the edge answer, and the origin sees about one request per TTL per edge location whatever the client count. Never for per-user data: that is private (HTTP semantics and caching).
• A server-driven interval. GitHub sends X-Poll-Interval and raises it under load; Retry-After on a 429 or 503 does the same while shedding.
• Jitter, around 20% either way. Clients that started together, after a deploy or a reload storm, otherwise poll together forever.
• Hidden tabs. Slow down while document.visibilityState is hidden and refresh on visibilitychange. Chrome throttles hidden tabs’ timers anyway: to once a second, and chained timers to once a minute after five minutes.

Proxies and load balancers set the hold

A long poll is one slow response, and every hop between client and app has an opinion about slow responses. Defaults as of 2026-10:

Hold below the smallest limit on the path, with margin, and set it on the server. The folk 25-30 seconds come from these defaults. Engine.IO lands in the same place: its server pings every 25 s, and under the polling transport a ping is a packet like any other, so a parked GET is answered at least that often.

Raising a limit to fit the hold is legitimate (ALB up to 4,000 s, nginx per location, API Gateway past 29 s for Regional REST APIs since 2024, Cloudflare only on Enterprise), but every hop needs it, and the next team to add a hop will not know. Cloudflare’s limit was long quoted as 100 s; it is 125 s now.

504 after exactly 60 seconds, never locally. The number names the hop. Locally there is no proxy; in production the ALB idle timeout or nginx’s proxy_read_timeout cut a hold of 60 s or more. 30 s points at a GCP backend service, 29 s at API Gateway, 15 s at Envoy, and a 524 at Cloudflare.

The browser has a limit too. Over HTTP/1.1 a browser opens at most six connections per origin, shared by every tab, so six tabs with a parked poll each leave a seventh unable to load anything from that origin. HTTP/2 carries the polls as streams on one connection (servers usually allow 100 or more), or one elected tab polls and shares results over BroadcastChannel.

Where it breaks in production

Deploys release every parked poll at once

On SIGTERM uvicorn stops accepting connections and waits for in-flight requests, and every parked poll is in flight. By default it waits as long as that takes, so each instance needs a full hold to drain, and a rolling deploy pays that per batch. A --timeout-graceful-shutdown shorter than the hold cancels the parked handlers instead, and each client gets a 500.

Either way the clients all re-poll within moments of each other, into whichever instances remain. Set the graceful timeout above the hold, randomize the hold per request (Consul adds up to wait / 16 to its blocking queries for exactly this), and make clients back off with jitter after a 5xx.

Dashboards and alerts

Every quiet poll is a 25-second “slow request”. Keep the route out of p99 latency alerts and slow-trace sampling, or the dashboards show a permanent incident. Watch what does mean something: a gauge of parked requests, the 200-to-204 ratio, 5xx from each proxy (a spike at a round number of seconds is a timeout), and the route’s own duration histogram.

Long polling in the wild

Long polling is not legacy; it is how most pull-based infrastructure APIs wait. The cursor is always the interesting part.

SQS short polling samples a subset of its servers and can come back empty while messages exist; long polling asks all of them, so there it is cheaper and more correct. Telegram keeps undelivered updates for 24 hours, its retention window. Consul tells clients to reset the index to 0 if it ever goes backwards. A Kubernetes watch streams like SSE, but its cursor rules are the ones above: about five minutes of history, 410 Gone for a stale resourceVersion, relist.

Long polling is also alive as a fallback: Socket.IO starts every connection on Engine.IO’s polling transport by default and upgrades to WebSocket once that works (Socket.IO: rooms, namespaces and scaling). What has faded is the hand-rolled long-poll fallback beside SSE; when an app still needs one, the network path is the reason, not the browser.

Between servers the push alternative is a webhook, paired with a periodic reconciliation poll, because a webhook missed during downtime is gone (Webhooks).

When plain polling is still right

• Few clients, relaxed freshness. An internal dashboard refreshing every 30 s does not need a held connection per viewer.
• Shared, cacheable data. Status pages, scoreboards, prices with a second of tolerance: the CDN absorbs the polls and the origin’s cost stays flat.
• Job status. After a 202 Accepted, polling the job URL with a server-chosen Retry-After is simple and finite (Long-running agent jobs).
• Platforms that bill by duration. A function behind API Gateway bills the whole hold as compute and holds a concurrency slot while it waits.
• Batch and cron consumers, which want a plain request, not a connection.

The trade not taken: SSE

For browser push, SSE beats long polling on almost every axis: the same held connection, no request or header block per event, resume via Last-Event-ID, and, since FastAPI 0.135, a native EventSourceResponse with built-in keepalive comments. The six-connection limit and the proxy timeouts apply to both.

Long polling still wins in two places. When something on the path buffers streaming responses (a corporate proxy, an antivirus, a misconfigured CDN), each long-poll answer is an ordinary complete response that buffering cannot delay. And for scripts and workers, a request/response API with a cursor is easier to retry, authenticate and reason about than a stream parser.

Related

• WebSocket vs Alternatives - the wider survey, with gRPC streaming and WebRTC
• Socket.IO: rooms, namespaces and scaling - Engine.IO’s polling transport and sticky sessions
• HTTP versions: 1.1, 2 and 3 - the six-connection limit and HTTP/2 multiplexing
• Nginx WebSocket Proxying - proxy_read_timeout and buffering on long-lived requests
• Cancellation, timeouts and error handling in asyncio - asyncio.timeout() and graceful shutdown
• Pub/sub and Streams - pub/sub as the doorbell, Streams as the log
• SQS — Visibility Timeout, Consumer Patterns, and Failure Handling - SQS long polling in a worker loop
• Timeouts, retries and backoff - backoff, jitter and Retry-After
• Streaming an LLM answer into a React UI - the browser side of the same decision
• Design: Notification Center - a design that makes this choice