Frontend / Frontend system design / 02_infinite_feed.md

Design: Infinite Feed (Twitter-Style)

Updated 6 interview angles 6 min read source
On this page13
  1. Requirements to clarify
  2. API contract
  3. Client data model
  4. Virtualization — non-negotiable above ~200 items
  5. Fetching trigger — IntersectionObserver sentinel
  6. Scroll restoration
  7. “New posts at the top” insertion
  8. Media optimization
  9. Failure modes
  10. Telemetry
  11. Related
  12. Further reading
  13. Interview angle

Design: Infinite Feed (Twitter-Style)

A vertically scrolling list that loads more as you reach the end. The interesting parts are cursor pagination (offset breaks under writes), virtualization (rendering 10K DOM nodes is fatal), scroll restoration (back-button), new-content insertion (the “12 new posts” banner), and media optimization (most rows have images/video). The senior answer is “list of decisions,” not “I’d use useInfiniteQuery.”

Requirements to clarify

  • Item count ceiling. 1K? 100K? Unbounded? Drives virtualization and unload strategy.
  • Insertion model. Append-only? Real-time push from the top? Read-position memory?
  • Media. Image-heavy, video-heavy, text-only? Drives bandwidth budget and lazy loading.
  • Item height. Fixed (easier) or variable (needs dynamic measurement)?
  • Multi-column / responsive. Single column on mobile, multi-column at desktop widths?
  • Offline / partial offline. Cached pages survive reload?

API contract

Cursor pagination only — offset breaks the moment writes happen.

text
GET /api/feed?after=<cursor>&limit=20
→ 200 {
    "items": [{ "id": "p123", "createdAt": "...", "author": {...}, "body": "...", "media": [...] }],
    "nextCursor": "cur_abc",
    "newerCursor": "cur_xyz"           // for "fetch newer than this" polls
  }

Server returns:

  • nextCursor (continue scrolling down)
  • newerCursor (cursor anchor for “newer than” polling — for the “12 new posts” banner)

See Pagination — Offset vs Cursor, and Infinite Queries for the cursor design and useInfiniteQuery recipe.

Client data model

  • TanStack useInfiniteQuery stores data.pages[] — each page is a server response. Use select to flatten for rendering.
  • Cursor for “new posts” banner kept in a separate useQuery polling every 30s with the saved newerCursor.
  • Scroll position mirrored to URL or sessionStorage for restoration.

Virtualization — non-negotiable above ~200 items

DOM rendering scales linearly; 10K nodes destroys scroll perf and memory. Use @tanstack/react-virtual (or react-window):

tsx
const parentRef = useRef<HTMLDivElement>(null);
const virtualizer = useVirtualizer({
  count: items.length,
  getScrollElement: () => parentRef.current,
  estimateSize: () => 120,         // average row height
  overscan: 5,                     // render 5 above/below viewport
  measureElement: (el) => el.getBoundingClientRect().height,  // for variable height
});

return (
  <div ref={parentRef} style={{ height: 600, overflow: "auto" }}>
    <div style={{ height: virtualizer.getTotalSize(), position: "relative" }}>
      {virtualizer.getVirtualItems().map(v => (
        <div
          key={v.key}
          ref={virtualizer.measureElement}
          data-index={v.index}
          style={{ position: "absolute", top: 0, transform: `translateY(${v.start}px)` }}
        >
          <FeedItem item={items[v.index]} />
        </div>
      ))}
    </div>
  </div>
);

Variable-height items need measureElement so the virtualizer caches measured heights and adjusts subsequent positions. Without that, scroll jumps as heights are discovered.

Fetching trigger — IntersectionObserver sentinel

tsx
<div ref={sentinelRef} />   // last element in list
useEffect(() => {
  const obs = new IntersectionObserver(([e]) => {
    if (e.isIntersecting && hasNextPage && !isFetchingNextPage) fetchNextPage();
  }, { rootMargin: "400px" });
  if (sentinelRef.current) obs.observe(sentinelRef.current);
  return () => obs.disconnect();
}, [hasNextPage, isFetchingNextPage]);

rootMargin: "400px" fires the fetch before the sentinel actually enters the viewport — the user shouldn’t see a loading spinner unless the network is genuinely slow.

Scroll restoration

User scrolls 200 items deep, clicks a post, hits back. Native behavior loses position.

Strategies:

  • Persist scroll offset + loaded-cursor list in sessionStorage keyed by route URL. On mount, re-prefetch all pages up to where the user was, then jump to the saved scroll offset.
  • Next.js App Router automatically restores scroll for <Link> navigation if the layout cache still has the pages — works if the data is in the router cache.
  • Cheat: scroll restoration only works if heights are deterministic — virtualized lists need cached measurements to restore precisely.

This is the #1 polish item users notice. Production feeds (Twitter, Reddit) lose scroll position constantly and people complain.

“New posts at the top” insertion

Polling every N seconds with newerCursor returns a count of new posts (not the posts themselves — they may be many).

UX pattern:

  • A subtle banner appears: “12 new posts — click to load.”
  • Clicking prepends them to the list only if the user is at the top — otherwise inserting at index 0 jumps their scroll position. If they’re scrolled down, the banner stays until they scroll up, then it inserts.
tsx
const { data: newCount } = useQuery({
  queryKey: ["feed", "newer", newerCursor],
  queryFn: () => fetch(`/api/feed/count?after=${newerCursor}`).then(r => r.json()),
  refetchInterval: 30_000,
});

{newCount > 0 && <Banner onClick={prepend}>{newCount} new posts</Banner>}

For Twitter-volume firehose, replace polling with SSE/WebSocket — see Design: Chat (Optimistic UI + WebSocket + Presence) for the connection pattern.

Media optimization

  • loading="lazy" on <img> for native lazy loading (modern browsers).
  • <picture> with srcset + sizes — serve the right resolution per device pixel ratio and viewport width.
  • AVIF / WebP with JPEG fallback via <picture> <source>.
  • Aspect ratio in CSS to reserve space (aspect-ratio: 16 / 9) — prevents CLS when images load.
  • Video poster + lazy embed — don’t load the video tag until in viewport.
  • Decode hints: decoding="async" lets the browser decode off-thread.

A feed with 30 unoptimized images can ship 50MB; with srcset + AVIF it’s 2-3MB.

Failure modes

  • Stale data from cache while user expects new — pair staleTime with manual refresh affordance (“pull to refresh” mobile, header click web).
  • Mutation creates an item not yet in cache (user posts) — optimistically prepend to page 0; on server confirmation, replace temp id with real.
  • Item disappears between fetches (deleted) — TanStack Query refetch reconciles; the row vanishes on next refresh, or you handle 410 Gone in detail-fetch.
  • Slow page load mid-scroll — show a small inline loader in the sentinel; never block scroll.
  • Network drop — TanStack retries; if all retries fail, surface a banner.
  • Backgrounded tab — pause polling; resume on focus.

Telemetry

  • p99 fetch latency per page.
  • Sentinel-trigger to first-paint of new page time (catches slow render even if network is fast).
  • Scroll-to-bottom abandonment.
  • LCP per page (first-screen content).
  • INP for scroll responsiveness.

Further reading

Interview angle 6

  • “How do you paginate a feed?” - cursors. Offset re-scans everything it skips so deep pages get slow, and an insert while the user is scrolling shifts the window so they see a duplicate or miss an item. Return a nextCursor for older content and an anchor cursor for polling what is new.
  • “When do you virtualize, and what breaks?” - past roughly 200 rows. Variable heights are the hard part: without measuring each item the scrollbar is wrong and the list jumps. Keep a small overscan so the user never sees a blank region while scrolling fast.
  • “How do you avoid the user ever seeing a spinner?” - trigger the next fetch before they reach the bottom, with an IntersectionObserver sentinel and a generous rootMargin. The request completes while there is still content on screen.
  • “New posts arrive while the user is reading. What do you do?” - put them behind a banner and let the user opt in. Inserting into a scrolled list moves the content under their eyes, which is the single most irritating thing a feed can do.
  • “What is usually the actual performance problem?” - the media, not the list. Lazy loading below the fold, srcset and sizes so phones do not fetch desktop images, modern formats, and an aspect-ratio on every image so nothing shifts as they load.
  • “What is the polish item people skip?” - scroll restoration. Persist the loaded cursors and the scroll offset keyed by URL, so going back to the feed returns to where they were rather than the top.