Frontend / Frontend system design / 07_file_uploader_with_resume.md

Design: File Uploader with Resume

Updated 6 interview angles 7 min read source
On this page16
  1. Requirements to clarify
  2. Architecture
  3. API contract
  4. Client state per file
  5. Chunking strategy
  6. Concurrency cap
  7. Progress UX
  8. Failure modes and recovery
  9. Integrity verification
  10. Validation and security
  11. Drag-and-drop UX
  12. When to use a library
  13. Telemetry
  14. Related
  15. Further reading
  16. Interview angle

Design: File Uploader with Resume

Drag-and-drop multi-file uploader that survives flaky networks, large files, and tab refresh. The senior topics: direct-to-storage (presigned URLs, not your server), chunked / multipart for resilience, resumable protocol (S3 multipart or tus), concurrency cap, per-file progress + cancel, integrity verification, and the security perimeter in the presign step.

Requirements to clarify

  • Max file size. 10MB photos? 5GB videos? Drives chunking strategy.
  • Connection profile. Mobile / hotel Wi-Fi (flaky) vs office (stable)?
  • File count per session. 1 at a time, or “drop 500 photos”?
  • Resume across tab refresh / day later? Yes → need persistent state and server-side upload sessions.
  • Storage destination. S3 (multipart upload API) / GCS / self-hosted (tus)?
  • Post-upload processing. Trigger Lambda for thumbnails? Notify the user when done?
  • Privacy / signed download. Public after upload, or signed-URL only?

Architecture

text
[Client]
   │ 1. validate file client-side
   ├──▶ [API] /uploads/init
   │         auth, validate, return UploadId
   │         + a presigned URL per chunk
   │         │
   │         └──▶ storage opens a multipart session
   │
   │ 2. PUT each chunk to its presigned URL
   │    in parallel, under a concurrency cap
   ├──▶ [Storage: S3 / GCS / tus]
   │
   │ 3. notify completion with the chunk ETags
   ├──▶ [API] /uploads/complete
   │         server calls CompleteMultipartUpload
   │         and may trigger post-processing
   │
   └──▶ user sees confirmation

Server is never on the data path. The bytes go directly from browser to S3.

See File Uploads — Multipart, Presigned S3, Chunked, Resumable for the protocol details (multipart, presigned, tus).

API contract

text
POST /api/uploads/init
  { "filename": "video.mp4", "size": 5_368_709_120,
    "contentType": "video/mp4", "checksum": "sha256:..." }
→ {
    "uploadId": "u_abc",
    "key": "uploads/u_ada/2026/05/video.mp4",
    "partSize": 8_388_608,            // 8 MB
    "parts": [
      { "partNumber": 1,
        "url": "https://s3...&partNumber=1",
        "expiresAt": "..." },
      { "partNumber": 2, "url": "...", "expiresAt": "..." },
      ...
    ]
  }

PUT to each part URL -> response has an ETag header
                        (the client must store it)

POST /api/uploads/:uploadId/complete
  { "parts": [{ "partNumber": 1, "etag": "..." }, ...] }
→ { "fileId": "f_123", "url": "https://cdn/.../video.mp4" }

POST /api/uploads/:uploadId/abort
→ {}     // cancel the S3 multipart upload, clean up

The server validates size and contentType against an allow-list before signing — that’s the security boundary, not the client.

Client state per file

ts
type UploadFile = {
  id: string;                        // local UUID
  file: File;
  status: "queued" | "uploading" | "paused" | "done" | "failed" | "cancelled";
  uploadId?: string;                 // server-issued
  key?: string;
  parts: PartState[];                // per-chunk state
  progress: number;                  // 0..1 derived from parts
  error?: string;
};

type PartState = {
  partNumber: number;
  url: string;
  expiresAt: string;
  etag?: string;
  status: "pending" | "uploading" | "done" | "failed";
  attempts: number;
};

Persisted to IndexedDB so a refresh/quit can resume. The File reference can’t be serialized — the user has to re-pick the file to resume (browsers don’t allow durable file handles, except via the experimental File System Access API). The recovered IDB state lets the UI say “you had a video at 60% — re-add it to resume.”

Chunking strategy

  • Min part size (S3): 5 MiB except the last part.
  • Max parts (S3): 10,000.
  • A 50 GB file at 5 MiB parts = 10K parts (boundary). Pick a part size proportional to file size: partSize = max(8 MiB, ceil(size / 9500)).
  • Larger parts = fewer requests, larger blast radius on a single failure. 8-16 MiB is the sweet spot for most files.

Concurrency cap

Upload N parts in parallel; cap at 4-6.

ts
async function uploadParts(file: UploadFile, concurrency = 4) {
  const queue = file.parts.filter(p => p.status === "pending");
  const workers = Array.from({ length: concurrency }, async function worker() {
    while (queue.length) {
      const part = queue.shift()!;
      try { await uploadOnePart(file, part); }
      catch (e) {
        part.status = "failed";
        part.attempts++;   // requeue per the retry policy
      }
    }
  });
  await Promise.all(workers);
}

Why cap: 100 parallel PUTs saturate the browser’s 6-per-origin (HTTP/1.1) limit, exhaust bandwidth, increase per-part latency, and trigger S3 throttling. 4-6 is the published sweet spot.

Progress UX

  • Per-part progress via XMLHttpRequest.upload.onprogress (not fetch, which is unreliable — see File Uploads — Multipart, Presigned S3, Chunked, Resumable).
  • Per-file progress = sum of part bytes uploaded / total bytes.
  • Aggregate progress = sum across all files in queue.
  • ETA = remaining bytes / recent throughput (compute over a rolling window so a momentary stall doesn’t lie).
  • Pause / Resume / Cancel per file (AbortController for in-flight requests; queue manipulation for queued).

Failure modes and recovery

  • Part fails (network error, 5xx) → retry that part with exponential backoff + jitter (cap at e.g. 5 attempts). Don’t restart the whole file. See Retries, Backoff, and Idempotency (from the Frontend).
  • Presigned URL expired mid-upload → ask the server for a fresh URL for that part (POST /uploads/:id/parts/:n/resign) and retry.
  • Tab refresh → IDB has the upload state; user re-picks the file; client compares file size + (optionally) hash; if it matches, resume from parts where status !== "done".
  • Network drop → all in-flight parts fail; on reconnect, retry the failed parts.
  • Full upload session expires (some servers expire multipart uploads after N hours) → restart with a fresh init.
  • Abort → client calls /uploads/:id/abort so S3 doesn’t keep paying for the orphan parts. Set a bucket lifecycle policy as a backstop: “Abort incomplete multipart uploads after 7 days.”

Integrity verification

  • Client computes the file SHA-256 (using crypto.subtle.digest on the File’s ArrayBuffer — stream it for big files via ReadableStream) and sends with init.
  • Server can verify by checking x-amz-meta-checksum or by retrieving and hashing post-upload. For mission-critical files, this catches corruption (truncated, bit-flipped) before the upload is marked done.
  • Per-part checksums — S3 supports Content-MD5 per part; client computes and sends with each PUT. S3 rejects on mismatch.

Validation and security

  • Client-side: extension allow-list, max size, magic-byte sniff for the file’s actual type (don’t trust extension).
  • Server-side (presign): re-validate size, content type allow-list, scope key prefix per user (uploads/${userId}/...), short URL expiry.
  • Server-side (complete): actually fetch the object’s HEAD and verify the stored size + content type match what was promised. The client can lie at PUT time; this is the last checkpoint.
  • Post-upload scan: anti-malware scan via S3 event → Lambda → ClamAV or commercial service before exposing.

See Frontend Security — Senior Interview Prep for the broader file-upload security checklist.

Drag-and-drop UX

  • Drop zone visible always; highlights on dragenter.
  • File picker fallback (<input type="file" multiple>) for keyboard / mobile.
  • Reject files that fail client-side validation with an inline error (don’t fail silently).
  • Show a queue with each file’s state inline.
  • Don’t onDragOver without preventDefault() — the default behavior of dropping a file onto a page is to navigate to it.
tsx
<div
  onDragOver={(e) => { e.preventDefault(); setHover(true); }}
  onDragLeave={() => setHover(false)}
  onDrop={(e) => {
    e.preventDefault();
    setHover(false);
    enqueue([...e.dataTransfer.files]);
  }}
>
  Drop files here or
  <input type="file" multiple
    onChange={(e) => enqueue([...e.target.files!])} />
</div>

When to use a library

  • Uppy — full uploader UI with provider plugins (Google Drive, Dropbox), tus, S3 multipart, progress, retry. The default for “I need this in production tomorrow.”
  • tus-js-client — protocol implementation; bring your own UI.
  • @aws-sdk/lib-storage — S3 multipart upload in the browser; lower level than Uppy.

A senior answer mentions one of these by name and explains why hand-rolling is rarely worth it.

Telemetry

  • p99 upload latency per MB.
  • Retry rate per part.
  • Resume rate (% of uploads that experienced a resume).
  • Abort rate (% of users who cancel mid-upload).
  • Avg parts per file (ties to chunk-size choice).

Further reading

Interview angle 6

  • “How would you architect large file uploads?” - presigned URLs direct to object storage so your servers are never on the data path, and multipart upload for anything large or on an unreliable network. A single PUT of a gigabyte has no recovery from a dropped connection; parts do.
  • “What does per-part reliability look like?” - retry each part with exponential backoff, and re-sign a part’s URL when the presigned URL expires mid-upload. A long upload will outlive its signatures, and that failure looks like a permissions error rather than a timeout.
  • “How many parts in parallel?” - four to six. Beyond that you hit the browser’s per-origin connection limit and the storage provider’s throttling at the same time, so throughput stops improving and error rates go up.
  • “How do you actually resume after a page reload?” - persist the upload session in IndexedDB: the upload id, the part list, and which parts completed. The File object cannot be re-attached durably, so the user re-picks the file, you verify size and a hash to confirm it is the same one, and resume the incomplete parts.
  • “Where is the security boundary?” - the presign endpoint. It validates size and content type, and scopes the key prefix to the user so nobody can write over someone else’s object. Verify the stored object’s metadata after the fact too, since the client controls what it actually sent.
  • “Would you build this?” - no. Uppy or tus-js-client handle the retry, resume and progress plumbing, and hand-rolling it is a lot of edge cases for no differentiation.