Frontend / APIs & data fetching / 09_file_uploads.md

File Uploads — Multipart, Presigned S3, Chunked, Resumable

Updated 5 interview angles 7 min read source
On this page12
  1. Show me the multipart/form-data upload to your own API.
  2. Why presigned URLs instead of routing through your API?
  3. Where do you enforce file-size and type limits?
  4. How do you show upload progress?
  5. When do you need chunked / multipart uploads?
  6. What is tus and when do you use it?
  7. How do you handle “user dragged 200 files into the browser”?
  8. Security pitfalls?
  9. Gotchas
  10. Related
  11. Further reading
  12. Interview angle

File Uploads — Multipart, Presigned S3, Chunked, Resumable

Three patterns, ordered by file size and reliability requirements:

  1. multipart/form-data to your API — simple, fine for small files (<10MB), routes the bytes through your server.
  2. Presigned URL straight to object storage — your API hands the client a short-lived signed URL; client PUTs directly to S3/GCS/Azure. Your server never touches the bytes. The right default for everything bigger than “a profile photo.”
  3. Chunked / resumable upload (multipart upload to S3, tus protocol) — split the file into parts, upload each (in parallel), resume failed parts without restarting. Required for large files (~100MB+) and flaky networks.

Plus: progress UX, drag-and-drop, validation before upload, and the security perimeter around the API that issues signed URLs.

Show me the multipart/form-data upload to your own API.

tsx 
async function upload(file: File): Promise<{ id: string }> {
  const form = new FormData();
  form.append("file", file);
  form.append("description", "vacation photo");

  const res = await fetch("/api/uploads", { method: "POST", body: form });
  if (!res.ok) throw new Error(`upload failed: ${res.status}`);
  return res.json();
}

Do not set Content-Type yourself — the browser sets multipart/form-data; boundary=... correctly. Manually setting application/json is the canonical bug.

Limits to mind: the server body limit (Express ~100KB by default; configure for uploads), the reverse-proxy body limit (nginx client_max_body_size), and your serverless runtime’s payload limit (API Gateway 10MB, Lambda 6MB sync — large files can’t go this route on serverless).

Why presigned URLs instead of routing through your API?

Three reasons:

  1. Your server bandwidth — direct-to-S3 means the bytes never traverse your origin. Major cost and bandwidth savings.
  2. Serverless limits — Lambda/API Gateway have payload caps; direct-to-S3 bypasses them entirely.
  3. Reliability — S3’s PUT is highly tuned; your origin is not.

Pattern:

ts 
// 1. Client asks server for a signed URL
const { url, key } = await fetch("/api/uploads/presign", {
  method: "POST",
  body: JSON.stringify({ filename: file.name, contentType: file.type }),
}).then(r => r.json());

// 2. Client PUTs directly to S3
await fetch(url, { method: "PUT", body: file, headers: { "Content-Type": file.type } });

// 3. Client tells server "I'm done, here's the key"
await fetch("/api/uploads/finalize", {
  method: "POST",
  body: JSON.stringify({ key }),
});

Server-side (presign), Python boto3 example:

python 
url = s3.generate_presigned_url(
    "put_object",
    Params={"Bucket": bucket, "Key": key, "ContentType": content_type},
    # 5 minutes
    ExpiresIn=300,
)

The signature binds the URL to the bucket+key+content-type+expiry — the client can’t upload to a different key or change the content type. Validation belongs in the presign step, not in the PUT.

Where do you enforce file-size and type limits?

Multiple layers (defense in depth):

  • Client side, pre-upload — quick UX feedback (reject 1GB videos before the user starts).
  • Presign endpoint — validate contentType and contentLength against your allow-list before signing. Refuse to sign if invalid.
  • S3 bucket policy — can enforce Content-Length ranges and content-type prefixes via conditions in the signed URL.
  • Post-upload server hook — on finalize, fetch the object’s headers and verify the actual size/type match what was promised. Without this, the client can lie at PUT time.

Client-side validation alone is not a security boundary — it’s UX. Always re-validate server-side.

How do you show upload progress?

fetch does not expose upload progress (it’s been broken/half-supported for years). The reliable options:

  1. XMLHttpRequest — old but xhr.upload.onprogress works:
    ts 
    const xhr = new XMLHttpRequest();
xhr.open("PUT", url);
xhr.upload.onprogress = (e) => {
  if (e.lengthComputable) setProgress(e.loaded / e.total);
};
xhr.send(file);
  2. fetch + ReadableStream for downloads (response progress) works fine, but for upload progress, XHR is still the answer in most browsers. Streams API spec for Request body progress (duplex: "half" etc.) is rolling out but uneven.
  3. Library: axios, tus-js-client, Uppy — all give you progress + retry + chunking out of the box.

When do you need chunked / multipart uploads?

  • Files > ~100MB.
  • Networks that drop mid-upload (mobile, hotel Wi-Fi).
  • “Resume from where it stopped” UX.

Strategy: split into 5-100MB parts, upload them (optionally in parallel), assemble server-side.

S3 Multipart Upload is the canonical implementation. Flow:

text 
1. POST /api/uploads/init       → server creates S3 multipart upload, returns UploadId + N presigned URLs (one per part)
2. PUT  each part to S3 in parallel → S3 returns an ETag per part
3. POST /api/uploads/complete   → client sends part numbers + ETags; server calls S3 CompleteMultipartUpload

If a part fails, retry just that part. If the user closes the tab, the partial upload sits in S3 (charged) until cleaned up — set a bucket lifecycle policy: Abort incomplete multipart uploads after 7 days.

What is tus and when do you use it?

tus is an open protocol for resumable uploads over HTTP — sequence of PATCH requests with Upload-Offset headers. The client resumes from the last acknowledged offset.

http 
POST /files                       → returns location URL with upload ID
PATCH /files/abc  Upload-Offset: 0 → write some bytes, server returns new offset
PATCH /files/abc  Upload-Offset: N → write more bytes
HEAD  /files/abc                  → client checks current server offset on resume

Pick tus over S3 multipart when you self-host storage or need a single uniform resumable-upload protocol across multiple backends (S3 + on-prem + GCS). S3 multipart is simpler if you’re already on S3.

How do you handle “user dragged 200 files into the browser”?

  • Concurrency cap. Upload 3-5 files at once max; queue the rest. Otherwise you saturate the browser’s 6-connection limit per origin (on HTTP/1.1) or hammer S3.
  • Per-file progress + cancel — each file gets its own row with a cancel button (tied to an AbortController).
  • Aggregate progress — total bytes uploaded / total bytes queued, ETA.
  • Recover failed individually — one failed file shouldn’t take down the queue.
ts 
async function uploadQueue(files: File[], concurrency = 4) {
  const queue = [...files];
  const workers = Array.from({ length: concurrency }, async function worker() {
    while (queue.length) {
      const file = queue.shift()!;
      try { await upload(file); } catch (e) { recordError(file, e); }
    }
  });
  await Promise.all(workers);
}

Security pitfalls?

  • Open presign endpoint — anyone authenticated can upload anywhere — set tight key prefixes per user (uploads/${userId}/...) and reject other paths in the signing logic.
  • No content-type lock — user signs as image/png, uploads an HTML file, you serve it — XSS. Bind Content-Type in the signature.
  • No size limit in the signature — user uploads a 50GB file, bills you. Enforce Content-Length ranges.
  • Trusting the file extension — evil.png.exe. Sniff magic bytes server-side or validate via a content-type allow-list.
  • Public-read uploads — the bucket should default to private; serve via signed GET URLs or via CloudFront with origin access control.
  • Path traversal in filename — never let filename from the client become the object key directly. Generate a UUID, store the original name in metadata.

Gotchas

  • fetch upload progress is unreliable cross-browser — use XMLHttpRequest for progress, period.
  • CORS for direct-to-S3 — the bucket needs CORS configured to allow PUT from your origin, and to expose ETag (otherwise the client can’t read it for multipart completion).
  • Pre-flight on PUT — non-simple Content-Type triggers a CORS preflight (OPTIONS); bucket CORS must allow it.
  • Browser memory — reading a 1GB file into a FormData blows up. Stream from the File blob directly to fetch/xhr; don’t arrayBuffer() it.
  • File objects can become invalid if the user moves/deletes the source file before upload — handle the read error.
  • Mobile uploads can be paused by the OS when the tab backgrounds — only resumable protocols recover cleanly.

Further reading

Interview angle 5

  • “How would you handle file uploads?” - three patterns by size and reliability. Multipart straight to your server for small files, a presigned URL direct to object storage as the default, and resumable multipart for large files or unreliable networks.
    • Follow-up: “Why is presigned the default?” - it keeps the bytes out of your application servers entirely, so a hundred concurrent uploads do not consume your request capacity or your bandwidth budget.
  • “Where does validation belong?” - in the presign step, because that is where the security boundary is. The signature can lock the content type and a maximum size, so the client cannot upload something else. Client-side checks are for a fast error message, not for enforcement.
  • “What else do you lock down on the bucket?” - key prefixes scoped per user so one cannot overwrite another’s object, and a lifecycle rule to abort incomplete multipart uploads. Without that rule, abandoned parts accumulate and you are billed for storage nobody can see or list.
  • “How do you show upload progress?” - XMLHttpRequest or a library, not fetch. fetch has no reliable upload progress event, which is one of the few remaining reasons to reach for the older API deliberately.
  • “A user drops fifty files in at once.” - cap concurrency at four or five in flight. More saturates the browser’s per-origin connection limit and triggers throttling at the storage provider, so the whole batch gets slower rather than faster.
Previous
Streaming an LLM answer into a React UI