Backend / Web frameworks / FastAPI / 19_deployment.md

Deployment

Updated 6 interview angles 4 min read source
On this page8
  1. The process model
  2. In a container, run one worker
  3. Health checks that mean different things
  4. Graceful shutdown
  5. Configuration
  6. Behind a proxy
  7. Related
  8. Interview angle

Deployment

Containers, CI and reverse proxies are covered generally in Docker, CI/CD and nginx. This is the part that is FastAPI’s own: what actually runs the app, and how many of it.

The process model

bash 
uvicorn app.main:app --host 0.0.0.0 --port 8000 --workers 4

--workers forks N independent processes, each with its own event loop and its own memory. That is the whole scaling story, and three consequences follow:

  • In-process state is per worker. A module-level dict, an lru_cache, a rate-limit counter — each worker has its own. Anything shared belongs in Redis or the database.
  • Lifespan runs once per worker. Four workers open four connection pools, so pool_size=20 is 80 connections against Postgres, not 20. This is the most common way a service exhausts max_connections.
  • Memory is per worker. A 400 MB model loaded at startup is 1.6 GB.

Rule of thumb: workers ≈ CPU cores for a CPU-bound service. For the I/O-bound service FastAPI is usually built for, fewer workers each handling high concurrency is better — the loop is idle during I/O, and more processes only multiply the connection pools.

Gunicorn with uvicorn workers is the older recipe:

bash 
gunicorn app.main:app -k uvicorn.workers.UvicornWorker -w 4

It buys process supervision — restarting a worker that dies or exceeds --max-requests. Under Kubernetes or ECS the orchestrator already does that, so plain uvicorn --workers is usually enough and one less layer.

In a container, run one worker

dockerfile 
FROM python:3.14-slim
WORKDIR /app
COPY requirements.txt .
RUN pip install --no-cache-dir -r requirements.txt
COPY . .
USER 1000
CMD ["uvicorn", "app.main:app", "--host", "0.0.0.0", "--port", "8000"]

No --workers. Scale with replicas, not with in-container processes: the orchestrator can then schedule, autoscale and roll them individually, and one crashed process is one dead container rather than a silently degraded one. Setting both means the HPA counts CPU across four hidden processes and scales on a number that does not mean what it says.

Health checks that mean different things

python 
@app.get("/livez")
async def livez():
    # no dependencies. none.
    return {"ok": True}

@app.get("/readyz")
async def readyz(session: Annotated[AsyncSession, Depends(get_session)]):
    await session.execute(text("SELECT 1"))
    return {"ok": True}

Liveness answers “is this process wedged” and must touch nothing. Readiness answers “can I serve right now” and may check dependencies. Put the database in liveness and a thirty-second database blip restarts every pod at once — see Load balancing and CDN.

Graceful shutdown

On SIGTERM uvicorn stops accepting connections and waits for in-flight requests. Two things make that work:

  • terminationGracePeriodSeconds longer than your slowest request, or the orchestrator SIGKILLs mid-request.
  • Lifespan teardown that actually closes things. The code after yield runs on shutdown; a client left open holds connections until the process dies.

A readiness probe that fails immediately on shutdown is what drains traffic before the process goes, so the load balancer stops sending work while the last requests finish.

Configuration

python 
class Settings(BaseSettings):
    database_url: PostgresDsn
    api_key: SecretStr
    model_config = ConfigDict(extra="forbid")

# raises at import, not at first request
settings = Settings()

Constructed at import so a missing variable stops the container at boot, where the deploy sees it, rather than on the first request that reaches that code. SecretStr keeps credentials out of tracebacks. See Pydantic in practice.

Behind a proxy

bash 
uvicorn app.main:app --proxy-headers --forwarded-allow-ips='*'

Without --proxy-headers, request.client.host is the proxy and every generated URL is http:// on the internal port — which breaks OAuth redirect URIs in a way that is confusing to debug. Only trust the header from a proxy you control; --forwarded-allow-ips is what scopes that.

Interview angle 6

  • “How do you run FastAPI in production?” - uvicorn, with the process count decided by where it runs. In a container, one worker and scale with replicas so the orchestrator can schedule and autoscale them; outside one, --workers or gunicorn with uvicorn workers for supervision.
  • “What breaks when you add workers?” - each is a separate process with its own event loop, memory and lifespan. Four workers open four connection pools, so pool_size=20 becomes 80 connections; in-process caches and counters stop being shared; a loaded model is duplicated per worker.
  • “How many workers?” - roughly CPU count if CPU-bound. For I/O-bound work, fewer workers each at high concurrency is better: the loop is idle during I/O anyway, and more processes just multiply connection pools and memory.
  • “Liveness or readiness?” - liveness must touch nothing and answers “is this process wedged”; readiness may check dependencies and answers “can I serve now”. A database check in liveness turns a brief blip into a fleet-wide restart.
  • “What does graceful shutdown need?” - a grace period longer than the slowest request, lifespan teardown that closes the pools, and a readiness probe that fails immediately so the load balancer drains traffic before the process exits.
  • “Your OAuth redirects point at http and an internal port. Why?” - uvicorn is not reading the proxy’s forwarded headers. --proxy-headers with --forwarded-allow-ips scoped to the proxy you control fixes it.
Previous
OpenAPI and the generated docs