Writing a client for an external API

One class per provider, constructed once, injected. Business code calls typed methods and never sees HTTP. Auth, logging and caching inside it are Outbound auth, logging and caching; this is the shape and the failure handling.

One client, for the whole process lifetime

@asynccontextmanager
async def lifespan(app: FastAPI):
    async with httpx.AsyncClient(
        base_url="https://api.example.com",
        timeout=httpx.Timeout(5.0, connect=2.0),
        limits=httpx.Limits(max_connections=100),
    ) as http:
        app.state.billing = BillingClient(http)
        yield

app = FastAPI(lifespan=lifespan)

Creating a client per request is the most common performance bug in this area. It throws away the connection pool and the TLS session, so every call pays a fresh handshake — tens of milliseconds, on every request, invisible in application profiling.

The timeout is not optional and is not one number:

A missing timeout means one unresponsive dependency holds a worker forever, which is how a slow third party becomes your outage.

The client owns error mapping

class BillingClient:
    def __init__(self, http: httpx.AsyncClient):
        self._http = http

    async def invoice(self, id: str) -> Invoice:
        r = await self._http.get(f"/invoices/{id}")
        if r.status_code == 404:
            raise InvoiceNotFound(id)
        if r.status_code == 429:
            raise RateLimited(retry_after(r))
        r.raise_for_status()
        return Invoice.model_validate(r.json())

Callers get InvoiceNotFound, not an httpx.HTTPStatusError they have to inspect. That is the boundary: HTTP vocabulary stops here, and swapping the provider does not ripple through the codebase.

Validating into a Pydantic model at the boundary is the other half. An API that starts returning null for a required field fails here, with a clear message, rather than three layers away as an AttributeError.

Retries: only what is safe, with jitter

TRANSIENT = (httpx.TransportError, RateLimited)

@retry(
    retry=retry_if_exception_type(TRANSIENT),
    wait=wait_exponential_jitter(initial=0.5, max=8),
    stop=stop_after_attempt(4),
    reraise=True,
)
async def invoice(self, id: str) -> Invoice:
    ...

Jitter is the part people omit. Plain exponential backoff makes every client that failed together retry together, so the recovering service is hit by a synchronised wave and fails again.

The other constraint: cap the total time, not just the attempt count. Four attempts at a 5-second timeout is a 20-second worst case, and whoever called you has their own budget.

Gotcha: a retried POST can charge a card twice. Retry non-idempotent calls only with an idempotency key the provider honours — see Idempotency.

Failing fast when it is genuinely down

Retries make a brief blip invisible and a sustained outage worse — every request now takes four attempts before failing. A circuit breaker is the fix: after N consecutive failures, reject immediately for a cooldown, then let one request through to test.

That converts a 20-second timeout into an instant, cheap error, which is what keeps your own latency budget intact. See Timeouts, retries and backoff.

Testing it

Define the client as a Protocol and service tests get a fake; the client’s own tests hit respx with recorded payloads:

class Billing(Protocol):
    async def invoice(self, id: str) -> Invoice: ...

See Mocking external APIs.