Backend / Networking / 17_email_deliverability.md

Email deliverability and sending infrastructure

Updated 4 interview angles 5 min read source
On this page7
  1. The three DNS records, and what each proves
  2. Reputation is per-domain and per-IP
  3. The metrics that decide whether you keep sending
  4. Verify addresses before sending, not after
  5. Where it breaks in production
  6. Related
  7. Interview angle

Email deliverability and sending infrastructure

Sending email is trivial. Getting it delivered is an infrastructure problem with its own protocols, its own reputation system, and failure modes that are invisible from your side — the API returns 200 and the message lands in spam, or nowhere, and nothing in your logs says so.

Relevant to any product that emails users, and central to outreach and enrichment systems where volume makes reputation the constraint.

The three DNS records, and what each proves

All three are TXT records on the sending domain — see DNS for record types.

Record Answers Mechanism
SPF may this IP send for us authorised sender list
DKIM was this message altered cryptographic signature
DMARC what to do when those fail policy plus reporting

SPF publishes which servers may send for your domain. Its trap is the ten-lookup limit: every include: costs a DNS lookup, nested includes count, and past ten the record is permerror — which many receivers treat as a fail. Adding a fourth vendor is what usually breaks a working setup.

DKIM signs headers and body with a private key; the receiver fetches the public key from DNS and verifies. Because it survives forwarding, it is the stronger of the two.

DMARC ties them together and is where the actual policy lives:

text
v=DMARC1; p=none; rua=mailto:dmarc@example.com; pct=100
                  ▲                ▲
                  │                └─ aggregate reports come here
                  └─ none | quarantine | reject

Alignment is the part people miss. DMARC does not just require SPF or DKIM to pass — it requires the passing domain to match the From: header domain. A message can pass SPF for your ESP’s domain and still fail DMARC, because the From: says yours. Fixing that means a custom return-path and a DKIM key on your own domain.

Roll out as p=none first, read the aggregate reports until you can account for every legitimate sender, then move to quarantine and only then reject. Going straight to reject silently destroys mail from systems nobody remembered — the CRM, the invoicing tool, the monitoring alerts.

Reputation is per-domain and per-IP

Receivers score you, and the score is earned slowly and lost quickly.

  • Shared vs dedicated IP. Shared pools borrow other senders’ reputation, good and bad; a dedicated IP is yours alone but starts at zero and needs volume to establish. Below roughly a few thousand a day, shared is usually better because a dedicated IP never accumulates enough signal.
  • Warm up. A new domain or IP sending thousands on day one looks exactly like a compromised account. Ramp over weeks, starting with recipients most likely to engage.
  • Subdomains isolate risk. Send transactional from one subdomain and marketing or outreach from another, so a campaign that draws complaints cannot take password resets down with it. This is the highest-value structural decision.

The metrics that decide whether you keep sending

python
# Complaints are the one with a cliff. Bounces are the one that
# gets you blocked fastest. Track both per sending domain.
def sending_health(window):
    return {
        "bounce_rate":    window.hard_bounces / window.delivered,
        "complaint_rate": window.complaints   / window.delivered,
        "open_rate":      window.opens        / window.delivered,
    }
Signal Watch for
Hard bounce rate above ~2% suspends accounts
Complaint rate above ~0.1% is serious trouble
Spam-trap hits any is a list-hygiene failure

Hard bounces must suppress permanently. A hard bounce is a confirmed non-existent address; sending to it again is the clearest signal of a non-consensual list, and ESPs suspend for it. Keep a suppression list and check it before every send — this is a correctness requirement, not an optimisation.

Soft bounces are different — a full mailbox, a temporary defer — and should retry with backoff, then convert to hard after repeated failures.

Feedback loops (FBLs) deliver complaint notifications from major providers; wire them to automatic suppression. A complaint means unsubscribe immediately, regardless of what the recipient clicked.

Verify addresses before sending, not after

Bounces are the expensive way to learn an address is invalid. In order of cost:

  1. Syntax and domain MX check — free, catches typos and dead domains.
  2. SMTP probe (RCPT TO without sending) — many servers accept everything or rate-limit probes, so treat the result as a hint.
  3. A verification vendor — worth it above meaningful volume, because it costs less than the reputation damage.

Catch-all domains accept everything and verify as valid regardless, so a “verified” address from one proves nothing. Treat those as a separate risk tier.

Where it breaks in production

  • SPF over ten lookups after adding one more vendor. Silent permerror.
  • DMARC p=reject rolled out before auditing senders. Internal systems stop delivering and nobody connects it to the DNS change.
  • No alignment, so DMARC fails while SPF and DKIM both “pass”.
  • One domain for transactional and outreach. A campaign’s complaints take password resets with them.
  • Bounces not suppressed, so the list degrades until the account is suspended.
  • Sudden volume change. Ramping from 200 to 20,000 in a day reads as compromise even with perfect authentication.
  • Measuring opens as truth. Privacy proxies pre-fetch images, so open rates are inflated and cannot be trusted as an engagement signal.

Interview angle 4

  • “What are SPF, DKIM and DMARC?” - SPF authorises sending IPs, DKIM signs the message so tampering is detectable and it survives forwarding, DMARC sets the policy when those fail and reports on it. Follow-up: what is alignment? DMARC requires the passing domain to match the From: header, so you can pass SPF for your ESP and still fail DMARC.
  • “Mail suddenly stopped being delivered after we added a vendor.” - SPF’s ten-DNS-lookup limit. One more include: tips it into permerror, which many receivers treat as a fail. Flatten the record or move senders to subdomains.
  • “How would you set up sending for a new outreach system?” - separate subdomain from transactional mail so complaints cannot take password resets down, authenticate all three records with alignment, warm up over weeks rather than starting at volume, and suppress hard bounces permanently from day one.
  • “What do you monitor?” - hard bounce rate and complaint rate per sending domain, because both have thresholds that get accounts suspended, plus spam-trap hits as a list-hygiene signal. Not open rate — privacy proxies pre-fetch images, so it is inflated.