Design an internal automation platform
The brief: a company wants to automate operational work across Finance, Operations and Guest Experience. Dozens of processes, mostly moving data between SaaS systems, some needing judgement that an LLM could supply. The people whose work it is are not engineers.
Distinct from Design an agent platform, which is a platform engineers build agents on. Here the users are the business, most workflows are integration glue rather than agents, and the hardest problems are ownership and trust rather than isolation and evaluation.
Clarify first
- Who owns a workflow after it ships? The single most important question. If the answer is “the automation team”, you are building a queue of maintenance requests, and it will saturate.
- What is the blast radius of a wrong action? Reading data and drafting a reply is a different risk class from issuing a refund or emailing a guest. This decides how much human-in-the-loop you build.
- Which systems, and do they have APIs? A process that depends on a system with no API means scraping or a human step, and that changes the estimate by an order of magnitude.
- Is this replacing headcount or freeing capacity? It changes what “success” measures, and people will answer the question you did not ask honestly.
The first one is what separates a platform answer from a project answer. Say it before drawing anything.
Do not automate first
The design starts before the architecture: map the process, time it, and eliminate what should not exist. A two-minute step inside a three-day approval loop is not worth automating, and encoding a policy with eleven exceptions makes the exceptions permanent. Method in Process mapping and choosing what to automate.
Bring one number to the table per candidate process: hours saved per month × months until the policy changes, against build plus maintenance.
The three-tier shape
┌─────────────────────────────────────────┐
SaaS │ bought tool e.g. the CRM's own │
├─────────────────────────────────────────┤
glue │ low-code canvas triggers, routing │
│ │ calls │
├────────▼────────────────────────────────┤
rules │ your service business logic, │
│ tested, in git │
└─────────────────────────────────────────┘The boundary is the whole design. Integration glue — triggers, connectors, retries, notifications — belongs in the canvas, where it is fast to build and a non-engineer can read it. Anything with a business rule, a calculation, or a compliance consequence belongs behind an HTTP endpoint you own, so it has tests, review and a git history.
A workflow that calls one endpoint of yours is reviewable. A workflow with fourteen branches and a pricing formula in a Set node is production code with no tests, and nobody can reconstruct what it did last Tuesday.
Platform choice — Zapier, Make, n8n, and the billing-unit argument that usually decides it — is in Zapier, Make and choosing an automation platform. For an internal platform with volume and sensitive data, self-hosted n8n is the common answer; the operational consequences are in n8n at scale: topology, connections and retention.
Where AI earns its place
Most of these workflows do not need a model. Be explicit about the test: an LLM is for judgement over unstructured input; a rule is for everything else. Classifying a free-text complaint, extracting fields from a supplier invoice, drafting a reply for a human to send — those are model jobs. Deciding whether an amount exceeds a threshold is not, and a model doing it will be wrong occasionally and expensively.
Where a model is used, the pattern that ships:
def handle(item: Ticket) -> Outcome:
result = classify(item) # the model step
if result.confidence >= THRESHOLD and result.action in AUTO_SAFE:
return execute(result) # the confident majority
return review_queue.add(item, suggestion=result) # everything elseThe routing rate is the metric, not accuracy. If 80% clears automatically and the queue is manageable, the system works; if the queue grows faster than people drain it, you have moved the work rather than removed it. Human-in-the-loop mechanics in Durable execution and human-in-the-loop.
Reliability for many small workflows
Per-workflow SLOs do not scale to sixty workflows and nobody will maintain them. What works is a small set of platform-level signals plus per-workflow classification:
| Signal | Why |
|---|---|
| Oldest queued job age | leading indicator of falling behind |
| Failed executions by workflow | finds the one that broke |
| Silent workflows | a stopped trigger emits no errors |
| Connection saturation | the usual outage cause |
Alert on absence. The characteristic failure here is not an error — it is an expired OAuth token, a disabled workflow, a renamed spreadsheet column. The data just stops moving. A heartbeat per workflow (“expected at least one run in the last N hours”) catches the class that error-rate alerting structurally cannot.
Classify workflows by criticality rather than treating them alike: tier 1 pages, tier 2 opens a ticket, tier 3 shows up in a weekly digest. Otherwise every alert is either noise or missing.
Ownership, which is where these die
- Name an owner per workflow, in the business, before it ships. Not the builder.
- Publish a catalogue — what exists, what it touches, who owns it. Without one, nobody can answer “what breaks if we change this field”, and after two years nobody knows what half of them do.
- Version the definitions. Export workflow JSON to git on a schedule so there is a diff and a rollback.
- Credentials in the platform vault, per-integration, with rotation and an owner. A personal OAuth token in a workflow leaves with the person.
- Plan for the shadow process. People keep the spreadsheet until they trust the automation. That is a signal about trust, not training — measure adoption and ask why, rather than mandating it.
Rollout
- Pick a process with high volume, low variability and a willing owner.
- Baseline it — volume, touch time, elapsed time, error rate.
- Build it, and shadow-run: the automation proposes, a human still acts.
- Compare proposals against what the human did for a period.
- Cut over with the review queue in place, and keep measuring.
Shadow-running is what buys trust cheaply, and it is the step candidates skip. It also gives you a labelled dataset for the model step for free.
Failure modes to name
- The automation team becomes a ticket queue. No business owners.
- Work moved rather than disappeared. A review queue that costs what the manual process did.
- The undocumented exception, surfacing in week three after the design set.
- Silent stoppage with no heartbeat, discovered by a customer.
- A canvas full of business logic that nobody can review or roll back.
- Automating around a broken system instead of fixing it — sometimes the right answer is a config change in the ERP.
Related
Interview angle 4
- “Design an automation platform for our operations teams.” - start with ownership: who maintains a workflow after it ships, because “the automation team” means you are building a maintenance queue. Then the three tiers — SaaS, low-code glue, your own service for business rules — and be explicit that the boundary between glue and rules is the design.
- “Where would you use an LLM here?” - judgement over unstructured input: classifying free text, extracting from documents, drafting for a human. Not thresholds or arithmetic. Route confident predictions automatically and queue the rest, then watch the routing rate rather than accuracy.
- “How do you monitor sixty workflows?” - not per-workflow SLOs. Platform signals — oldest queued job age, failures by workflow, connection saturation — plus a heartbeat per workflow, because the characteristic failure is silent stoppage from an expired token rather than an error.
- “How do you get people to trust it?” - shadow-run: the automation proposes while a human still acts, compare for a period, then cut over. It buys trust cheaply and gives you a labelled dataset for the model step for free.