Production and debugging

What separates a graph that works on your laptop from one you can operate.

Durability: when the checkpoint is written

Checkpointing has a cost, so the runtime lets you choose when it happens.

Sync is the one that survives a hard kill, because the checkpoint is durable before the next node starts. Async writes concurrently and can lose the last step if the process dies mid-write. Exit gives you no mid-run recovery at all.

The rule: match durability to the cost of repeating a step. A graph whose nodes call paid APIs or move money wants sync — repeating that step is expensive or wrong. A read-only summarisation pipeline is fine on exit.

Determinism is a requirement, not a nicety

Resuming replays the interrupted node from its start. So a node must be safe to run twice — the same constraint as Human in the loop, and it applies to crash recovery too.

That last one is quiet and real. A node that branches on datetime.now() can take a different path on replay than it did originally, which makes a resumed run diverge from the one you were debugging. Generate the value once, store it in state, and read it from there.

Testing a graph

The thing that makes graphs testable is that nodes are ordinary functions.

def test_route_stops_at_limit():
    state = {"steps": 6, "score": 0.1}
    assert route(state) == "finish"

Three layers, in the order of value:

1. Node and router unit tests. No graph, no model, no mocks. Most of your logic lives here if node bodies stay thin.
2. Graph tests with a fake model. Compile the real graph with a stub that returns canned tool calls, and assert the path taken. This catches wiring errors — an edge to the wrong node — that unit tests cannot.
3. Evals against the real model. A separate, gated job, not part of the unit suite. See The eval and observability tooling.

Keeping (3) out of CI’s fast path matters: it is slow, costs money and is non-deterministic, so mixing it with unit tests makes the suite untrustworthy.

Observability

A graph without tracing is undebuggable — the whole point is that the runtime owns control flow, so you cannot read the path off the code.

What to capture per run: the thread id, the node sequence, each node’s state delta, model and prompt versions, tokens and cost per call, and retrieved chunk ids. See LLM observability.

The debugging workflow that actually works:

bad answer reported
  └─▶ find the thread id
      └─▶ read the state history
          └─▶ find the step where it went wrong
              └─▶ was retrieval right?
                  no  → chunking or query
                  yes → prompt or model

The state history is the advantage over a plain agent loop: you have every intermediate state, not just the final answer, so “which step broke” is a lookup rather than a reconstruction.

Time travel as a debugging tool

# Find the checkpoint before the bad step
history = list(graph.get_state_history(config))
target = history[3].config

# Change one value and re-run from there
graph.update_state(target, {"query": "corrected"})
graph.invoke(None, target)

This forks rather than overwrites, so the original run survives for comparison. Being able to say “I replayed the failing thread from the checkpoint before the bad retrieval with a corrected query” is a concrete answer to how you debug a non-deterministic system.

The failure modes to name

• Runaway loop. A cycle with no step counter. The recursion limit raises rather than finishing — own the limit yourself (Nodes, edges and routing).
• State bloat. Documents accumulating across steps, multiplied by every checkpoint. Keep ids, fetch bodies.
• Blocking the event loop. A sync checkpointer or a sync HTTP call inside an async graph (Python Concurrency Models: Processes, Threads, and Asyncio).
• Swallowed interrupts. A bare except around interrupt() silently disables an approval gate.
• Unbounded cost. No per-run token budget, so one pathological input spends without limit (Concurrency and backpressure).

Deployment shape

A graph is a Python object, so it deploys like any Python service: a container behind an API, with Postgres for checkpoints. There is a managed platform if you want the runtime hosted, but it is not required and treating it as required is a misconception worth correcting.

The two things that make it operable rather than merely deployed: Postgres checkpoints so state survives a restart and scales past one process, and a per-run budget and timeout so one bad input cannot consume the shared provider quota.

Related

• Persistence and threads — what durability writes
• Agent failure modes — the general failure catalogue
• Incident response for AI systems — when it breaks in production
• Design an agent platform — running many of these