AI & ML / Agents & orchestration / LangGraph / 05_persistence_and_threads.md

Persistence and threads

Updated 5 interview angles 4 min read source
On this page8
  1. Two systems, different scopes
  2. Attaching one
  3. Choosing a backend
  4. Inspecting and editing state
  5. What this actually buys you
  6. The cost to name
  7. Related
  8. Interview angle

Persistence and threads

The feature that makes LangGraph more than a nicer loop. Attach a checkpointer and every step is saved, which buys you conversation memory, resumption after a crash, human-in-the-loop and time travel — all from the same mechanism.

Two systems, different scopes

Checkpointer Store
Scope one thread across threads
Holds graph state per step application data
For conversation, resume, HITL user preferences, facts

Checkpointers are short-term and thread-scoped. Stores are long-term and cross-thread — the place a fact about a user survives after the conversation ends. Most people only need the checkpointer at first.

Attaching one

python 
from langgraph.checkpoint.memory import InMemorySaver

checkpointer = InMemorySaver()
graph = builder.compile(checkpointer=checkpointer)

config = {"configurable": {"thread_id": "conv-42"}}
graph.invoke({"question": "hi"}, config)

Two things happened. Every superstep wrote a checkpoint, and the run is labelled conv-42.

The thread_id is the whole interface. Invoke again with the same one and the graph continues that conversation, state intact. Use a new one and it starts fresh. There is no session object to manage — the id is the cursor.

python 
# Same thread: the graph still has the earlier messages
graph.invoke(
    {"messages": [
        {"role": "user", "content": "and after?"}
    ]},
    config,
)

Choosing a backend

Backend Use for
InMemorySaver tests, notebooks
SqliteSaver local development, single process
PostgresSaver production

AsyncPostgresSaver is the one for an async application — mixing the sync saver into an async graph will block the event loop, which is the same mistake as any blocking call in async code (Python Concurrency Models: Processes, Threads, and Asyncio).

Since state is already in Postgres, you get its guarantees for free — backups, replication, and the ability to query runs with SQL. That is a real argument for the Postgres saver in a regulated environment over anything bespoke.

Gotcha: keep thread_id under 255 characters. Longer values hit a database column limit and fail at write time, not at validation.

Inspecting and editing state

The checkpointer is not write-only. You can read the current state, list history, and modify it.

python 
snapshot = graph.get_state(config)
snapshot.values          # the state dict
snapshot.next            # nodes about to run

for past in graph.get_state_history(config):
    print(past.config, past.values)

update_state writes a new checkpoint by hand:

python 
graph.update_state(config, {"answer": "corrected"})

This is the mechanism behind time travel: pick a past checkpoint, update it, and resume from there. The run forks — the original history is untouched, which is what makes it safe for debugging a production thread.

Being able to say “I can replay a failed run from the checkpoint before it went wrong, with one value changed” is a strong answer to “how do you debug an agent”.

What this actually buys you

  1. Conversation memory without a session store you wrote.
  2. Crash resumption — the process dies, the state is on disk, the run continues from the last completed step.
  3. Human-in-the-loop — see Human in the loop. Pausing for hours is just a checkpoint nobody has resumed yet.
  4. Auditability — every intermediate state is recorded, which in FinTech is a compliance artefact rather than a debugging nicety. See Model governance and responsible AI.

The cost to name

Checkpointing writes the whole state on every superstep. A state holding fifty retrieved documents across twenty steps is twenty copies of those documents.

The fixes are the ones from State and reducers: keep ids in state rather than bodies, and summarise accumulating history. There is also a retention question — checkpoints containing prompts and completions contain PII, so they need the same policy as any other trace store.

Interview angle 5

  • “How does LangGraph handle memory?” - a checkpointer writes state after every superstep, scoped to a thread_id. Invoking with the same id continues that conversation; a new id starts fresh. There is no session object — the id is the cursor.
  • “Checkpointer or store?” - the checkpointer is thread-scoped short-term state: conversation, resumption, human-in-the-loop. A store is cross-thread long-term data like user preferences that should outlive the conversation.
  • “Which backend in production?” - PostgresSaver, or AsyncPostgresSaver in an async app — the sync one blocks the event loop. State in Postgres inherits backups, replication and SQL queryability.
  • “How do you debug a bad run?” - read the state history, pick the checkpoint before it went wrong, update_state to change one value, and resume. The run forks rather than overwriting, so the original history survives.
  • “What does it cost?” - the full state is written every superstep, so accumulating document bodies in state multiplies storage by step count. Keep ids in state and fetch bodies in the node, and treat checkpoints as PII-bearing for retention.
Previous
create_agent and middleware