Streaming

An agent that takes forty seconds and shows nothing feels broken. Streaming is a product requirement before it is a technical one, and LangGraph gives you several kinds of it — the skill is picking the right one.

The stream modes

mode = "updates"
for chunk in graph.stream(inputs, stream_mode=mode):
    print(chunk)

updates is the one to default to. It gives you the node name and its state delta, which is what a progress UI needs — “retrieving”, “reranking”, “writing” — without shipping the whole state on every step.

values gets expensive. The full state after every step means large payloads if state holds documents; useful for debugging, wasteful for a UI.

checkpoints and tasks require a checkpointer, since there is nothing to report otherwise.

Token streaming versus step streaming

These answer different questions and people conflate them.

messages  -> "the answer is being typed"
updates   -> "we are on step 3 of 7"

messages gives you the token-by-token effect users expect from a chat box. updates tells them the system is working through stages. A long agent run needs both: stage progress while tools run, then token streaming when the final answer generates.

Pass a list to get both, and branch on the chunk type:

for chunk in graph.stream(
    inputs, stream_mode=["updates", "messages"]
):
    ...

Gotcha: messages mode only works with LangChain-integrated models, because the tokens are surfaced by the integration. A node calling a raw provider SDK produces nothing there — emit your own via custom instead.

Custom progress from inside a node

When a node does long work with no model call — parsing a hundred documents, say — nothing appears on the stream. Emit progress explicitly:

from langgraph.config import get_stream_writer

def ingest(state: State) -> dict:
    write = get_stream_writer()
    for i, doc in enumerate(state["docs"]):
        write({"stage": "parse", "done": i})
        parse(doc)
    return {"parsed": True}

This is the escape hatch that makes progress reporting work for arbitrary work, and it is what you use when messages cannot apply.

Subgraphs

By default a subgraph’s internal steps do not appear. Pass subgraphs=True to include them, and each chunk carries the namespace it came from so you can tell whose step it is. See Multi-agent and subgraphs.

Whether you want them is a product question: a user watching a support agent does not need to see a sub-agent’s tool calls, but an engineer debugging it does.

Streaming does not make it faster

The point worth making in an interview, because it separates a product answer from a naive one:

“Streaming changes perceived latency, not actual latency. The run takes just as long. What changes is that the user sees progress at 200ms instead of staring at a spinner for forty seconds — and separately, they can tell early that it is going wrong and stop it.”

That second clause is the underrated half. A visible plan lets a user abort a run that has misunderstood them, which saves both money and their patience.

For work too long to hold a connection at all, streaming is not the answer — a job with a status URL is. See Long-running agent jobs.

Wiring it to a client

Server-sent events are the usual transport: one-way, plain HTTP, reconnects built into the browser API. WebSockets only earn their place when the client also needs to send mid-run — for example to answer an interrupt.

async def sse(request):
    async def gen():
        async for chunk in graph.astream(
            inputs, stream_mode="updates"
        ):
            yield f"data: {json.dumps(chunk)}\n\n"
    return StreamingResponse(
        gen(), media_type="text/event-stream"
    )

Two practical details: disable proxy buffering or nothing arrives until the end, and decide what a disconnect means — either the run continues server-side and the client can reattach by thread id, or it is cancelled. That choice belongs in the design, not in a default.

Related

• Human in the loop — the pause a client must handle
• Long-running agent jobs — when streaming is not enough
• WebSockets — the transport options