The agent loop

Article.

The loop is ten lines. Everything that makes it work in production is the code around it — the limits, the message protocol it must not violate, and the fact that its cost grows with the square of the number of steps.

Being able to write the ten lines from memory and then name where they break is the practical test.

ReAct

Reason + Act. The model alternates between thinking and calling tools, feeding each observation back in.

Thought:      I need the customer's order history.
Action:       search_orders(email="a@b.com")
Observation:  [3 orders, latest #1042 shipped 2d ago]
Thought:      #1042 is likely. Check its tracking.
Action:       get_tracking(order_id="1042")
Observation:  Delivered yesterday 14:20.
Answer:       Order #1042 was delivered yesterday at 14:20.

Modern implementations do not parse Thought: and Action: out of text — native tool calling returns structured calls — but the loop shape is identical and “ReAct” still names it.

The loop itself

messages = [{"role": "user", "content": query}]

for step in range(MAX_STEPS):
    response = await llm(messages, tools=TOOLS)
    messages.append(response)          # append BEFORE executing

    if not response.tool_calls:
        return response.content        # the model decided it is done

    for call in response.tool_calls:
        result = await execute(call)   # validate args inside execute()
        messages.append({
            "role": "tool",
            "tool_call_id": call.id,   # must match, exactly
            "content": truncate(result),
        })

raise AgentStepLimitExceeded(step)      # loudly, never a partial answer

Four details there separate working code from a demo, and each is load-bearing:

• MAX_STEPS. Without it, a confused agent runs until the bill notices.
• truncate(result). One unbounded tool result can consume the whole window.
• Append the assistant message before executing. If the process dies mid-tool, you still have a record of what it intended.
• Fail loudly on exhaustion. Returning a partial answer hides the failure from the caller and from your metrics.

The mechanism: the message protocol

This is the part frameworks hide and that you must understand the moment you manage the transcript yourself.

An assistant message containing n tool calls must be followed by exactly n tool results, each carrying the matching id. It is not a convention — the providers validate it:

messages = [
    {"role": "user", "content": "where is order 1042"},
    {"role": "assistant", "tool_calls": [
        {"id": "call_a", "name": "get_tracking", "args": {...}},
        {"id": "call_b", "name": "get_customer",  "args": {...}},
    ]},
    {"role": "tool", "tool_call_id": "call_a", "content": "delivered"},
    # call_b's result is missing
]

Send that and you get a 400 before the model sees anything:

BadRequestError: an assistant message with 'tool_calls'
must be followed by a tool message for each
'tool_call_id'. Missing: 'call_b'.

Even a failed tool needs a result. Swallowing the exception and appending nothing breaks the sequence; append the error text as the result instead and let the model react to it. Models recover from “connection refused” surprisingly well when they can see it.

Why you cannot truncate a transcript arbitrarily

The pairing has a consequence that catches every team that writes its own context management: messages are not independently droppable.

# Looks reasonable. Produces a 400.
messages = [messages[0]] + messages[-10:]

If that window starts in the middle of a tool exchange, you have kept a tool result whose assistant call is gone, or an assistant call whose result is gone. Either way the sequence is invalid.

Drop in pairs — an assistant message and all of its tool results move together, or none of them do:

def compact(messages, keep_last=10):
    """Trim history without splitting a tool-call group."""
    head, tail = messages[:1], messages[1:]
    cut = max(0, len(tail) - keep_last)
    # walk forward to the next assistant turn that opens a clean group
    while cut < len(tail) and tail[cut]["role"] == "tool":
        cut += 1
    return head + tail[cut:]

That is the smallest correct version. Anything more sophisticated — summarisation, selective eviction — has to respect the same invariant.

The economics: cost is quadratic in steps

The loop appends and re-sends. Step 1 sends the prompt; step 2 sends the prompt plus the first exchange; step k sends everything before it.

step 1:  2,000 tokens
step 2:  2,000 + 800          =  2,800
step 3:  2,000 + 800 + 1,200  =  4,000
...
step 10: ~18,000
         total ≈ 95,000 input tokens, 10 steps

Total input tokens grow as O(n²) in the number of steps, not linearly. An agent that takes twenty steps instead of ten does not cost twice as much; it costs roughly four times as much. This is why step count is a cost metric and not just a latency metric, and why “let it think a bit longer” is an expensive instruction.

Two things bend the curve:

• Prompt caching. The transcript is a growing prefix, which is the ideal shape for it — each step re-sends a prefix the provider has already seen. It turns most of that quadratic into cache reads at a fraction of the price. It does not reduce the token count, so context limits still bite.
• Not rewriting history. Summarising old turns saves tokens and invalidates the cache from the point of the edit, so the next call re-prefills everything after it. A compaction that saves 20% of tokens can cost more than it saves. Measure both numbers before shipping it.

See KV cache.

Termination

An explicit finish(answer) tool is underrated. It turns “I am done” into an action the model takes rather than an absence you infer, and it lets you enforce a schema on the final answer.

Loop detection

The most common runtime failure is an agent repeating a call because the result was not what it hoped for.

key = (call.name, json.dumps(call.args, sort_keys=True))
seen[key] += 1

if seen[key] > 2:
    messages.append({
        "role": "tool", "tool_call_id": call.id,
        "content": (
            f"You have called {call.name} with these arguments {seen[key]} "
            f"times and received the same result. Try a different approach, "
            f"or explain what is blocking you."
        ),
    })
    continue

Feed the detection back rather than aborting. The model very often recovers when told it is stuck; aborting converts a recoverable situation into a failure. Note this still appends a result for the id — the protocol does not care that you generated the content yourself.

Parallel tool calls

A model can return several calls in one response, and running them concurrently is usually free latency:

async with asyncio.TaskGroup() as tg:
    tasks = {c.id: tg.create_task(execute(c)) for c in response.tool_calls}

Two cautions. Tools with side effects may not be safe to interleave — a send_email and a cancel_order issued together have an order you did not choose. And every call still needs its result appended, including any that raised, or you are back to an invalid sequence. See TaskGroup and structured concurrency.

Where it breaks in production

The tool that returns a database. A search tool with no limit returns 40,000 rows, and one observation consumes the entire window. The agent cannot recover because it cannot see past its own tool output. Bound results at the tool, not after: return the top 20 and a count, never everything.

The step limit tuned by vibes. Too low and legitimate multi-step tasks fail at the limit; too high and a stuck agent burns budget quadratically before anyone notices. Set it from the observed distribution — if p95 of successful runs is 6 steps, a limit of 12 is generous and 50 is an accident waiting.

Silent partial success. The agent hits the limit having already sent the email and updated the record. Aborting does not undo those. If steps have side effects, the loop needs the same treatment as any partial failure: idempotent tools, or a compensating action — see Idempotency keys.

No per-step trace. An agent without tracing is undebuggable, because the only artefact of a wrong answer is the answer. Log the model call, its inputs and outputs, every tool name, arguments, result, latency, tokens and cost, per step — see LLM observability.

Planning variants

Plan-and-execute earns its place when the plan should be reviewable before anything executes, which turns an opaque process into something a human can approve.

The trade not taken

Most tasks called “agentic” are workflows: a fixed sequence with one model call per stage. A workflow is cheaper, has linear cost, is trivially testable, and fails predictably.

The loop earns its cost only when the number of steps genuinely cannot be known in advance. If you can draw the flowchart, write the flowchart — and say so in an interview, because reaching for an agent where a for loop would do is the most common judgement error in this area.

Related

• Agent harnesses - everything around the loop
• What is an agent - whether you need one at all
• Context engineering - compaction done properly