Agent Handoffs and Supervisor Patterns: Who Decides What Runs Next

The moment you split one agent into several, you inherit a question single-agent systems never ask: who decides what runs next? Every multi-agent framework answers it the same way underneath, and the answer is less exotic than the architecture diagrams suggest. A handoff is a tool call. The model does not "pass control" through some special channel; it calls a tool named something like transfer_to_refund_agent, and your runtime interprets that call as a routing decision.

What is a handoff, mechanically?#

It is a tool the model can call whose effect is routing rather than computation. The model sees a normal function in its tool list, decides the request belongs elsewhere, and calls it. The runtime intercepts that call and starts a different agent instead of returning a result to the same one.

That framing explains most handoff bugs. If an agent never hands off when it should, the problem is usually the tool description, exactly as it would be for any other underused tool. If it hands off constantly, the description is too inviting or the agent's own instructions are too narrow. You are debugging tool selection, not some separate coordination layer, which is why the reliability work on structured outputs and tool calling applies here directly.

Nothing here is a special coordination channel. It is tool selection plus an interception in your runtime.

LangGraph reaches the same place from a different direction. It models agents as graph nodes, and a node returns a Command object that combines control flow and a state update in one move. A handoff there specifies a destination, meaning the agent to navigate to, and a payload, meaning the information travelling with it. Different vocabulary, same 2 ingredients: where next, and with what.

Supervisor or swarm: who decides where control goes?#

Two shapes dominate. In a supervisor system, one coordinating agent holds the routing decisions and workers report back to it. In a swarm, every agent can hand off directly to any other, so routing is distributed. The difference is whether control returns to a center after each step.

The single variable is whether control returns to a center after each step. Everything else follows from it.

LangGraph documents both, and ships them as separate libraries: a supervisor implementation where the supervisor hands off to individual agents, and a swarm implementation where an individual agent can hand off to other agents. Its guidance for reaching for the supervisor pattern is concrete: multiple distinct domains such as calendar, email, CRM, or database, each with multiple tools or complex logic, where you want centralized workflow control and sub-agents do not need to converse with users directly.

Supervisors are easier to reason about and easier to instrument, because every transition passes through 1 place you can log. They also become a bottleneck, since each hop costs an extra model call from the supervisor to decide the next move. Swarms remove that toll and remove the single point of observation with it, which tends to be a bad trade until you have a specific latency reason to make it.

For deeper trees, LangGraph notes you can build multi-level hierarchies by creating a supervisor that manages other supervisors. That works, and it multiplies the coordination overhead at every level, so it earns its place only when the domains genuinely nest.

There is a third shape worth naming because it is often mislabelled as multi-agent: the subagent-as-tool pattern. Here the "worker" never receives control at all. The coordinating agent calls it like any other function, gets a result back, and keeps the conversation. Nothing is handed off, so there is no routing decision to debug and no shared history to filter. It is strictly simpler, and it covers a surprising share of what teams build supervisors for. The distinction that matters is whether the second agent needs to keep talking to the user or make further routing decisions of its own. If it just needs to answer a question and return, it is a tool, and calling it an agent adds vocabulary rather than capability.

Choosing between these is mostly a question of who owns the conversation. A supervisor owns it and lends it out. A swarm passes ownership around. A tool-shaped subagent never gets it. Ranked by how hard they are to reason about when something goes wrong, that ordering runs in reverse.

What actually gets passed when control moves?#

Whatever you choose to pass, which is the part teams get wrong. By default a receiving agent may inherit the entire conversation so far, including tool outputs and reasoning that have nothing to do with its job. Filtering that inheritance is an explicit decision, not a framework default you can ignore.

The OpenAI SDK exposes 3 controls for exactly this. An input_filter receives a HandoffInputData object holding the conversation history and generated items, and returns what the next agent should actually see. An input_type defines a schema for the handoff tool call's arguments, so the handing-off agent must state its payload explicitly. An on_handoff callback fires when the handoff triggers, useful for fetching data or logging. Where both a per-handoff filter and a run-level filter exist, the SDK documents the per-handoff one taking precedence.

Each item is a default worth overriding. Inheriting everything is the expensive path.

Unfiltered inheritance is expensive twice over. It costs tokens on every hop, and it degrades the receiving agent's focus by burying its actual instruction under someone else's transcript. That is the same pressure described in context engineering for agents, arriving at a specific architectural seam where you have a clean place to act on it.

What does the coordination actually cost?#

Roughly an order of magnitude in tokens. Anthropic reports that agents typically use about 4 times more tokens than chat interactions, and that multi-agent systems use about 15 times more tokens than chats. That multiple is the real budget question behind any topology decision.

The same write-up is where the strongest published result for the pattern comes from. Anthropic describes an orchestrator-worker system in which a lead agent coordinates while specialized subagents work in parallel, and reports that a system with Claude Opus 4 as lead and Claude Sonnet 4 subagents outperformed single-agent Claude Opus 4 by 90.2% on their internal research evaluation. Source

All 4 figures come from Anthropic's own write-up. The gain is domain-specific; the overhead is structural.

Both numbers deserve to be read together, and read carefully. This is a vendor evaluating its own system on its own internal benchmark, in the research-and-search domain where parallel exploration is unusually well suited to being split up. A 90.2% gain does not transfer automatically to a customer-support router or a data-entry pipeline. The 15 times token figure, on the other hand, transfers to almost everything, because coordination overhead is structural rather than domain-specific.

How many agents should a task get?#

Fewer than the diagram tempts you to draw. Anthropic's own scaling guidance is that simple fact-finding needs just 1 agent, direct comparisons might need 2 to 4 subagents, and complex research might use more than 10, with its typical case spinning up 3 to 5 in parallel.

That is a useful calibration because it anchors on the shape of the work rather than the number of tools. A common mistake is giving every tool its own agent, which produces a system where most hops exist to move a request past an agent that had nothing to contribute. Agents should map to domains that need genuinely different instructions or different context, not to individual capabilities.

Parallelism is the one place where extra agents pay for themselves cleanly. When subtasks are independent, running them concurrently converts token spend into wall-clock savings. When subtasks are sequential, more agents just add hops, and you would often be better served by decomposing the task inside 1 long-horizon agent than by splitting it across several.

When is a single agent still the right answer?#

Whenever the work fits in 1 context window and does not branch into independent subtasks. Multi-agent systems buy parallelism and instruction isolation. If a task needs neither, the extra agents add token cost, latency, and failure modes while delivering coordination you were not short of.

Each check removes a topology. Most systems stop before they reach the distributed end of the chart.

The honest default is to start with 1 agent and split only when you can name what the split fixes. Good reasons exist and they are specific: the instructions for 2 parts of the job genuinely conflict, subtasks are independent enough to run at the same time, or 1 stage needs a different model tier than another. Vague reasons, such as wanting the architecture to look modular, tend to produce systems that cost 15 times as much to do the same work.

It also helps to remember what the pattern does not fix. Splitting into agents does not make an unreliable tool call reliable, does not repair a retrieval layer that returns the wrong documents, and does not create durability, which is a property of the runtime you execute on rather than of your topology. Before adding a supervisor, it is worth confirming the single-agent version fails for a reason a supervisor would actually address, and instrumenting it with proper agent tracing is usually how you find out.