Multi-Agent Systems: When & When Not
The hype-free take on multi-agent systems: orchestrator-worker vs swarms, the coordination tax, when one well-tooled agent wins, and the anti-patterns.
Your bar: decide whether a problem actually warrants more than one agent, choose orchestrator-worker vs swarm when it does, and name the failure modes that only appear once agents must coordinate. By the end you can answer the production question every PM eventually asks: should this be a fleet of agents, or one well-tooled agent in a loop? 1
The advanced, cost-aware sequel to the AI Agents track’s single-vs-multi intro — here we add the trade-off math and the anti-patterns.
The whole skeptical case fits in four moves. Each chip is one of them:
Default to one well-tooled agentadd agents only for genuinely parallel workprefer orchestrator-worker over emergent swarmsand pay the coordination tax only when it pays you back
1
Multi-Agent Systems
Multiple agents — each its own LLM + tools + context window — working on one goal, coordinating by passing messages or results. It’s the distributed-systems version of an agent: more throughput on parallel work, paid for with coordination.2
One agent, or many? Start from the default
When does the work justify more than one agent?
| Property of the task | Single agent | Multi-agent |
|---|---|---|
| Shape of the work | Sequential / one thread | Genuinely parallel, independent sub-tasks |
| Shared context | Sub-steps share a lot of state | Sub-tasks need little of each other’s state |
| Scale | Small enough to not amortize overhead | Large enough that the coordination tax pays back |
| What you optimize | Cost, latency, debuggability | Wall-clock throughput on parallel work |
| Honest default | Most production features | The exception you must justify |
Is A multi-agent system is several agents (each its own LLM + tools + context) on one goal, coordinating by passing messages or results. A scaling pattern, not a smarter brain.
Why it exists Some goals decompose into independent sub-tasks that can run in parallel. Splitting them across agents turns serial work into concurrent work — when the work is actually parallel.
Like (world) A newsroom on a breaking story: an editor assigns reporters to separate angles in parallel, then stitches the pieces. It only helps because the angles are independent.
Like (code) A monolith vs microservices: you split for independent scaling, not for fun. The split pays only when the boundary is real and the volume justifies the network cost.
✗ “More agents make the system smarter.”
✓ More agents add parallel hands, not a smarter brain. A fleet of weak reasoners doesn’t out-think one strong agent on a sequential problem.
✗ “Multi-agent is the modern, default way to build.”
✓ The default is one well-tooled agent in a loop. Multi-agent is the exception you reach for only when the work is genuinely parallel.
📥 Memory rule: One well-tooled agent is the default. A multi-agent system is the distributed-systems version of an agent — reach for it only when the work is genuinely parallel.
Memory check
- What is a multi-agent system in one line? → several agents (each LLM + tools + context) on one goal, coordinating by passing messages/results
- What does adding agents actually buy you? → parallel throughput on independent work — not more intelligence
- What's the honest default? → one well-tooled agent in a loop; multi-agent is the exception you must justify
M1 — A PM wants you to “use a multi-agent system” for a feature. How do you decide whether that’s the right shape at all?
Hit these points: start from the default — one well-tooled agent in a loop is simpler, cheaper, and easier to debug, so the burden of proof is on adding agents → ask whether the work is genuinely parallel and decomposable into independent sub-tasks that don’t need each other’s intermediate state → if it’s sequential, or the sub-tasks share a lot of context, multiple agents just pay coordination tax (extra tokens, latency, failure surface) for no parallelism → if it IS parallel and the value of the result outweighs the spend, an orchestrator-worker fan-out can pay off → the rule: don’t add agents for the same reasons you wouldn’t add microservices — premature distribution buys you a distributed-systems problem you didn’t have.
2
Orchestration: Orchestrator-Worker vs Swarm
Two ways to wire a fleet. Orchestrator-worker: one lead decomposes, delegates, and synthesizes (scatter-gather). Swarm: peer agents, no central planner, behavior emerges. For production you almost always want the control point.3
The two patterns, side by side
What each pattern buys and costs
| Dimension | Orchestrator-worker | Swarm |
|---|---|---|
| Control | Central lead plans & integrates | None — peers act on local state |
| Predictability | High — one plan, one synthesis | Low — behavior emerges |
| Failure attribution | Clear — trace to a worker or the join | Hard — no single owner of an outcome |
| Parallelism | Bounded by the lead’s fan-out | Maximal, fully decoupled |
| Best for | Production tasks needing an audit trail | Exploration / simulation where emergence is the point |
Is Orchestrator-worker = a lead agent decomposes the goal, delegates sub-tasks to workers, and synthesizes results. Fan-out-fan-in with one owner of planning and integration.
Why it exists Production needs a control point: somewhere to plan the split, validate the results, and attribute failures. The lead is that point; workers stay isolated on their sub-problem.
Like (world) A general contractor: subcontractors do isolated jobs in parallel; the contractor scopes the work, checks it, and signs off the whole build.
Like (code) Scatter-gather / map-reduce: a coordinator fans work to mappers and reduces their outputs. The reduce step is the synthesis — and the place bad partials get caught.
✗ “A swarm is more advanced, so it’s the better default.”
✓ A swarm trades away the things production needs most — predictability and failure attribution — for emergence you rarely want. Default to orchestrator-worker.
✗ “The orchestrator is just a router; the workers do the real work.”
✓ The orchestrator owns planning, delegation, AND synthesis. The synthesis (and validating partials) is where quality is won or lost.
📥 Memory rule: Orchestrator-worker gives you one control point — plan, validate, attribute. A swarm trades that for emergence you rarely want in production.
Memory check
- What does the orchestrator own? → planning the split, delegating to workers, and synthesizing their results
- What does a swarm trade away? → predictability and failure attribution — there's no central plan or owner of the outcome
- Which is the production default? → orchestrator-worker — you want the audit trail and the synthesis step
M2 — Contrast orchestrator-worker with a swarm. When would you reach for each, and what do you give up?
Hit these points: orchestrator-worker = one lead agent decomposes the goal, fans out independent sub-tasks to workers, then synthesizes — scatter-gather, the lead owns planning and integration → swarm = many peer agents, no central planner; behavior emerges from local interactions and shared state → orchestrator-worker buys a clear control point, easier debugging, and a place to attribute failures; you pay a planning bottleneck and a single coordination point → swarm buys maximum parallelism and decoupling; you give up predictability and the ability to attribute a bad outcome to any one agent → default to orchestrator-worker for production: you almost always want the audit trail and the synthesis step more than emergent behavior.
3
Failure Modes & Anti-Patterns
Three failures appear only once agents must coordinate: cascading errors, context fragmentation, and cost / latency blowups. Each is a distributed-systems failure wearing an LLM costume — and each is why you bound a fleet like one.4
The three multi-agent-specific failures
Failure mode → root cause → defense
| Failure mode | Root cause | Defense |
|---|---|---|
| Cascading errors | A wrong partial is trusted and folded into the synthesis | Validate / score worker outputs at the join; reconcile, don’t blend |
| Context fragmentation | Each agent sees a slice; no one sees the whole | Decide shared vs isolated context up front; orchestrator owns the global plan |
| Cost / latency blowup | Tokens scale with agents × turns × retries; unbounded fan-out | Hard caps on agents/turns/budget; per-agent token attribution; kill switch |
| Excessive agency | Autonomous agents act with broad tool access | Least privilege, no arbitrary egress, confirmation on irreversible actions |
Is The multi-agent failure set: cascading errors, context fragmentation, and cost/latency blowup — failures that exist only because agents must coordinate. Add excessive agency once they can act.
Why it exists Coordination introduces joins (where bad partials compound), boundaries (where context fragments), and amplification (where tokens multiply). Distribution creates them; a single agent has none.
Like (world) A relay race: one dropped baton (cascading), runners who don’t know the course (fragmentation), and a roster you keep growing for no speed gain (cost blowup).
Like (code) Microservice sprawl: cascading failures across calls, state split awkwardly across services, and a bill that balloons with chatty inter-service traffic. Same lessons apply.
✗ “We’ll add specialist agents because that’s how the impressive demos work.”
✓ Demos optimize for looking smart on a curated task; production optimizes for cost, latency, debuggability, and an audit trail. Don’t generalize from the demo.
✗ “Adding agents is a low-risk way to improve quality.”
✓ Each agent is a new failure surface. You inherit cascading errors, fragmentation, and cost blowups — a distributed-systems problem — for parallelism you may not need.
📥 Memory rule: Don’t add agents for the reasons you wouldn’t add microservices. Bound a fleet like a distributed system: validate at joins, cap the blast radius, attribute spend.
Memory check
- Name the three multi-agent-specific failures. → cascading errors, context fragmentation, cost/latency blowup
- What is a cascading error? → one worker's wrong partial is trusted downstream and folded into a confident wrong synthesis — fix by validating at the join
- What's the governing analogy? → don't add agents for the reasons you wouldn't add microservices — distribute only when the boundary is real and the volume pays the tax
M3 — Your multi-agent feature’s quality is unstable and the bill is 10x what you modeled. Diagnose the failure modes.
Hit these points: separate the three multi-agent-specific failures → cascading errors: a worker’s wrong intermediate result is trusted downstream and the orchestrator synthesizes confident nonsense — fix with validation at the join, not blind trust → context fragmentation: each agent sees a slice, no one sees the whole, so they duplicate work or contradict each other — fix by deciding shared vs isolated context up front → cost/latency blowup: tokens scale with agents × turns × retries; one chatty loop or a runaway fan-out multiplies spend fast (multi-agent can burn many times the tokens of a single chat turn, illustrative) — fix with hard caps on agent count, turns, and budget → measure: per-agent token attribution, a turn cap, and a kill switch; if you can’t attribute the spend, you can’t control it.
M3 — Make the explicit analogy: why is “don’t add agents for the reasons you wouldn’t add microservices” the governing rule?
Hit these points: splitting a monolith into microservices trades in-process calls for network calls — you gain independent scaling and team autonomy but inherit partial failure, serialization cost, and distributed debugging → splitting one agent into many trades in-context reasoning for message-passing — you gain parallelism but inherit coordination overhead, cascading errors, and context fragmentation → in both cases the split only pays when the boundary is real (genuinely independent work) and the volume justifies the overhead → people add both for resume-driven or demo reasons, not load reasons; the result is a distributed system with all the failure modes and none of the benefit → so the test is identical: is the work truly independent and at a scale where the coordination tax pays for itself? If not, keep it in one process / one agent.
Retrieval practice — test the three modules
Q1. Adding more agents to a system primarily gives you…
- A model that reasons more deeply because several brains debate the problem first
- Lower cost and latency since the work is now spread thin across many helpers
- Parallel throughput on independent work, paid for with coordination overhead
- Stronger safety guarantees because no single agent holds all of the context
Q2. The honest default shape for most production agent features is…
- One well-tooled agent in a tight loop, with multi-agent kept as the exception
- A swarm of specialist agents so each narrow skill has a dedicated owner
- An orchestrator with as many workers as there are sub-questions to answer
- Whichever shape the most impressive recent product demo happened to use
Q3. Compared with a swarm, orchestrator-worker mainly gives you…
- Strictly more parallelism because the lead can spawn unlimited extra workers
- Lower token cost in every case since a central lead removes all the chatter
- Emergent behavior that adapts on its own without any central planning step
- A central control point to plan, validate, and attribute failures cleanly
Q4. A worker returns a wrong partial and the lead folds it into a confident final answer. This is…
- Context fragmentation, because the worker could not see the whole problem set
- A cascading error, fixed by validating worker outputs before the synthesis step
- A cost blowup, fixed by lowering the per-call token budget for that one worker
- Excessive agency, fixed by removing the tool that the worker used to act outward
Q5. The governing rule "don't add agents for the reasons you wouldn't add microservices" means…
- Multi-agent and microservices are unrelated, so the comparison does not hold up
- Always split work into the smallest possible agents to keep each one simple to test
- Distribute only when the boundary is real and the scale repays the coordination tax
- Microservices are obsolete, so avoid both patterns in any modern system you build
Interview — pick your bar
Answer out loud in ~60s, then reveal. Core = recall · Senior = trade-offs & failure modes · Staff = synthesis under ambiguity · System Design = open design round (a different axis, not a harder level).
What is a multi-agent system, in one breath?
Hit these points: multiple agents — each its own LLM + tools + context window — working on one goal and coordinating by passing messages or results → it's the distributed-systems version of an agent: you trade a single reasoning thread for several that must agree → the upside is throughput on genuinely parallel work; the downside is coordination overhead — the tokens, latency, and failure surface spent getting them to hand off and not duplicate work → so it is a scaling pattern, not a smarter brain: more agents mean more parallel hands, not more intelligence.
What is orchestrator-worker and what does the orchestrator actually own?
Hit these points: one lead (orchestrator) agent decomposes the goal into independent sub-tasks, delegates each to a worker, then synthesizes the results into the final answer → it's the scatter-gather / fan-out-fan-in pattern → the orchestrator owns planning (how to split), delegation (who does what), and integration (combining results) — workers own only their isolated sub-problem → that single control point is the win: one place to plan, one place to validate, one place to attribute a failure.
What is an agent swarm, and what does it trade away?
Hit these points: many peer agents with no central orchestrator; behavior emerges from local interactions and a shared environment or state → it maximizes parallelism and decoupling — no planning bottleneck, agents act on what they see locally → the trade is predictability and attribution: with no central plan you can't easily say why the system did what it did, or which agent caused a bad outcome → so swarms suit exploratory or simulation-style work where emergence is the point, not production tasks that need an audit trail and a synthesis step.
What is coordination overhead, and why is it the central anti-pattern of multi-agent?
Hit these points: coordination overhead = the extra tokens, latency, and failure surface spent getting agents to agree, hand off, and not duplicate work → every multi-agent design pays this tax whether or not the task benefits → if the task is genuinely parallel, the throughput gain can exceed the tax; if it's sequential or context-heavy, the tax is pure loss → that's the anti-pattern: adding agents to a task that isn't parallel buys coordination cost with no payoff → the discipline is to keep the default at one agent and make the multi-agent design justify the tax.
When does a single well-tooled agent beat a fleet? Walk me through the call.
Hit these points: a single agent wins whenever the work is sequential, shares a lot of context, or isn't large enough to amortize coordination cost → one agent keeps all state in one context window, so there's no fragmentation, no handoff loss, and one place to debug → it's cheaper (no inter-agent chatter) and lower-latency (no fan-out / join) → the honest default: most production features are a single agent with good tools and a tight loop → reach for multiple agents only when you've shown the work is genuinely parallel and the value of the outcome outweighs the extra spend — separate that real case from the demo that just looks impressive.
Your fan-out spawned far more workers than the task needed and the bill exploded. What happened and how do you prevent it?
Hit these points: the orchestrator over-decomposed — it spun up a worker per trivial sub-question because nothing bounded it → tokens scale with agents × turns × retries, so an unbounded fan-out multiplies spend with no quality gain → root cause is missing limits, not a model bug: no cap on worker count, no per-task budget, no floor on what's worth delegating → fix: cap concurrent workers, set a token/turn budget per run with a kill switch, and only delegate sub-tasks above a complexity threshold → add per-agent token attribution so you can see which branch ran away → the lesson: an autonomous decomposer needs the same guardrails you'd put on any process that can fan out work.
Quality swings run to run and you suspect cascading errors. Diagnose and fix.
Hit these points: log each worker's intermediate output and the orchestrator's synthesis for a good and a bad run — you're auditing the handoffs, not the final answer → look for a worker that returned a wrong or low-confidence result that got trusted downstream and folded into a confident final answer → that's a cascading error: no validation at the join, so one bad slice poisons the synthesis → fix: validate or score worker outputs before integration, have the orchestrator reconcile disagreements instead of blending them, and let workers signal low confidence → add a check that contradictory results trigger a re-run or escalation, not a silent merge → the framing: in a pipeline of probabilistic steps, error compounds — you must inspect at the joins.
Two workers duplicated each other's work and a third contradicted both. Why, and how do you design against it?
Hit these points: this is context fragmentation — each agent saw only its slice, none saw the whole, so they overlapped and disagreed → root cause is an unmade decision: what context is shared across agents vs isolated to each → too much isolation duplicates and contradicts; too much sharing reintroduces the cost and bloat you split to avoid → fix: give the orchestrator authority over the global plan so sub-tasks are carved to be genuinely non-overlapping, share the minimal common context (the goal, constraints, what's already done), and isolate the rest → have the synthesis step reconcile, not concatenate → the principle: decide the context boundaries deliberately, the way you'd define service boundaries — fragmentation is a design smell, not bad luck.
Reason about the trade-offs across coordination cost, latency, and reliability in a multi-agent design.
Hit these points: name the axes — cost (tokens scale with agents × turns), latency (fan-out can run workers in parallel, but the join waits for the slowest, and an orchestrator adds planning turns), reliability (more independent steps means more places to fail and compound) → parallelism can cut wall-clock latency for genuinely independent work, which is the main reason to pay the rest → but it raises cost (inter-agent chatter, retries) and lowers reliability (cascading errors, fragmentation) → sequence the call: prove the work is parallel first; if not, one agent dominates on all three axes → if it is, bound the blast radius — cap agents/turns/budget, validate at joins, attribute spend per agent → measure parallel speedup against the cost multiple; if you're not getting the speedup, you're paying the tax for nothing.
A teammate wants every feature to be a swarm of specialist agents "because that's how the smartest demos work." Tear it apart as a system design.
Hit these points: two flawed assumptions → "more agents = smarter" is false: agents add parallel hands, not intelligence; a swarm of weak reasoners doesn't out-think one strong agent on a sequential problem → "demos generalize" is false: demos optimize for looking impressive on a curated task; production optimizes for cost, latency, debuggability, and a clean audit trail → a swarm gives up what production needs most — predictability and failure attribution — for emergence you rarely want → map the failure modes you'd inherit: coordination cost, cascading errors, context fragmentation, cost blowup → the framing, same as microservices: don't distribute until the work is genuinely independent and at a scale that pays the tax; default to one well-tooled agent and make the fleet justify itself with measured parallel speedup, not vibes.
Multi-agent demos look magical; production multi-agent often disappoints. Separate the real capability from the demo.
Hit these points: a demo is a curated happy path — one impressive trace on a task chosen to show emergence; it hides cost, tail latency, and the runs where coordination went wrong → production is the distribution, not the best trace: you pay for the average and the tail, you debug the failures, and you carry the bill every call → the real, durable capability is narrow: genuinely parallel work (gather from many independent sources, fan out independent checks) where the orchestrator can synthesize and validate → the fake capability is "more agents = smarter on a sequential problem" — that just adds coordination tax and failure surface → the test: does the fleet beat a single well-tooled agent on the metric that matters, measured, with the failure modes bounded? If you can't show that, you have a demo, not a system.
Design-round framework — narrate a multi-agent decision in this order so the interviewer hears default-first, then justification, then guardrails:
- Clarify scope: is the work genuinely parallel? How much shared context? What tools can act, and what's the cost of a wrong action vs a slow answer?
- State the default: one well-tooled agent in a loop — cheaper, lower-latency, one place to debug. Make the fleet justify replacing it.
- If parallel, choose the pattern: orchestrator-worker for a control point + audit trail; swarm only when emergence is the actual goal.
- Bound the blast radius: caps on concurrent agents, total turns, and per-run token budget, with a kill switch.
- Defuse the failure modes: validate at joins (cascading errors), define shared vs isolated context (fragmentation), per-agent token attribution (cost blowup).
- Safety for autonomy: least privilege on tools, no arbitrary egress, confirmation on irreversible actions — map to OWASP LLM Top 10 (excessive agency).
- Measure vs the single-agent baseline: parallel speedup against the cost multiple; if the fleet doesn't win on the metric that matters, collapse it back.
Design the architecture for a research assistant that must answer a broad question by gathering from many independent sources.
A strong answer covers: clarify whether the work is genuinely parallel — many independent sources to gather is the textbook case where fan-out pays, so this is a fair multi-agent candidate → choose orchestrator-worker over swarm: a lead agent plans the sub-questions, fans out a bounded set of worker agents to gather in parallel, then synthesizes — you keep one control point, an audit trail, and a place to validate → bound the blast radius: cap concurrent workers, set a per-run token/turn budget with a kill switch, only delegate sub-tasks above a complexity floor so you don't over-decompose → handle the failure modes: validate worker results at the join (cascading errors), share the goal + what's gathered and isolate the rest (context fragmentation), attribute tokens per worker (cost blowup) → compare honestly against the single-agent baseline: if one agent with a search tool and a tight loop gets close, ship that → measure parallel speedup vs the cost multiple, and treat the multi-agent design as something to justify, not assume (multi-agent token use is many times a single chat turn, illustrative).1
Design the guardrails and observability for a multi-agent system you have to run in production.
A strong answer covers: one principle drives the design — every multi-agent system is a distributed system, so bound it like one → limits: hard caps on concurrent agents, total turns, and per-run token budget, with a kill switch that stops a runaway fan-out → attribution: per-agent / per-task token and latency accounting so you can see which branch ran away and what the parallel speedup actually was → validation at the joins: score or check worker outputs before the orchestrator synthesizes, reconcile disagreements instead of blending them (defuses cascading errors) → context discipline: define shared vs isolated context explicitly so agents don't duplicate or contradict (defuses fragmentation) → safety: autonomy raises blast radius, so least privilege on tools, no arbitrary egress, confirmation on irreversible actions — map to OWASP LLM Top 10 (excessive agency) → measure against a single-agent baseline; if the fleet isn't beating it on the metric that matters, collapse it back to one agent.5