Back to Fundamentals: What Is an Agent, Really
To understand multi-agent systems, you first have to take the word "agent" apart completely — because every design decision in multi-agent architecture is a response to the fundamental limits of a single agent.
An LLM by itself is a stateless function: it takes a sequence of tokens in and emits a probability distribution over the next token, over and over. It remembers nothing between calls — what feels like "conversation memory" is simply the entire history being re-fed into the input every time. That input is the context window: the model's one and only working memory, finite in size (mainstream models today range from 200K to 1M tokens), and one that must be fully re-read on every call.
An agent wraps a loop around this stateless function. What the loop does is entirely mechanical:
┌─────────────────────────────────────────────┐
│ Agent Loop │
│ │
│ context ──► LLM ──► output │
│ │ │
│ ┌────────┴────────┐ │
│ │ a tool call? │ │
│ └────────┬────────┘ │
│ yes│ │no │
│ ▼ ▼ │
│ run the tool done — │
│ (query DB, reply to │
│ search, code) the user │
│ │ │
│ append result back to context │
│ │ │
│ └──────► back to LLM │
└─────────────────────────────────────────────┘
A tool call here is not magic either: the model has been trained so that when it needs an external capability, it emits a specially formatted piece of structured text (e.g. {"name": "query_oracle", "input": {"sql": "..."}}). The outer program (the harness — the execution framework) parses that text, actually runs the corresponding code, appends the result back into the context as text, and hands the now-longer context back to the model for the next iteration. The model itself only ever does one thing — read text, write text; the "ability to act" is entirely conferred by the outer loop.
Once you see this loop clearly, the three fundamental limits of a single agent fall out naturally — and each one is a direct consequence of the mechanism:
First, the context window is a hard ceiling, and it gets duller as it fills. Every turn of the loop appends another tool result. One Oracle query returns three thousand rows; one web fetch returns twenty thousand tokens; after a few dozen turns the context is full. Worse, the attention mechanism (the computation by which the model decides, at each token, which earlier tokens to draw on) dilutes over very long inputs — key facts drown in the noise of tool results, and the model starts losing track of the original goal. This is not an engineering bug; it is a necessity of the mechanism.
Second, the loop is sequential. It does one thing at a time: plant B's data cannot be queried until plant A's is done; the second document waits for the first. Even when tasks are completely independent, wall-clock time adds up linearly.
Third, one prompt struggles to play several roles at once. You can write "you are simultaneously a security reviewer, a performance optimizer, and a technical writer" into a system prompt, but those roles have conflicting attentional priorities, and mixed into one context they interfere — the reviewer must nitpick, the writer must flow, and a single generation pass rarely serves both.
A multi-agent system is the structural answer to exactly these three limits: open more context windows, run them in parallel, and let each focus on one thing.
The Underlying Flow: Multi-Agent Is Just Recursive Tool Calling
A common assumption is that multi-agent systems require some special low-level machinery. They don't — and realizing this is the single most important step to understanding them: in the dominant implementations, "spawning a subagent" is itself just a tool call.
Look back at the agent loop: if the model can call query_oracle or web_search, then give it one more tool called spawn_agent, whose input is "a task description in text" and whose output is "the completed result of that task, in text" — from the orchestrator's point of view, a subagent is fundamentally indistinguishable from a query tool. The difference is only inside that "tool": when the harness receives a spawn_agent call, it starts a brand-new agent loop — a fresh, empty context window loaded with the subagent's own system prompt plus the task description the orchestrator wrote, equipped with the tools it needs, running its own loop until completion, with its final output fed back to the orchestrator as a tool result.
Orchestrator's context window
┌──────────────────────────────────────────────────┐
│ Task: analyze MES anomalies across three plants │
│ │
│ tool call: spawn_agent("investigate KH plant…") │──┐
│ tool call: spawn_agent("investigate SZ plant…") │──┼─ parallel!
│ tool call: spawn_agent("investigate MY plant…") │──┘
│ …waiting… │
│ tool result: "KH: 3 anomaly classes…" (2K chars) │◄─┐
│ tool result: "SZ: nominal, except…" (2K chars) │◄─┼─ summaries only
│ tool result: "MY: lot-genealogy gap…" (2K chars) │◄─┘
│ │
│ → synthesize the three summaries, write report │
└──────────────────────────────────────────────────┘
│ │ │
▼ ▼ ▼
Subagent A's Subagent B's Subagent C's
context window context window context window
(independent — (its 100K tokens (none of them
may hold 100K of intermediate can see that
tokens of raw work is used the others
query results) and discarded) exist)
This diagram contains the entire essence of multi-agent systems, and each point deserves unpacking:
Context isolation is the core payoff. To find the KH plant's anomalies, subagent A may run twenty Oracle queries and read a hundred thousand tokens of raw data — all of that "intermediate process" stays inside A's own context window, and only a two-thousand-character conclusion returns to the orchestrator. The orchestrator's context stays clean, holding only "task assignments" and "conclusions from each party." This effectively multiplies the system's working memory by N: three subagents each get 200K tokens of room to dig through raw data, while the orchestrator keeps its own full 200K for synthesis. A single agent attempting the same job would exhaust its context on the first plant alone.
Parallelism is the second payoff. The three spawn_agent calls can be issued in the same turn (the model emits multiple tool calls at once and the harness executes them concurrently), so the three plant investigations run simultaneously and wall-clock time collapses from "three units" back to "one." Anthropic's research system works exactly this way: a lead agent analyzes the query and plans strategy, then spawns multiple subagents that explore different aspects simultaneously — each independently iterating on searches, evaluating quality, identifying gaps, and returning findings to the lead agent for synthesis [1].
The cost must also be derived from the mechanism, not memorized as a number. Every subagent is a full agent loop, and every turn of a loop re-reads its whole context — so token consumption compounds. Anthropic's measured magnitudes: a single-agent task uses roughly 4× the tokens of a plain chat, and a multi-agent system roughly 15× [1]. This means multi-agent only pays off when the task's value covers that multiplier — a conclusion we will collect again when discussing where to use it.
The communication bandwidth bottleneck. Note that subagents cannot see each other: A doesn't know B exists, let alone what B queried. All coordination flows through the orchestrator, and the orchestrator only sees whatever summary each subagent chose to report. Summarization means loss — and this structural weakness is precisely what the next section on memory sharing tries to solve, and where the strongest argument against multi-agent originates.
How Memory Is Shared: Four Layers of Mechanism
"How do multiple agents share memory" sounds like one question, but underneath it are four distinct layers of mechanism, each with different bandwidth, latency, and cost. Confusing these layers is the most common mistake in multi-agent design.
Layer 1: Message passing — return values and handoffs. The most basic form of sharing is what we saw above: a subagent's final output becomes one tool result in the orchestrator's context. This is "active, compressed" sharing — the subagent decides what is worth reporting. A variant passes the full history on handoff: LangGraph's supervisor pattern, by default, hands the entire message history so far to the worker it delegates to [10], so the receiving agent sees all prior context — at the cost of starting with a heavily pre-loaded context. The essence of this layer is "copying context window contents": immediate, but expensive and one-directional.
Layer 2: Shared external storage — the blackboard. Let all agents read and write one external space: a shared filesystem, a scratchpad directory, a database table. Classical AI called this the blackboard architecture — the name comes from a group of specialists solving a problem around one blackboard: anyone with progress writes it up; anyone can read the current state to decide their next move. MetaGPT's shared message pool is the modern version: all agents publish structured messages into the pool, and other agents subscribe to messages relevant to their role (publish-subscribe: publishers don't address recipients; subscribers pick up by topic), preventing every agent from being flooded with everything [4]. The essence of this layer: "persistent, many-to-many, but pull-based" — an agent doesn't automatically know the blackboard changed; it must read it during its loop.
Layer 3: Shared state objects — the graph-framework approach. Frameworks like LangGraph build a multi-agent system as a state graph: the system maintains one typed state object (e.g. {messages: [...], current_plant: "KH", findings: {...}}); each agent is a node in the graph that reads the state and returns updates to it; the framework merges updates and routes to the next node based on the state's contents [10]. A handoff, in this model, is just "one tool call updated the routing field in the state." The difference from Layer 2: a blackboard is free-form text held together by convention; state is structured, schema'd, and merged by the framework — more controllable engineering-wise, at the price of flexibility.
Layer 4: Long-term memory — hierarchical storage across contexts and sessions. The first three layers live within "this one run"; for memory to outlive a single context window's lifetime, you need a memory hierarchy. MemGPT makes the analogy to operating-system virtual memory explicit: the context window is "main memory (RAM)," external stores (databases, vector stores) are "disk," and the agent itself decides via tool calls what to page in and out — important facts get written to external storage and retrieved back into context when needed [6]. Retrieval typically relies on vector search (embed text into vectors, recall by similarity — the same mechanism a pgvector-backed RAG pipeline uses to find documents). Anthropic's system uses this layer too: the lead agent saves its research plan to external memory before approaching the 200K-token limit, so even after truncation it can recover the plan and continue rather than losing the mission [1]. Generative Agents demonstrates the full form of this layer: each agent keeps a memory stream recording all experiences in natural language, retrieves them weighted by recency, importance, and relevance, and periodically reflects low-level memories into higher-level conclusions stored back into the stream — put 25 such agents in a sandbox town and social behaviors emerge, from spontaneously organizing a party to spreading news [5].
Layer Mechanism Bandwidth Latency Persistence Typical impl.
──────────────────────────────────────────────────────────────────────────────────────
1 Message pass tool result/handoff low(summary) immediate none subagent returns
2 Shared space blackboard/files medium poll-based per-run MetaGPT pool
3 Shared state typed state object medium per-step per-run LangGraph graph
4 Long-term mem ext. store+retrieval low(query) cross-run permanent MemGPT, mem stream
──────────────────────────────────────────────────────────────────────────────────────
Here we must honestly face the counterargument. Cognition (the makers of Devin) published a pointed essay, "Don't Build Multi-Agents," whose core claim is: actions carry implicit decisions. Two parallel subagents each make countless unstated micro-decisions — style, assumptions, naming — invisible to one another, and these decisions collide at merge time. Their example: two subagents were asked to build a game's background and its character; one produced a Mario-style background, the other a completely mismatched bird, and the final agent was left to reconcile two contradictory half-products. Their prescription: "share context, and share full agent traces, not just individual messages" — and for tightly coupled tasks, simply go back to a single-threaded agent [9].
This argument does not actually contradict Anthropic's practice; the two mark opposite sides of the same boundary. For read-heavy, decomposable tasks with independent branches (research, investigation, scanning), the coordination bandwidth needed between subagents is low, summary-level sharing suffices, and multi-agent wins decisively. For write-heavy, tightly coupled, decision-dense tasks (jointly developing the same codebase), what must be shared is the full set of implicit decisions — any summary drops critical assumptions — the communication bandwidth of multi-agent simply isn't there, and single-threaded is the correct call [1][9]. Remembering this boundary matters more than remembering any framework's API.
The Design Patterns of Multi-Agent Systems
With the two foundations in place — "spawn is a tool call" and "memory sharing has four layers" — the design patterns stop being a catalog of names and become permutations of a few mechanisms. Academic surveys usually organize the field along three axes — how agents are profiled, how they communicate, and how their capacities grow [8]; a more practical engineering cut is one question: who holds control? The patterns below are organized accordingly.
Orchestrator-workers / supervisor. Control is centralized: one lead agent decomposes the task, dispatches subagents, and synthesizes results; subagents never talk to each other, only to the lead. This is the architecture of Anthropic's research system [1] and the shape of LangGraph's supervisor pattern [10]. The strengths derive directly from the structure: coordination logic lives in one place, so it is observable and debuggable; subagent contexts are fully isolated, so parallelism is safe. The weaknesses come from the same structure: the lead agent is a single point of bottleneck — the quality of its task decomposition determines everything, and all information must pass through its context. Anthropic's hard-won lesson: the prompt-engineering effort for the lead agent centers on teaching it to write good task descriptions — each subagent needs a clear objective, output format, tool guidance, and boundaries, or subagents duplicate work and miss key angles [1].
Pipeline / prompt chaining. Control passes in sequence: A's output is B's input, B's output is C's input, with programmatic checkpoints allowed in between. This is the first workflow pattern in Anthropic's "Building Effective AI Agents" [2]. It trades parallelism for "every step standing on the complete result of the previous one" — right for tasks with inherent ordering (outline, then draft, then translate). MetaGPT takes the pipeline to its logical extreme: it encodes a software company's standard operating procedures (SOPs) into an agent assembly line — product manager writes the requirements doc, architect produces the design, engineer writes code, QA tests — with each role emitting structured documents rather than free-form chat, using documents as the interface between roles, which significantly suppresses cascading hallucination (a small error in one stage being amplified as fact by the next) [4].
Routing. A lightweight classification step determines the request type, then dispatches it to the corresponding specialist agent [2][10]. Mechanically, this is "spend one cheap call to buy prompt focus for all subsequent calls" — a support system routes refunds, technical issues, and sales inquiries to three separately tuned agents, each with a short, precise system prompt.
Parallelization. Two subtypes: sectioning — the task splits into independent chunks that run simultaneously (three plants each investigated separately); voting — the same task runs several times and results are aggregated by majority or union (three agents independently review the same code for security flaws; any single alarm triggers human review) [2]. Voting deserves one extra thought: its benefit comes from errors being uncorrelated across samples. If all agents use the same model and the same prompt, errors correlate strongly and voting's marginal value shrinks — which is why in practice each voter gets a different perspective prompt.
Evaluator-optimizer / critic loop. One agent generates; another criticizes against explicit criteria; the loop iterates until the bar is met [2]. Recall the earlier limit — "one prompt struggles to be both creator and reviewer" — this pattern turns that limit into the solution: the two roles live in two contexts; the critic never sees the creator's self-justification, only the artifact itself, and judges more coldly and effectively.
Group chat and role-playing. Control is decentralized: multiple agents take turns speaking in one shared conversation thread, with a selection mechanism (rule-based or itself an LLM) deciding who speaks next. AutoGen built a framework on this abstraction: everything is a "conversable agent," and humans are wrapped as just another agent who can interject at any time [3]. CAMEL studied the minimal form — can two role-playing agents (one instructing, one executing) complete tasks without step-by-step human guidance — and introduced inception prompting to prevent role drift (agents gradually forgetting who they are mid-conversation) [7]. The mechanical weakness of group chat is plain: everyone shares one message history, so context consumption grows quadratically with the number of turns (every new message is re-read by every agent on its next turn), and the quality of "who speaks next" decides whether the conversation converges or circles.
Handoff / swarm. Control transfers between peer agents: the current agent decides "this is outside my remit," calls a handoff tool that transfers control — along with conversation state — to a named colleague, who then faces the user directly [10]. The difference from supervisor: there is no central coordinator; routing decisions are distributed into each agent — more flexible, and harder to trace as a whole.
Cross-system interoperability (A2A). All the patterns above assume every agent lives in the same program. When agents belong to different companies and frameworks, a standard protocol is needed: the Linux Foundation–hosted A2A (Agent2Agent) protocol lets agents discover each other via "agent cards" (machine-readable capability manifests) and delegate work via standardized task objects — without exposing internal memory, tools, or proprietary logic [11]. It complements MCP: MCP standardizes how an agent connects to tools; A2A standardizes how an agent finds other agents.
Pattern Control Inter-agent comms Parallel Best for
────────────────────────────────────────────────────────────────────────────
orchestrator central (lead) summary returns ◎ breadth-first search
pipeline sequential structured docs/output ✕ ordered production lines
routing one-shot entry none — heterogeneous requests
parallelization central none (independent) ◎ splittable / votable work
evaluator-opt alternating artifact + critique ✕ explicit quality bars
group chat decentralized full shared history ✕ multi-perspective debate
handoff/swarm current holder state + history moves ✕ clear division of remit
────────────────────────────────────────────────────────────────────────────
Where It's Actually Used, and Where the Advantage Comes From
Gathering all the mechanics above, the advantage of multi-agent systems compresses into one sentence: it spends tokens to buy three things a single agent cannot buy — effective memory capacity, parallel time, and role focus. The real-world uses are exactly the scenarios that need those three most.
Deep research and due diligence (the most mature deployment). This is the textbook fit: the problem is naturally breadth-first — "investigate this supplier's financial, legal, and technical risk" splits into independent directions, each requiring heavy reading of raw material but only a conclusion in return. Anthropic's internal evaluations showed the multi-agent research system dramatically outperforming a single agent on such tasks, with 80% of the performance variance explained by token spend alone — multi-agent is, at bottom, an architectural device for productively spending more tokens on one problem: a single agent hits the context ceiling as you add tokens; multi-agent distributes tokens across several contexts and routes around that ceiling [1].
Large-scale code and data scanning. "Find every SQL-injection risk in the codebase," "inventory the deprecated API calls across 200 report programs" — each file's check is independent, read-heavy, and parallelizable: home turf for the sectioning pattern. Note this is not the same as "multiple agents editing code simultaneously": scanning is read-type (safe); co-development is write-type (the minefield Cognition warns about) [9].
Routing heterogeneous request streams. Customer-service and internal-tool entry agents do nothing but classify, routing each request to its specialist — one agent for refund flows, one for technical debugging, one for data lookups. The advantage follows directly from the routing mechanism: each specialist's prompt is short, its tools few, its behavior predictable — far easier to test and iterate than one "does-everything" mega-agent [2][10].
A concrete manufacturing scenario. Take MES data analysis: a user asks "why did lot yield drop at the KH plant last week?" The orchestrator spawns three subagents — one traces process-parameter time series, one checks material lots and supplier changes over the same window, one pulls equipment-maintenance and exception work orders — each running dozens of queries against different Oracle schemas, reading tens of thousands of rows, and each returning a one-page conclusion; the orchestrator then cross-references the three timelines for intersections. A single agent attempting this would burn most of its context on the first direction's raw query results alone — by the third direction the model has long forgotten the original question. This is what "context isolation = effective memory capacity" looks like on a factory floor.
Social simulation and evaluation. The direction Generative Agents demonstrated: use many memory-stream-equipped agents to simulate user populations, market participants, or organizational behavior, and observe the emergent interactions [5]. The engineering cousin of this use is testing agents with agents — generating diverse simulated users to stress-test your customer-service agent.
When not to use it. Three red lines, each derived from the mechanism: (1) tightly coupled, decision-dense tasks (many hands editing the same code) — communication bandwidth is insufficient and implicit decisions will collide [9]; (2) tasks whose value cannot cover the ~15× token multiplier — casual chat, single lookups, simple rewrites are served by a single agent or even a single call [1][2]; (3) processes requiring strict reproducibility and auditability — composing multiple nondeterministic components amplifies variance; such flows belong to fixed workflows (pipelines with programmatic checkpoints), not autonomously cooperating agent crews [2]. Anthropic's own advice matches: start with the simplest composable pattern, and add complexity only after a single agent demonstrably falls short [2].
Closing: One Logic Chain Through the Whole Article
An LLM is a stateless function whose only working memory is the context window → the agent loop gives it the ability to act via "tool call, result appended back," but its three limits — finite context, sequential looping, role mixing — all stem from having one context → the underlying move of multi-agent is simply making "spawn a subagent" one more tool call, where each fresh context window buys isolation and parallelism → but isolation's flip side is mutual blindness, hence the memory-sharing mechanisms, from light to heavy: message passing, shared workspace (blackboard), shared state objects, long-term memory (hierarchical storage + retrieval) → the design patterns (supervisor, pipeline, routing, voting, critic, group chat, handoff) are permutations of "where control sits" × "how memory is shared" → therefore its winning scenarios are read-heavy, decomposable, breadth-first tasks (research, scanning, routing, simulation), trading ~15× tokens for N× effective memory and parallel time — while for write-heavy, tightly coupled tasks, implicit decisions cannot survive summarization, and a single-threaded agent is the right answer.
To decide whether to reach for multi-agent, three questions suffice: Can the task split into loosely coupled independent branches? Does each branch "read a lot, report a little"? Does the task's value cover the token multiplier? Three yeses — multi-agent is a structural win. Any no — go back and make your single agent better first.
References
[1] Anthropic (2025). How we built our multi-agent research system. — https://www.anthropic.com/engineering/built-multi-agent-research-system
[2] Anthropic (2024). Building Effective AI Agents. — https://www.anthropic.com/engineering/building-effective-agents
[3] Wu, Q. et al. (2023). AutoGen: Enabling Next-Gen LLM Applications via Multi-Agent Conversation. — https://arxiv.org/abs/2308.08155
[4] Hong, S. et al. (2023). MetaGPT: Meta Programming for A Multi-Agent Collaborative Framework. — https://arxiv.org/abs/2308.00352
[5] Park, J.S. et al. (2023). Generative Agents: Interactive Simulacra of Human Behavior. — https://arxiv.org/abs/2304.03442
[6] Packer, C. et al. (2023). MemGPT: Towards LLMs as Operating Systems. — https://arxiv.org/abs/2310.08560
[7] Li, G. et al. (2023). CAMEL: Communicative Agents for "Mind" Exploration of Large Language Model Society. — https://arxiv.org/abs/2303.17760
[8] Guo, T. et al. (2024). Large Language Model based Multi-Agents: A Survey of Progress and Challenges. — https://arxiv.org/abs/2402.01680
[9] Cognition (2025). Don't Build Multi-Agents. — https://cognition.com/blog/dont-build-multi-agents
[10] LangChain. Multi-agent architectures (documentation). — https://docs.langchain.com/oss/python/langchain/multi-agent
[11] A2A Project / Linux Foundation. Agent2Agent (A2A) Protocol Documentation. — https://a2a-protocol.org/latest/