4 Architecting and building multi-agent systems
This chapter explains how to architect and build multi‑agent systems, outlining when to graduate from a single agent to a coordinated team. It compares three foundational patterns—flow (assembly line), orchestrator (hub‑and‑spoke), and collaboration (peer‑to‑peer)—and frames design choices around the core levers of decision‑making, control, and communication, with coordination as a fourth dimension. The discussion also surveys common communication styles (selective message passing, shared conversational context, and standardized agent protocols) and execution strategies (sequential and parallel pipelines, hierarchical delegation, iterative critique and refinement, voting/ensembles, role‑playing, conditional routing, and decentralized peer networks), emphasizing trade‑offs in cost, latency, predictability, and fault tolerance.
Practically, the chapter shows how overloaded single agents are decomposed into focused, specialized agents connected in an agent‑to‑agent flow to improve reliability, cost, and clarity of roles. It recommends structuring interfaces with typed inputs/outputs to make handoffs explicit and easier to validate, and using external code for key branching decisions when determinism is required. Multiple handoff approaches are contrasted: passing outputs explicitly between agents, sharing a common conversation thread, and using framework‑level handoffs that let agents transfer control without bespoke glue code. Developers are encouraged to visualize flows and instrument handoffs to understand what data moves between agents and why.
To keep multi‑agent workflows robust, the chapter introduces guardrails that validate and, when needed, correct inputs, outputs, and inter‑agent transfers—either with straightforward programmatic checks or by delegating the validation itself to specialized “guardrail agents.” It illustrates recovery patterns such as retries and fallbacks, and shows how guardrails integrate with pass‑off flows for fine‑grained control. While orchestration can centralize planning and delegation, the guidance is to start simple with flows, prefer structured data and minimal context, add deterministic checkpoints where variability hurts, and layer guardrails and monitoring as complexity grows—mixing patterns judiciously only after establishing a stable, comprehensible baseline.
The three well-established patterns for building multiple agent systems, from the agent flow (assembly line), orchestrator (hub-and-spoke), to the collaboration (peer to peer).
Decision-making (command) and control demonstrated for more specialized multi-agent architectures, the flow, orchestration and manager architectures.
Comparison of different agent communication patterns represented in a flow.
Various ways agents may coordinate execution sequentially or in parallel.
A single agent is transformed into a multi-agent flow of agents. The agent role is broken down into three well-defined and distinct roles that encapsulate a well-defined set of tools.
shows an agent-to-agent flow with a deterministic (coded) decision point added. Taking the decision away from the agents and making it deterministic keeps the workflow more consistent.
three agent flow communication patterns demonstrating how agents can pass messages from one another and in turn pass command and control from agent to agent.
There are two ways to visualize the agent flow using draw_graph and the Traces page that can be fournd on the OpenAI Dashboard under logs.
Guardrails can be used to validate and control input and output of an agent.
Traces page after executing the Orchestration agent shows X and Y.
Summary
- Single-agent designs often hit scalability walls; converting a monolithic agent into an agentic flow (a chain of specialised agents) restores clarity, extensibility, and performance.
- Agent-to-agent flows work like prompt-chaining with superpowers: each node can invoke tools and reason independently yet pass concise, typed outputs downstream to keep the context lean.
- Insert deterministic decision points (code or schema checks) wherever the flow must repeat reliably; don’t rely on stochastic LLM judgment for pass/fail branches.
- The OpenAI Agents SDK supports two hand-off styles: conversational (shared thread) and pass-off (explicit code routing). Choose conversational for speed and pass-off for fine-grained control.
- Use the
handoffsfield of the agent plus clear instructional prompts to enable internal hand-offs that require zero extra orchestration code. - Visualise complex flows early using
draw_graph(),and the Dashboard Traces view reveals hidden loops, tool chains, and latency hotspots. - Wrap risky transfers in guardrails—input/output validators that reject, retry, or correct data before it corrupts the flow; tripwires surface as explicit exceptions you can loop on.
- Guardrails themselves can be LLM-powered agents, giving you natural-language policies without brittle regex or length checks—remember they, too, need schemas and tests.
- Tool limits still matter in multi-agent worlds: every registered tool inflates every call; keep each agent’s tool list tight (< 10) and scoped to its role to avoid token bloat.
- Flow, orchestration, and manager-worker are the three canonical decision patterns. Start with plain flows and graduate to orchestrators only when centralised delegation is truly required.
- Choose one communication layer (shared memory, message passing, MCP, or emerging A2A protocol) per project; mixing channels multiplies debugging pain.
- A production-ready agentic flow blends typed I/O, deterministic checkpoints, visual traces, scoped tool sets, and guardrails—yielding pipelines that can scale, recover, and evolve without surprise.
- Agents may be coordinated using multiple different strategies: Sequential Flow, Parallel Delegation, Hierarchical Coordination, Iterative Debate and Refinement, Voting / Best-of-N (Ensemble), Role-Playing Collaboration, Conditional Routing (Branching), and Peer-to-Peer Network
FAQ
When should I move from a single agent to a multi-agent flow?
Move when a single agent becomes overloaded with tools or long prompts, when specialization would improve quality, or to reduce cost/latency by limiting each agent’s tools and context. Splitting into focused agents typically makes behavior easier to reason about and maintain.What are the three core multi-agent architecture patterns?
- Flow (agent-to-agent pipeline): command, control, and context pass along an assembly line. Simple and reliable starting point.- Orchestrator (hub-and-spoke): a central agent decides and delegates control to workers. Good for complex decomposition but adds overhead.
- Collaboration (peer-to-peer): agents share a common channel and coordinate together. Flexible but harder to design and debug.
How do decision-making, control, and communication apply to agents?
- Decision-making (command): who decides what to do and when. Limit context to what’s necessary so decisions stay effective.- Control: who can act (use tools, perform tasks). Often passed between agents in a flow or by an orchestrator.
- Communication: how agents share context (fully shared vs. restricted). Restricting context can reduce confusion and token cost.
What communication strategies can agents use?
- Pass-off messaging: code passes only the necessary inputs/outputs to the next agent (maximum control, more code).- Shared conversation/memory: all agents see the same thread (easy, but more tokens and possible confusion).
- Protocol/tool calls (e.g., MCP): treat another agent/tool as a function, tightly constraining what’s exchanged.
How can I coordinate agents (sequentially or in parallel)?
- Sequential pipeline: simple, deterministic handoffs; slower, brittle if a step fails.- Parallel delegation: run independent steps concurrently, then merge results.
- Hierarchical coordination: a manager decomposes tasks and mixes parallel/serial work.
- Iterative debate/refinement: agents critique and improve outputs over rounds for higher quality.
What’s the difference between handoffs and manual pass-offs?
Handoffs are built-in transitions where one agent internally transfers control to the next (less glue code, faster to build). Manual pass-offs are coded transitions that let you validate, filter, or transform data between agents (more control, more engineering effort).How do I validate agent inputs/outputs and handoffs?
Use guardrails to check or correct data at flow boundaries or between agents. You can implement input and output guardrails with code (e.g., length checks) or with “guardrail agents” that inspect and annotate data. Tripwires let you fail fast, retry, or route to fallback behavior.How can I make multi-agent flows more deterministic and predictable?
- Add external, coded decision points for critical branches.- Strongly type inputs/outputs (schemas/models) to reduce parsing ambiguity.
- Limit each agent’s tools and context to only what’s necessary.
- Use guardrails and retries to enforce minimum quality and recover from variability.
How do I monitor and visualize multi-agent systems?
- Use a handoff wrapper/callback to inspect what data triggered a handoff and what was passed to the next agent.- Visualize the graph of agents and handoffs to understand structure and dependencies.
- Use tracing to inspect calls, tool use, and data flow for debugging.
AI Agents in Action, Second Edition ebook for free
