AI Agents 101 | 07. Orchestration Patterns
Orchestration Patterns
Once an agent can use tools, the next question is how work should be divided.
Should one agent handle the whole task? Should a manager delegate to specialists? Should subtasks run in parallel? Should the user be handed off to a different agent? Should another model critique the answer? Or should the system stay simple and avoid orchestration entirely?
Orchestration is the design of who does what, in what order, with what state, under what constraints.
Level 1: Fundamentals
Anthropic's agent guidance makes a useful distinction between workflows and agents. Workflows follow predefined code paths. Agents dynamically choose their path through tools and observations [1]. Most production systems use both: deterministic workflow where the path is known, agentic control where the path is uncertain.
OpenAI's orchestration docs describe two common multi-agent patterns: handoffs, where a specialist takes over the conversation, and agents-as-tools, where a manager agent calls specialist agents while retaining control [2].
That difference is small in code but large in product behavior.
handoff:
user -> triage agent -> billing agent -> user
agents-as-tools:
user -> manager agent -> billing specialist tool -> manager agent -> user
The first changes who owns the next turn. The second keeps one visible owner and uses specialists behind the scenes.
The Hitchhiker guide connects this to broader multi-agent systems and A2A-style communication: as tasks become larger, specialization, parallelism, fault isolation, and delegation become architectural concerns [5].
Level 2: Concepts
Use orchestration when a single agent is becoming too broad, too slow, too unreliable, or too opaque.
| Pattern | Best For | Watch Out For |
|---|---|---|
| Single agent | narrow tasks, early prototypes, predictable tool set | prompt bloat and overloaded instructions |
| Prompt chain | known sequence of transformations | brittle if task path varies |
| Router | classifying work into known paths | wrong route creates silent failure |
| Parallel workers | independent subtasks or multiple perspectives | merge conflicts and cost growth |
| Orchestrator-workers | dynamic decomposition of complex tasks | manager overhead and unclear ownership |
| Evaluator-optimizer | iterative improvement with clear criteria | endless refinement loops |
| Handoff | user should talk to a specialist | context transfer and user confusion |
| Agents-as-tools | manager needs specialist capability | hidden failures behind manager summary |
Anthropic describes parallelization, orchestrator-workers, and evaluator-optimizer workflows as patterns for different task shapes [1]. The key is not to memorize names. The key is matching the pattern to uncertainty.
Routing
Routing works when categories are clear:
refund question -> billing agent
login problem -> account agent
bug report -> technical support agent
legal request -> human escalation
Routing fails when categories overlap or the system hides uncertainty. A good router can say "unclear" and ask a question.
Handoffs
Handoffs are useful when the specialist should own the interaction. OpenAI recommends keeping routing legible: give each specialist a narrow job, write concrete handoff descriptions, and split only when a branch truly needs different instructions, tools, or policy [2].
Handoff state should include:
- user goal
- relevant history
- tool results
- constraints
- pending approvals
- reason for handoff
- what the previous agent already tried
Do not make the next agent rediscover the whole case.
Agents as Tools
Agents-as-tools are useful when the user should experience one coherent agent but the system needs expert subroutines.
Examples:
- a research manager calls a source-finding agent
- a coding agent calls a test-analysis agent
- a support agent calls a policy specialist
- a planning agent calls a cost-estimation agent
OpenAI positions this pattern as a manager workflow: the manager agent invokes specialists and synthesizes their outputs [2].
Subagents
OpenAI Codex docs describe subagents as specialized agents that can be spawned in parallel for work such as exploration, tests, or log analysis, while keeping the main context focused [3]. That is a practical orchestration lesson: specialization can reduce context clutter, not just divide labor.
The caution is that parallel write-heavy work can conflict. Read-heavy exploration is a safer starting point.
Level 3: Engineering Detail
An orchestration design should specify ownership, state transfer, and merge rules.
Pattern Selection
if task is narrow and tool set is small:
use one agent
elif task has known deterministic stages:
use workflow / prompt chain
elif task has known categories:
use router
elif subtasks are independent:
use parallel workers
elif subtasks are unknown until runtime:
use orchestrator-workers
elif quality improves through critique:
use evaluator-optimizer
elif user should speak to specialist:
use handoff
else:
keep manager visible and call specialists as tools
The decision should be revisited after evals. Orchestration is justified when it improves reliability, capability, latency, or maintainability enough to pay for added complexity.
Agent Boundary Contract
specialist:
name: "policy_research_agent"
owns_user_turn: false
allowed_tools:
- search_policy_docs
- cite_policy_excerpt
input:
required:
- user_question
- policy_domain
- known_facts
output:
schema:
answer: string
citations: array
uncertainty: string
escalation_needed: boolean
limits:
max_turns: 4
max_sources: 6
This boundary is what prevents multi-agent systems from becoming a meeting with no agenda.
Merge Rules
Parallel or specialist outputs need merge logic.
| Merge Situation | Rule |
|---|---|
| specialists agree | summarize with citations |
| specialists conflict | surface disagreement and inspect evidence |
| one specialist fails | decide whether fallback is acceptable |
| partial result arrives late | either ignore by deadline or update asynchronously |
| output lacks evidence | mark as unusable, ask for repair |
| action recommendation is risky | require human approval |
Without merge rules, the manager model may smooth over contradictions. That is dangerous because orchestration can create false confidence: many agents speaking in chorus can still be wrong.
A2A Boundary
A2A, the Agent2Agent protocol, is aimed at agent-to-agent interoperability. Its docs describe it as an open protocol for communication and collaboration between opaque agentic applications, with features such as interoperability, complex workflows, secure collaboration, and extensibility [4].
In architecture terms:
MCP: agent to tools/data
A2A: agent to agent/application
Use A2A-style boundaries when the other participant is not just a callable function but another agentic system with its own state, policy, identity, and task lifecycle.
Current Landscape
The field is converging on a few durable orchestration patterns.
Anthropic's workflow taxonomy is helpful because it starts from task shape rather than framework preference: chain, route, parallelize, orchestrate, evaluate, or let an agent operate dynamically [1].
OpenAI's handoff and agents-as-tools docs give a concrete product distinction: sometimes the user should move to a specialist, and sometimes the specialist should stay behind the manager [2]. Codex subagents add a practical coding-agent variant: use specialized parallel agents to explore without polluting the main context [3].
A2A is emerging as a protocol-level answer to cross-agent interoperability [4]. It is not a replacement for MCP. MCP connects agents to tools and resources; A2A connects agentic applications to one another.
The Hitchhiker guide is useful here as a map from orchestration patterns to multi-agent communication, specialization, fault isolation, and protocol design [5].
Practical Takeaways
Start with one agent unless the task shape demands more. Orchestration should solve a concrete failure, not decorate the architecture.
Use routing for known categories, handoffs for visible specialist ownership, and agents-as-tools when the manager should remain the user's interface.
Keep specialists narrow. Give them clear tools, schemas, limits, and success criteria.
Be explicit about state transfer. The next agent needs enough context to help, but not the whole transcript by default.
Define merge rules before running workers in parallel. Contradictions should be handled, not averaged away.
Evaluate orchestration as a system. A clever manager is not enough if specialist outputs are ungrounded, late, or impossible to audit.
References
[1] "Building effective agents." Anthropic. https://www.anthropic.com/engineering/building-effective-agents. Published/updated: 2024-12-19. Accessed: 2026-07-06. Used for: workflow-vs-agent distinction, parallelization, orchestrator-workers, evaluator-optimizer, autonomy tradeoffs, and start-simple guidance.
[2] "Orchestrating multiple agents." OpenAI API documentation. https://developers.openai.com/api/docs/guides/agents/orchestration. Published/updated: not listed. Accessed: 2026-07-06. Used for: handoffs, agents-as-tools, routing descriptions, manager workflows, and specialist design.
[3] "Subagents." OpenAI Codex documentation. https://developers.openai.com/codex/concepts/subagents. Published/updated: not listed. Accessed: 2026-07-06. Used for: specialized parallel subagents, context isolation, read-heavy exploration, and cautions about write-heavy parallel work.
[4] "Agent2Agent Protocol." A2A Protocol documentation. https://a2a-protocol.org/latest/. Published/updated: 2026. Accessed: 2026-07-06. Used for: agent-to-agent interoperability, secure collaboration, complex workflows, and the MCP-vs-A2A distinction.
[5] "The Hitchhiker's Guide to Agentic AI: From Foundations to Systems." Haggai Roitman. Local source: agent-101/The Hitchhiker’s Guide to Agentic AI/book.tex. Published/updated: 2026. Accessed: 2026-07-06. Used for: multi-agent motivation, specialization, coordination, A2A framing, task lifecycle, and orchestration patterns.
Enjoy Reading This Article?
Here are some more articles you might like to read next: