AI Agents 101 | 01. Why Agents, Why Now
Why Agents, Why Now
AI agents matter because software is moving from "answer my question" toward "help me finish the task." A chatbot can explain a refund policy. An agent can inspect the order, classify the case, retrieve the right policy, call the refund tool, ask for approval if needed, and update the case record. That shift is powerful, but it is also where many teams overbuild.
The simple thesis: build an agent when the task needs goal-directed work across tools, data, uncertainty, and multiple steps. Do not build one when a normal function, workflow, search system, or single model call is enough.
Level 1: Fundamentals
Google Cloud defines AI agents as software systems that use AI to pursue goals and complete tasks on behalf of users, with capabilities such as reasoning, planning, memory, autonomy, and the ability to make decisions or adapt over time [1]. The important word is not "AI"; it is "pursue." An agent has a goal and an execution path that can change as it observes new information.
A useful beginner mental model is:
agent = model + instructions + tools + state + loop + guardrails
The model interprets the task. Instructions define the role, constraints, and success criteria. Tools let the agent affect the world: search, read files, call APIs, write code, submit forms. State lets it remember what happened so far. The loop lets it plan, act, observe, and recover. Guardrails decide what the agent is allowed to do without help.
The local Hitchhiker guide uses the same stack-shaped view of agentic systems: RAG and knowledge, memory, harness and orchestration, design patterns, environments and benchmarks, MCP, agent skills, A2A, frameworks, and agentic UI all become supporting parts around the model loop [10].
This is different from a chatbot. A chatbot is usually reactive: the user asks, the system answers. An assistant may recommend actions while leaving decisions to the user. A bot often follows fixed rules. Google Cloud frames agents as more proactive and goal-oriented than assistants and bots, able to perform complex multi-step actions and make decisions with more independence [1].
That does not mean "agent" has one clean industry definition. Anthropic notes that teams use the word for everything from fully autonomous systems to more prescriptive workflows, then draws a practical architectural distinction: workflows are systems where LLMs and tools follow predefined code paths, while agents let the model dynamically direct its process and tool use [2]. If you can draw the path ahead of time, you may need a workflow. If the path depends on what the system discovers, you may need an agent.
The beginner mistake is to treat autonomy as the product. Autonomy is not the product. Autonomy is the cost you accept when flexibility is more valuable than predictability.
Level 2: Concepts
Agents became more practical because several capabilities matured at once.
First, models became better at reasoning over messy, multi-step tasks. The ReAct paper made the pattern concrete: interleave reasoning traces with actions so a model can update its plan, call external sources or environments, and handle exceptions from observations [3]. Modern agent systems are often variations on this idea, even when the loop is hidden behind an SDK.
Second, tool use became a first-class platform primitive. OpenAI described its March 2025 agent-building release as a set of blocks for useful and reliable agents: the Responses API, built-in tools such as web search, file search, and computer use, an Agents SDK for orchestration, and observability for tracing execution [4]. These are not just convenience features. They are the runtime parts that let a model inspect, decide, act, and be debugged.
Third, implementation patterns are becoming less mysterious. Hugging Face's smolagents article describes agency as a spectrum: from LLM output that has no impact on program flow, to routing, to tool calls, to multi-step agents where the model controls iteration and continuation [5]. That spectrum is useful because it breaks the false binary between "no agent" and "fully autonomous agent." Most production systems live somewhere in the middle.
Fourth, integration standards started to matter. Anthropic introduced the Model Context Protocol in November 2024 as an open standard for connecting AI assistants to data systems such as repositories, business tools, and development environments [6]. Google later announced Agent2Agent, a protocol intended to let agents from different vendors or frameworks communicate, exchange information, coordinate actions, advertise capabilities, manage tasks, and return artifacts [7]. The current A2A docs describe it as an open protocol for agent-to-agent interoperability, now under Linux Foundation governance and complementary to MCP's agent-to-tool role [11].
The core conceptual shift is this:
old app = user clicks deterministic controls
chatbot = user asks, model answers
workflow = code chooses the next step
agent = model helps choose the next step, using tools and state
That last line is the source of both the value and the danger. Once a model can choose actions, the engineering problem is no longer just "how good is the answer?" It becomes "what actions can happen, under what permissions, with what evidence, and with what rollback path?"
Level 3: Engineering Detail
A production-style agent is better understood as a small distributed system than as a fancy prompt.
client or trigger
-> task intake and policy checks
-> agent runtime
-> model call
-> context planner
-> tool broker
-> state store
-> trace/log/event stream
-> eval and guardrail hooks
-> human approval or final action
-> artifact, ticket, PR, message, report, or database update
The model is only one component. The engineering work is deciding which tools exist, how their schemas are shaped, how much context is loaded, where state lives, when actions require approval, which traces are recorded, and how the system knows it is done.
The runtime loop
A minimal agent loop looks like this:
state = start(task)
while not done(state):
context = select_context(task, state)
next_step = model(instructions, context, state)
if next_step.requires_approval:
pause_for_human(next_step)
continue
result = execute_tool(next_step.tool, next_step.args)
state = append_observation(state, next_step, result)
if result.failed and retry_budget_available(state):
state = plan_recovery(state, result)
return final_answer_or_artifact(state)
That loop is simple, but every line hides design choices. select_context can be a few strings or a retrieval system with source ranking. execute_tool can be an in-process function, an API call, a browser action, a shell command, or another agent. done can mean "the model says done," but in production it should usually mean the system has observed objective evidence: test passed, ticket updated, refund staged, metric threshold met, approval received, or artifact written.
The design surfaces
| Surface | Engineering Question | Concrete Design Choice |
|---|---|---|
| Task intake | What is the user asking the agent to accomplish? | typed task object, priority, requester, risk tier, success criteria |
| Context | What should the model know right now? | retrieval policy, source ordering, freshness, permissions, summarization |
| Tools | What can the model do? | function schemas, auth scopes, rate limits, retries, structured errors |
| State | What must survive across steps? | conversation state, task state, memory, artifacts, event history |
| Loop | Who decides the next step? | fixed workflow, model-directed loop, evaluator-optimizer, orchestrator-worker |
| Guardrails | What must be blocked or reviewed? | input checks, output checks, tool approval gates, sandboxing |
| Observability | How do we debug behavior? | traces, spans, tool-call logs, cost, latency, source usage, final outcome |
| Evals | How do we know it works? | scenario tests, task-success metrics, regression suites, adversarial cases |
Anthropic's production guidance is a useful anchor here: start with the simplest solution possible and increase complexity only when the task needs it, because agentic systems often trade latency and cost for better task performance [2]. That means each design surface should earn its place.
Tool contracts
Good agents depend on boring, explicit tool contracts. A vague tool makes the model guess. A narrow tool makes the model choose.
{
"name": "stage_refund",
"description": "Stage a refund for human approval. Does not move money.",
"input_schema": {
"type": "object",
"required": ["order_id", "reason_code", "amount_cents"],
"properties": {
"order_id": { "type": "string" },
"reason_code": { "type": "string", "enum": ["DAMAGED", "LATE", "DUPLICATE", "OTHER"] },
"amount_cents": { "type": "integer", "minimum": 0 }
}
},
"approval_required": true
}
The important details are the boundary and consequence. This tool stages a refund; it does not move money. It has a typed schema. It constrains the reason code. It requires approval. Those constraints matter more than clever prompt wording.
Anthropic makes a similar point in tool-design language: tool definitions deserve as much attention as prompts, with clear descriptions, examples, edge cases, input requirements, and boundaries between similar tools [2].
When to build an agent
Good agent use cases tend to have four properties:
- ambiguity: the user can state the outcome, but not the exact steps
- unstructured data: the work touches emails, PDFs, tickets, logs, web pages, codebases, or notes
- tool use: the system must retrieve, inspect, calculate, edit, create, or submit something
- multi-step feedback: the next action depends on what happened after the previous action
Bad agent use cases are just as important.
| Situation | Better First Design |
|---|---|
| Deterministic, well-known steps | normal code or workflow engine |
| One answer from known context | retrieval plus one model call |
| Known multi-step path with clear branches | prompt chain or coded workflow |
| High-volume classification | classifier or structured model call |
| High-risk irreversible action | workflow with explicit human approval, or no automation |
| Unknown steps plus tools plus feedback | agent, with guardrails and evals |
Do not use an agent to wrap a stable API call. Do not use an agent when a SQL query, cron job, form, rules engine, or search index would be clearer. Do not give an agent write access to important systems before you have permissions, tracing, evaluation, rollback, and human checkpoints.
Failure modes
Agents fail differently from chatbots because they can act.
| Failure Mode | Example | Engineering Response |
|---|---|---|
| Wrong tool | agent calls cancel_order instead of lookup_order
|
narrow tool names, examples, approval on destructive tools |
| Bad arguments | refund amount uses dollars instead of cents | typed schemas, validation, unit tests for tools |
| Stale context | agent follows an old policy | source freshness, versioned docs, retrieval timestamps |
| Loop drift | agent keeps working after success | objective stop conditions, max turns, success probes |
| Hidden partial failure | tool fails but final answer sounds confident | structured errors, trace review, eval cases for tool failure |
| Excessive agency | agent sends external email without review | permission tiers, human approvals, sandboxing |
| Prompt injection | web page or email instructs the agent to ignore policy | untrusted-content handling, tool isolation, security tests |
OWASP lists prompt injection, insecure output handling, sensitive information disclosure, insecure plugin design, excessive agency, and overreliance among LLM application risks [8]. These risks matter more for agents because the model is no longer just generating prose. It may be reading untrusted content, calling tools, passing data between systems, and making decisions that users assume have been verified.
Evaluation and operations
A practical agent eval suite should test the loop, not only the final answer.
Measure:
- task success: did the user-visible job actually complete?
- trajectory quality: did the agent use the right tools in the right order?
- source quality: did it rely on permitted, fresh, relevant sources?
- approval behavior: did it pause before risky actions?
- recovery: did it handle tool errors and ambiguous inputs?
- cost and latency: how many model calls, tool calls, tokens, retries, and seconds?
- safety: did it avoid leaking secrets, following injected instructions, or exceeding scope?
Operationally, the agent also needs a runtime home. Google Cloud notes that agents often need flexible compute for reasoning, planning, and tool use, and describes Cloud Run as one deployment option for containerized agent code with autoscaling, scale-to-zero behavior, HTTPS endpoints, and integration with agent orchestration [1]. The specific platform can differ, but the runtime concerns remain: concurrency, timeouts, secrets, retries, queueing, logs, traces, cost controls, and incident response.
NIST's AI Risk Management Framework is meant to help organizations incorporate trustworthiness into the design, development, use, and evaluation of AI systems [9]. For agents, that framing should be applied at the system boundary: task intake, data access, tool permissions, evaluation, monitoring, audit logs, and escalation paths.
Current Landscape
The agent ecosystem in 2026 is less about one magical architecture and more about infrastructure catching up with the idea.
OpenAI's agent platform work emphasizes APIs, built-in tools, orchestration, and observability, including the Responses API, web search, file search, computer use, an Agents SDK, and tracing support [4].
Anthropic's guidance emphasizes simple composable patterns, the workflow-vs-agent distinction, and careful design of the "agent-computer interface" [2]. Its Model Context Protocol addresses another practical bottleneck: connecting AI systems to the data and tools where work actually happens [6].
Google's agent material emphasizes agents as goal-oriented systems with reasoning, acting, observing, planning, collaboration, and memory, and its A2A announcement points to a world where agents need capability discovery, task management, collaboration messages, artifacts, and long-running task updates across vendors and frameworks [1][7]. The maintained A2A protocol docs make the complementary split sharper: MCP connects agents to tools and context, while A2A connects agents to other agents [11].
Hugging Face's smolagents work is interesting because it keeps the abstraction small and exposes the mechanics: tools, model, multi-step loop, and code-based actions for cases where programmatic composition is a better action language than JSON [5].
So "why now" has two answers. The capability answer is that models, tools, and orchestration have become good enough to make agents useful. The engineering answer is that the surrounding system - protocols, observability, evaluation, guardrails, deployment patterns, and governance - is finally becoming part of the default conversation.
Practical Takeaways
Use an agent when the path is not fully knowable in advance and the system needs to choose actions from context.
Start with the least agentic design that can work. A reliable workflow beats a clever agent when the task is predictable.
Design tools like product interfaces. Give them narrow names, typed inputs, clear errors, permission boundaries, examples, and logs.
Persist state deliberately. Keep task state, tool observations, artifacts, approvals, and failures in a form the system can inspect later.
Evaluate the whole loop, not just the final paragraph. A good-looking answer can hide a bad source, skipped approval, wrong tool, or lucky retry.
Keep humans in the loop for money movement, data deletion, account changes, external communication, security-sensitive actions, and anything embarrassing if done wrong.
Treat autonomy as a scoped design decision. The best agents are not the ones that seem most autonomous. They are the ones whose autonomy is deliberately placed where flexibility helps, and constrained everywhere else.
References
[1] "What is an AI agent?" Google Cloud. https://cloud.google.com/discover/what-are-ai-agents. Published/updated: 2026-04-02. Accessed: 2026-07-05. Used for: current definition of AI agents, agent capabilities, agent-vs-assistant-vs-bot distinction, and deployment considerations.
[2] "Building effective agents." Anthropic. https://www.anthropic.com/engineering/building-effective-agents. Published/updated: 2024-12-19. Accessed: 2026-07-05. Used for: workflow-vs-agent distinction, start-simple guidance, agent fit criteria, tool-design advice, and implementation patterns.
[3] "ReAct: Synergizing Reasoning and Acting in Language Models." Shunyu Yao, Jeffrey Zhao, Dian Yu, Nan Du, Izhak Shafran, Karthik Narasimhan, and Yuan Cao. https://arxiv.org/abs/2210.03629. Published/updated: 2023-03-10. Accessed: 2026-07-05. Used for: reasoning-plus-action loop background.
[4] "New tools for building agents." OpenAI. https://openai.com/index/new-tools-for-building-agents/. Published/updated: 2025-03-11. Accessed: 2026-07-05. Used for: Responses API, built-in tools, Agents SDK, orchestration, and observability as agent-building primitives.
[5] "Introducing smolagents, a simple library to build agents." Hugging Face. https://huggingface.co/blog/smolagents. Published/updated: 2024-12-31. Accessed: 2026-07-05. Used for: agency spectrum, multi-step agent loop, code-agent framing, and lightweight implementation mechanics.
[6] "Introducing the Model Context Protocol." Anthropic. https://www.anthropic.com/news/model-context-protocol. Published/updated: 2024-11-25. Accessed: 2026-07-05. Used for: MCP as an open standard for connecting assistants and agents to data systems and tools.
[7] "Announcing the Agent2Agent Protocol (A2A)." Google Developers Blog. https://developers.googleblog.com/en/a2a-a-new-era-of-agent-interoperability/. Published/updated: 2025-04-09. Accessed: 2026-07-05. Used for: A2A interoperability goals, capability discovery, task management, collaboration messages, artifacts, and long-running task support.
[8] "OWASP Top 10 for Large Language Model Applications." OWASP Foundation. https://owasp.org/www-project-top-10-for-large-language-model-applications/. Published/updated: 2025. Accessed: 2026-07-05. Used for: LLM application security risks including prompt injection, insecure output handling, insecure plugin design, excessive agency, and overreliance.
[9] "AI Risk Management Framework." National Institute of Standards and Technology. https://www.nist.gov/itl/ai-risk-management-framework. Published/updated: 2026-04-07. Accessed: 2026-07-05. Used for: trustworthy AI risk management framing and governance relevance for agentic systems.
[10] "The Hitchhiker's Guide to Agentic AI: From Foundations to Systems." Haggai Roitman. Local source: agent-101/The Hitchhiker’s Guide to Agentic AI/book.tex. Published/updated: 2026. Accessed: 2026-07-05. Used for: agentic AI stack framing across RAG, memory, harness and orchestration, MCP, A2A, frameworks, and agentic UI.
[11] "A2A Protocol." A2A Project / Linux Foundation. https://a2a-protocol.org/latest/. Published/updated: 2026. Accessed: 2026-07-05. Used for: current A2A interoperability framing, Linux Foundation governance, and the MCP-vs-A2A split.
Enjoy Reading This Article?
Here are some more articles you might like to read next: