AI Agents: From Chain-of-Thought to Enterprise Agents

Summary: An AI agent is an AI system that can reason on its own, call tools, and take actions to accomplish a goal — not just generate a block of text. Agents grew out of 2022 reasoning methods (chain-of-thought, ReAct), through 2023's tool calling (Function Calling) and the autonomous-agent hype, multi-agent frameworks, and on to today's enterprise agents that operate computers and connect to business systems. For companies, the real question isn't how smart an agent is, but which capabilities are reliable enough to put into production: Gartner expects up to 40% of enterprise applications to include task-specific AI agents by 2026 (up from less than 5% in 2025) — yet also predicts that over 40% of agentic AI projects will be canceled by the end of 2027. This article walks through where AI agents came from, using one concrete example, and how businesses can deploy them without joining that cancellation statistic.

Are AI agents as complicated as they sound?

No. AI agents sound sophisticated, but at their core they're an engineering application of large language models (LLMs) — you take a model that only "generates the next word," bolt on tools, memory, and a loop, and let it actually get work done.

How an LLM works is fairly plain: given the preceding text, it predicts the most likely next word, one word at a time. A model like that is great at "talking" but can't "do" — ask it today's weather and it can only make something up from its training, not actually go check. (We covered that mechanism, and where it fits in a business, in what today's AI actually is.)

The turning point came with ReAct in 2022. What it does is simple once you see it: let the model pause mid-generation, emit an "action request" such as Action: search[Colorado orogeny], then hand control to an external program — the program actually calls the search API, gets the result, pastes it back, and lets the model continue. The model does the "thinking," the program does the "doing," and alternating between them runs a "reason–act–observe" loop.

Early on, that step was stitched together with text conventions and regex parsing, which was brittle: if the model's format drifted even slightly (a round bracket instead of a square one, an extra pair of quotes), parsing failed. OpenAI's Function Calling, launched in June 2023, made it robust — the model outputs structured JSON that conforms to a schema, and the system guarantees the format is valid. The loop didn't change; the "act" step just went from improvised to a standard engineering part. Keep that loop in mind — everything else is built on top of it.

How does an AI agent handle a real task? A sales email example

By decomposition: break a complex task into subtasks, then use a "supervisor" AI to coordinate them. That single idea is what turns one tool call into an AI agent that handles real work.

Picture an everyday task: your manager asks a staff member to write a sales outreach email to a prospect, based on the company's product manual and the supplier's price list. Routine for a person; not so simple for an AI.

Without tools, the model can't do it at all — it can't read your manual or pull live pricing. Even with Function Calling, the task is still too complex: cram dozens of manual pages and several price tables into the model's limited context window, and it will often "drift" partway through and start inventing things — that's hallucination.

The engineering answer is to split the work: treat "read the product manual," "look up supplier pricing," and "draft the email" each as a subtask, handed to a focused AI. Then a "supervisor" AI coordinates — breaking the big task apart, dispatching it to the sub-agents, collecting the results, and owning the final deliverable. That is what an AI agent looks like in the real world.

A production-grade implementation has plenty of detail on top — how to retry a failed subtask, how sub-agents pass intermediate results, how the supervisor judges quality — but the skeleton is "decompose, dispatch, aggregate."

So what exactly is an AI agent?

An AI agent is an AI system that autonomously completes multi-step tasks, whereas a chatbot typically handles single-turn Q&A. The same LLM may sit behind both; the difference is whether it can "decide the next step on its own and actually carry it out."

An AI agent usually has four parts: reasoning (breaking a goal into steps), tool use (querying data, doing math, calling APIs, or operating a system), memory (holding context and intermediate results), and an action loop (execute — observe — decide again, until done). The "supervisor coordinating subtasks" example above is what you get once those four come together. A plain LLM is just a "text generator"; add tools, memory, and a loop controller, and it becomes an agent.

What stages did AI agents go through?

AI agents passed through roughly six stages: reasoning foundations, tool calling, the autonomous-agent hype, multi-agent frameworks, new capabilities and standardization, and today's enterprise agentic phase. Each step patched a weakness the previous one exposed.

• Reasoning foundations (2022): before an agent could "call tools," the model first had to learn to "think as it acts." Chain-of-Thought had the model spell out its reasoning; ReAct (Yao et al., 2022) wove "reason–act–observe" into a loop that became the skeleton of every later agent.
• Tool calling (2023): reasoning alone wasn't enough — it also had to really check the weather, do math, call an API. Meta's Toolformer, ChatGPT plugins, and OpenAI's Function Calling in June made tool use reliable.
• The autonomous-agent hype (2023): once it could loop and call tools, people gave it a goal and let it run. AutoGPT and BabyAGI went viral, then cooled fast — prone to infinite loops, drifting off course, and burning huge compute without delivering. They proved the direction and exposed how unreliable long-horizon tasks were.
• Multi-agent and frameworks (2023–2024): since a single agent was unreliable over long horizons, split the work. Microsoft AutoGen, CrewAI, and MetaGPT had specialized agents cooperate; frameworks like LangChain standardized tools, memory, and orchestration — the engineering version of that "supervisor + subtasks" pattern.
• New capabilities and standardization (2024): Anthropic's Computer Use let an agent operate a graphical interface like a person; Cognition's Devin showcased a coding agent; Anthropic's MCP (Model Context Protocol) set a common standard for "how agents connect to tools and data," often called the "USB-C of AI."
• The enterprise agentic phase (2025 onward): the theme shifted from "can it demo?" to "can it run reliably in production?" Browser agents, deep research, and coding agents landed one by one, and reasoning models (such as the o-series) raised the success rate of long-horizon planning — patching the biggest weakness of the 2023 wave.

In one line: from "just generates text" → "structured tool calls" → "autonomous loop" → "reliable long-horizon agent."

What comes next for AI agents?

In Yangshu's view, AI agents are already mature at decomposing tasks — that part is solid engineering. The next real frontier is letting them improve themselves from feedback.

Today's agents still mostly rely on developers to tune them — rewriting prompts, adjusting how subtasks cooperate. A step further: back to that outreach email — if the staff member gives feedback on the draft, the supervisor AI could use it to optimize how the sub-agents collaborate and their prompts on its own, even pull in more tools, without a developer lifting a finger. An agent that improves itself — that's the next step we see.

Will a fundamentally new interaction paradigm emerge? We don't see one yet, and we lean toward thinking there won't be — because at root, humans tackle complex tasks the same way, by breaking them into subtasks, which is exactly the path AI agents take. Engineering probably won't sprout something entirely new here. That said, this is only our read of the moment, and everything remains uncertain.

What does an enterprise-ready AI agent actually look like?

A reliable enterprise AI agent is narrow in scope, grounded in your own data, traceable in every conclusion, and keeps a human in the loop — not an all-powerful autonomous black box. Gartner predicts over 40% of agentic AI projects will be canceled by the end of 2027, citing runaway costs, unclear value, and weak controls. What usually fails isn't the technology — it's putting an agent in the wrong place with no guardrails.

But more fundamental than which model or framework you pick is your own data. A staff member's understanding of a company was never only on paper or in a system — company culture, the boss's temperament, a client's unspoken preferences: an AI agent can't absorb these unquantifiable things. And an AI is only as good as its input: feed it thin or low-quality data and the output will be poor. So the real foundation isn't the model — it's getting your underlying data in order.

On top of that, making an agent run reliably in a business usually comes down to four things, which map neatly onto a concrete stack:

• Traceable knowledge (RAG) — let the agent answer from your own documents, with exact citations, instead of relying on memory. That's exactly what an AI knowledge base does. To be candid: RAG is the most practical grounding method available today, and we use it too — but it isn't the endpoint. Cross-language retrieval and detail loss remain real problems, and we expect it to be superseded by something better in time. A topic for another post.
• Tools that read documents (OCR) — enterprise data is largely locked inside scans. AI OCR turns invoices, receipts, cheques, IDs, forms, and contracts into structured fields — the agent's "eyes."
• Hands and feet that execute (RPA) — once the agent decides what to do, something has to actually do it. RPA software robots handle cross-system execution: data sync, reports, approvals, and connecting to legacy systems with no API — the "act" step in "reason–act–observe." For which tasks to automate first, see our RPA guide.
• Data stays in-house + human review — choose on-premise or private cloud so documents never enter a shared public AI service; keep human sign-off for high-risk decisions, positioning the agent as an accelerator, not an unattended autopilot.

This "narrow the scope, ground it in your own data, keep a human in the loop" approach is also the core argument in our piece Is AI a bubble?: the technology is real, but the value comes from restraint. Longer term, whoever organizes their data assets early holds a genuine differentiator in the AI era — something we'll write about separately.

How do AI agents relate to RAG, RPA, and Function Calling?

They aren't competitors; they're different layers of one agent: Function Calling is the interface the agent uses to call tools, RAG is the memory that supplies it with traceable knowledge, and RPA is the hands and feet that execute actions across systems.

• Function Calling / tool use: lets the model request "do this thing" in a structured way — the standard interface between an agent and the outside world.
• RAG (retrieval-augmented generation): retrieves relevant content from your knowledge base before answering, then answers with citations — used to suppress hallucination and make answers auditable.
• RPA (robotic process automation): handles the actual execution — moving data and running processes across systems. Traditional RPA follows fixed rules; combined with an AI agent, it can handle more flexible steps that require judgment.

A complete enterprise agent is usually the combination — "a reasoning model + a Function Calling interface + RAG knowledge + RPA execution" — not any single technology.

How should a business adopt AI agents?

Start with a single high-value use case, ground it in your own data, keep a human in the loop, and measure ROI — rather than chasing "fully autonomous" from day one. This matches Gartner's own advice: put agents where the value is clear first.

The usual path: map how the business actually runs (Discover & Design), pick a use case with a clear return (document Q&A, invoice entry, reconciliation), build a small-scope pilot on your own data, prove it, then expand. The underlying model can be a cloud service (OpenAI, Anthropic, Google) or an open-source model like DeepSeek deployed on your internal network so sensitive data never leaves. In our experience, a pilot focused on a single document set or use case can be running within 2–4 weeks.

Key takeaways

• An AI agent is an LLM plus three additions: tool use, memory, and an action loop that keeps running until the goal is met.
• The loop itself came from ReAct in 2022; Function Calling in 2023 didn't change it, it just made the "act" step reliable enough to engineer around.
• Real agents work by decomposition — a supervisor AI splits the task, dispatches sub-agents, and owns the final deliverable.
• The 2023 autonomous-agent wave failed on long-horizon reliability, not on intelligence. That is still the thing to design around.
• In a business, the working stack is Function Calling as the interface, RAG as the memory, OCR as the eyes, and RPA as the hands — with a human reviewing high-risk steps.
• Your data quality decides the outcome more than your choice of model or framework does.
• Yangshu's view: the next real frontier is agents that improve themselves from feedback, without a developer rewriting prompts.

External data in this article is drawn from Gartner press releases; the history section references the public timeline of ReAct (Yao et al., 2022), Toolformer, AutoGPT, MCP, and Computer Use. The views on self-improving agents, the limits of RAG, and data as an asset are Yangshu's.