“We want an AI chatbot” and “we want an AI agent” sound like the same request. They are not, and the difference decides whether your project ends up as a helpful demo or a system that actually removes work. Here is the distinction that matters, and how to tell which one your problem needs.
A Chatbot Answers. An Agent Acts.
A chatbot takes a question and returns an answer from what it was given — its training, a prompt, maybe a knowledge base. That is genuinely useful for FAQs, first-line support, and search. But it stops at words. Ask it to actually process the refund, update the record, or check three systems and file a ticket, and it can only tell you how to do those things.
An agent does them. It has tools — access to your APIs, databases, and systems — and it decides which to use, calls them, reads the results, and takes the next step. It can retrieve a customer’s order, check its status across systems, draft a response, and escalate if something looks wrong. The output is not a paragraph; it is a completed task.
The Difference Is Architecture, Not Model
Here is the part most buyers miss: the leap from chatbot to agent is not about using a smarter model. It is about the scaffolding around the model — tools, permissions, memory, and orchestration. A capable agent needs:
- Tools — defined, safe ways to reach your systems, typically through an MCP server that exposes exactly the capabilities you approve
- Permissions — scoped access so the agent can only do what it should, for the user it is acting on behalf of
- Memory and context — so it carries the thread of a task rather than starting fresh each turn
- Orchestration — the logic that decides when to act, when to ask, and when to stop
- Approval gates — human sign-off before consequential actions
Get that scaffolding right and a mid-tier model performs like a professional. Get it wrong and the smartest model on the market is still just a chatbot with extra steps.
Which One Does Your Problem Need?
Use a chatbot when the job is to answer: knowledge lookup, support triage, internal search, onboarding questions. If “a good answer with a source” is the finish line, you do not need the complexity of an agent.
Use an agent when the job is to do: anything that spans systems, takes multiple steps, or ends in a change to your data. If a person currently reads something, decides, and then acts in three different tools, that is agent-shaped work.
And be honest about the ones in between. Plenty of “agent” projects are really a chatbot with one tool call bolted on — which is fine, and cheaper, if that is all the problem needs.
Why the Distinction Costs or Saves You Money
Scoping the wrong one is the most common way AI budgets get wasted. Build an agent when a chatbot would do, and you have paid for permissions, tools, evaluations, and approval workflows you did not need. Build a chatbot when the problem needed an agent, and you ship something that impresses in the demo and gets abandoned in month two because it cannot actually finish the work.
That is why we start every engagement by naming the task, not the technology. If you can describe the job a person does today — what they read, what they decide, what they change — we can tell you honestly whether it is chatbot-shaped or agent-shaped, and scope accordingly.
Software Depo builds both, and production AI agents in particular — with the tools, permissions, evaluations, and approvals that make acting safe. Tell us the task you want handled and we will tell you which one it is.
The Questions That Actually Separate Them
Feature comparisons do not settle this; five questions do. Does the job end in a write to a system of record, or does it end when a person reads an answer? Does the path vary from case to case, or does every case follow the same steps? How many systems hold the data, and does each one expose an API that can act on behalf of the person asking rather than through a shared service account? If an action turns out to be wrong, what does it cost to undo, and would anyone notice it happened? And when the work runs unattended at two in the morning, whose name is on the result?
These are about what the work touches, how often the path changes, and who owns the consequences. Two of them can usually be answered in an afternoon by opening the systems involved and reading what their APIs allow. The other two need someone with the authority to say what an error costs, which is why scoping stalls so often: the technical questions are the easy ones, and the accountability question is the one nobody wants to answer first.
The second question does the most work in practice. Once a task qualifies as agent-shaped by touching several systems, taking multiple steps, and changing something, path variation is what separates it from its near neighbors. Take the runbook a person follows today and count the decision points where the next step depends on what the last step returned. Zero or one, and code with a conditional does the job. Several, where the branch depends on judgment about unstructured content rather than the value of a field, and you have the shape an agent exists for. The count of branches tells you more than the count of steps: a twelve-step procedure that never varies is a script, and a three-step procedure whose middle step depends on reading a free-text note is not.
Three Cases Where the Obvious Answer Is Wrong
Spans systems and changes data puts a task on the shortlist; what settles it is whether the path varies. The most commonly overbuilt project clears the first screen and fails the second: pull the file, extract the fields, validate against a schema, write the rows, notify on exception. It takes multiple steps, it touches several systems, and it ends in a write, so it belongs on the shortlist. Then you count branches and find one, at the exception. A scheduled job with a parser and a schema validator handles every case that follows the standard path deterministically, with no model in the loop, and the model earns its place only at the exception queue, where the input is unstructured and the next step depends on what it says. The shortlist test tells you the task is worth automating. The branch count tells you how much of it actually needs to reason.
The second case runs the other way. A job that ends in words looks chatbot-shaped by definition, but if answering requires calling three systems, joining the results, and knowing which of those records the person asking is allowed to see, the read side needs most of an agent's plumbing: defined tools, scopes tied to the requesting user, and a way to say "I could not reach billing" instead of inventing a number. What it does not need is the expensive half. Approval gates, rollback, idempotency and action audit all exist because of writes, and read-only work skips them entirely. If you treat agent as a single package that arrives all at once, this is where you pay for controls the problem never asked for.
The third is the swivel-chair case, and the honest answer is often neither. When a person reads from one system, retypes into a second and updates a third, the reason is almost always that the systems were never connected. An agent driving those three interfaces automates the symptom and inherits every fragility of the original: it breaks when a field is renamed, it needs credentials in three places, and it costs a model call per record forever. If the mapping between the systems is stable and expressible, the integration is cheaper to build, cheaper to run and far easier to reason about than the agent that imitates the person doing it by hand. Automate the judgment, not the retyping.
What Each Choice Costs You in Month Six
A chatbot's running cost is content. Someone has to own the corpus, because answers rot when the policy changes and the document does not. Evaluation is grading text, which is imprecise but cheap: run a question set, read the results, argue about the borderline ones. An agent's running cost is an integration surface. Every API it touches becomes a dependency you monitor: credentials rotate, rate limits apply per token, and a downstream field gets renamed so the task fails silently at step four. None of that is hard. All of it is permanent.
That difference shows up in hiring before it shows up anywhere else. A chatbot is maintained by whoever owns the knowledge, plus someone who understands retrieval. An agent needs someone comfortable with the failure modes of distributed systems: retries, timeouts, partial writes, and reading a trace to find which call in a long chain returned the wrong thing. Same line item on the budget, different job description.
On lock-in, the model is the least sticky part of the stack and the easiest thing to swap. What sticks is the tool contracts, the permission model and the memory schema. Tools defined inline in one framework's syntax become a rewrite when you change frameworks; the same tools behind an MCP server outlive the loop that calls them, because the interface is the server and any client that speaks the protocol can use it. Approval gates get sticky in a different way: once people are signing off on an action, removing that gate is a conversation with the people who sign, not a code change.
Chatbot to agent is additive rather than a rewrite, but only if you leave certain seams open from the beginning. Agent to chatbot is always easy. So when the answer is genuinely unclear, the cheap direction is to build the smaller thing while making these four decisions as though you were building the larger one.
- Keep retrieval separate from the chat surface, so the corpus and the evaluations built against it survive a change of interface.
- Reach every system of record through one defined tool layer instead of scattering calls through prompt logic, so adding a write later means adding a tool rather than rewiring.
- Carry the end user's identity through the whole call chain from day one, even while everything is read-only. Retrofitting per-user scopes onto a shared service account is the part that turns an upgrade into a rebuild.
- Log what the system did and what it was asked to do, not only what it said, so you already have a regression set of real cases on the day you need one.
How to Test the Decision Before You Commit
Most of this is measurable before an agent loop exists. The first measurement costs no code: have the people who do the work log each case for a couple of weeks, recording which systems they opened, what they decided and where they departed from the standard path. Deviation rate is the branch count sampled from reality rather than from the runbook, and it is usually lower than the team believes.
The second is draft mode, and it is the most useful test in this whole space. Run the model with read access only and have it emit the exact call it would make, the endpoint and the payload, then have a person execute or reject it. You get accuracy figures for the action layer without granting a single write scope, and the accept, edit and reject rates per action type tell you which actions could plausibly run unattended and which never will.
- Replay history. Run closed cases through the system read-only and compare the action it proposes against the action that was actually taken. Disagreements are either model errors or rules nobody wrote down, and both are worth finding now.
- Build the tool surface first and drive it by hand. Exposing the capabilities through an MCP server lets you exercise them from a script before any agent exists, which is when you find out whether the APIs support the task at all: whether the fields are there, whether idempotency keys are accepted, whether you can act as a user instead of a service account, whether the rate limits fit the volume.
- Try to write the tool descriptions. If you cannot state a tool's inputs, outputs and failure behavior in a paragraph a new hire could follow, the model cannot infer it either. That is a specification problem, and no amount of prompting fixes it.
- Inject failures on purpose. Make the second system return a 500 or a timeout during the pilot and watch what the agent does next. The behavior you see there is the behavior you get in production, and a sandbox is the cheap place to see it.
The Failure Modes Are Not Comparable
A chatbot fails loudly and cheaply. It gives a confident wrong answer, and the reader can often tell, or can check the citation if you made it cite. The quieter version is worse: retrieval returns the right document and the model summarizes the wrong section of it, or the corpus holds two versions of a policy from different years and nothing marks which one is current. But the failure that costs the most is not wrongness at all, it is abandonment. Nobody files a ticket to say the assistant was unhelpful; they stop asking and go back to email. Log unresolved sessions, not just answered ones.
An agent fails quietly and expensively, because the failure is a state change rather than a sentence. The characteristic one is partial completion: it succeeds in system A, system B times out, and now there is a refund recorded that no ledger entry matches. Retries without idempotency keys turn one timeout into two of everything. Step budgets exist because otherwise the loop retries a call that will never succeed. And the failure with the worst tail is authority: an agent holding a broad service token has more access than the person it is acting for, so any instruction embedded in the content it reads, a ticket body, an email, an attached document, becomes an action it is actually able to take. Treat everything the agent reads as data and never as instructions, and scope the token to the user rather than to the service.
The slow failure is drift between what a tool's description says and what the tool does. Someone tightens an API's validation or renames a status value, the description the model reads still says the old thing, and the agent keeps calling it confidently and wrongly. Nothing errors at the model layer, because the model did exactly what its documentation said. This is why an agent needs monitoring that a chatbot does not: alerts on the tool layer itself, action logs keyed to the request that caused them, and a periodic replay of known cases against the live tools. A chatbot that breaks generates complaints. An agent that breaks generates results that look fine until somebody reconciles the month.
Frequently Asked Questions
Can one system be both a chatbot and an agent?
Yes, and most useful deployments end up that way: one interface, with requests routed to an answering path or an acting path depending on what is asked. The thing that must not be shared is the authorization boundary. The answering path can run on broad read access to a corpus; anything that acts should carry the identity of the person asking and a scope narrow enough to describe in a sentence. Practically, that means one tool layer with two permission profiles rather than two separate products, and the routing decision logged so you can see later how often people asked it to do rather than to answer.
Does adding retrieval to a chatbot make it an agent?
No. The test is whether anything outside the conversation is different afterward, and with retrieval nothing is: the worst outcome is a wrong sentence. Retrieval does bring its own hard problem, which is not agentic but is frequently underestimated: enforcing document permissions at query time. If you build one index that every user can search, you have quietly given everyone read access to whatever was ingested, including the HR folder someone dropped in. Either filter results by the asking user's entitlements at query time or keep restricted material out of the index entirely.
If the difference is architecture, where does model choice still matter?
In three specific places. Instruction adherence across a long tool chain, where the failure is not misunderstanding the request but skipping a required step or passing a wrong argument nine calls in. Reliability of structured output, when the payload has to validate against a schema every time rather than usually. And cost and latency multiplied by step count, since a task that takes twenty calls pays the per-call price twenty times, which can make a slower, stronger model the right pick for a queue and the wrong one for an interactive assistant. Context length matters too, but usually only when a task must reason over a long transcript or a large document in one pass.
Should every agent action require human approval?
Gate by reversibility and blast radius, not uniformly. Reversible, internal and low value: run it and log it. Irreversible, customer-visible, financial, or hard to detect when wrong: gate it. Uniform gating produces approval fatigue, and a reviewer who approves without reading gives you the full cost of a control and none of its protection. The number to watch is the edit rate from draft mode. An action type that people accept unchanged across a long run of real cases is a candidate for unattended execution; one they routinely edit is telling you the rule they are applying has never been written down.
How do you evaluate an agent, when answer-grading does not apply?
You assert on end state and trajectory instead of text. Stand up a sandbox tenant or a seeded copy of the systems, take a set of real past cases, and define the final state each one should produce. Then score whether the systems ended where they should have, which tools were called, and how many steps it took to get there. Add failure injection to the same suite: a 500, a timeout, a rate limit, a malformed record, with an assertion that the agent stops safely rather than proceeding on a bad read. Chatbot evaluation asks whether the answer was faithful to its source. Agent evaluation asks whether the world matches the expected state and whether it got there without side effects nobody asked for.
What is the earliest sign we scoped the wrong one?
For an overbuilt agent, it is an approval queue where nearly every proposed action is accepted unchanged, alongside branch statistics showing one path taken almost always. That is a workflow wearing an agent's clothes, and the fix is to promote the common path to ordinary code and keep the model for the exceptions. For an underbuilt chatbot, it is usage that spikes at launch and decays, plus transcripts whose last message is the user asking it to actually do the thing. Both signals live in logs you already have, provided you are recording unresolved sessions and per-action outcomes rather than volume.
