Most of what gets demoed as an agent is a workflow. A developer decided the order of the LLM calls, and the model fills in each step. That's fine. Usually it's what you should build.
The real agents, the ones where the model decides what to do next, are rare in production for a plain reason. The model is the easy part. The code that keeps it safe is most of the work, and a demo can skip all of it.
Workflow or agent
Anthropic's article Building Effective AI Agents draws the line clearly. Workflows are systems where models and tools follow "predefined code paths". Agents are systems where the model directs its own process and decides which tools to use.
So the test is simple: who picks the next step? If your code picks it, you have a workflow. If the model picks it, you have an agent.
The same article advises finding the simplest solution that works and adding complexity only when it is needed. Six practitioners who wrote up a year of building with LLMs go further and tell teams to prioritise deterministic workflows for now, because agents are hard to deploy reliably.
There is also simple arithmetic behind this. Chip Huyen's AI Engineering points out that mistakes compound across steps: a model that is right 95% of the time per step is right only about 60% of the time over ten steps. A workflow lets you check the result between steps. An agent that runs ten steps on its own needs those checks built into the loop.
What the frameworks gave us
When the agent frameworks arrived, I tested a few of them, among them Google's ADK and the OpenAI Agents SDK. They gave us the loop and tool calling for free. The OpenAI Agents SDK describes its own job well: a built-in loop that runs until the task is complete, handoffs between agents, guardrails and tracing.
I trained my team on ADK because of what it let them see. Its web interface, ADK Web, shows every event an agent produced while it worked on a task. For people who had only built request and response APIs, watching each step of an automated workflow was the fastest way to understand what was going on inside it. Google's docs are clear that ADK Web is for development and debugging, not for production, and that is exactly how we used it.
Where we outgrew them
We outgrew the frameworks on control. Three things in particular:
- What goes into each model call.
- What is remembered between turns.
- When the loop stops.
When those are hidden inside a framework, debugging gets harder. The Anthropic article says the same: frameworks can hide the underlying prompts and responses, and wrong guesses about what a framework does underneath are a common source of errors.
So we built our own structure for memory, conversation and context, with looped reasoning made of goal-based LLM calls. Instead of one big prompt that asks the model to do everything, each call has one goal. Find the relevant documents. Pull out this value. Check this answer against the source. The loop picks the next goal from the result of the last one.
The best thing about this shape is testing. Each call is small enough to test on its own. When a run goes wrong, you can point at the call that went wrong, instead of staring at one long transcript.
Reading is easy, writing is the line
The agents I'm comfortable letting run on their own are the ones that read and suggest. An agent that searches documents and drafts an answer can be wrong, and a person reads the draft before anything happens.
Anything that writes is different. An agent that changes records needs three things before I let it near real data: someone who approves the change, a log of what it changed, and a way to undo it.
On one platform we built, we gave the agent its own set of tables. Nothing it wrote touched the existing data. Its values showed as unconfirmed until a person confirmed them, and a person's answer won over every later run of the agent. Old values were kept, not overwritten.
Then we built undo. Each run records the values it replaced, so a person can roll back a whole run, not just fix one value at a time. Keeping old values is not the same as being able to roll back. The second is what made me comfortable letting that agent near real data.
What has to exist before you ship
This is my list. Every item is code you write around the model, not something the model does.
- Permissions enforced in code, not in the prompt. A prompt can say "never delete". Only your code can make delete impossible. OWASP lists this risk as Excessive Agency and names three causes: tools that do more than the task needs, permissions wider than the task needs, and high-impact actions with no human check.
- A log of every step. What the model was given, which tool it called, with which arguments, and what came back. Without it, you can't answer a customer who asks what the agent did.
- Limits on steps and cost. A maximum number of loop turns, and a budget in tokens or money, checked before each call.
- A way to stop it. For one run, and for every run, without a deploy.
- Undo for anything it writes.
- A set of test cases you run before every change. Including ones where the right answer is "I can't do this".
The reason the model can't be its own guard is that models follow instructions wherever they find them, including inside a document or web page a tool just returned. Simon Willison calls the dangerous mix the lethal trifecta: access to private data, exposure to untrusted content, and a way to send data out. An agent with all three can be talked into leaking what it can read. A rule in the prompt won't stop that. A permission check in your code will.
Why agents stay rare
Look at that list again. None of it shows up in a demo. A demo runs once, on inputs someone picked, with a developer watching. Production runs thousands of times, on inputs nobody picked, with nobody watching.
Production agents are not rare because the models are weak. They are rare because the list above is real engineering work, and most teams stop at the demo.
My advice is the same as Anthropic's: start with a workflow. Write the steps you already know as code, and call the model inside each one. Use an agent loop only where the number and order of steps truly depend on what the tools return. When you do, build the list first. That's the work we do in agentic automation, and the same thinking sits under what you build around an LLM call. Where the agent runs changes the list too, which is the subject of server-side vs client-side agents.
One thing to do this week
Take one agent you have built or are planning. Write down every tool it can call, and mark each one read or write. For every write tool, answer three questions: who approves it, where is it logged, and how is it undone.
Any write tool with a blank answer is not ready. Turn it into a suggestion until it is.