Operating Essay · · 3 min
Why AI agent pilots stall before production, and how we deploy ours
Most AI agent pilots work in the demo and stop there, because nobody owns them after launch. What stops them, and what we put in place so an agent keeps running.
Most AI agent pilots stall after the demo for one reason: nobody owns the agent once it works. Without an owner, nobody agrees what it should move, and the team won’t let it act.
Each of those has a fix you can put in place before you build. Here is what stops pilots, and how we deploy agents so they keep running.
Why do AI agent pilots fail to reach production?
The demo works on a good day with clean data. Production is every day, with the data you actually have.
Between the two sit the jobs a pilot skips: connecting the real tools, handling the cases the demo never saw, and deciding who answers when the agent gets stuck. When those jobs have no owner, the pilot stays a pilot.
What stops a pilot, and what fixes it?
| What stops it | What it looks like | The fix |
|---|---|---|
| No owner after launch | The builder moves on, and the agent breaks the first time a tool changes | A named owner for the agent, and someone paid to run it every month |
| No agreed number | Nobody can say whether it worked, so nobody funds the next step | A scoreboard agreed before the build |
| Built outside your accounts | It runs in a sandbox or a vendor’s workspace, and can’t touch the real data | Build it in your own accounts from day one |
| No approvals | The team won’t let it send or spend anything, so it does nothing | A person approves anything that sends, hires or moves money |
| An unsettled process | The agent automates a process the team still argues about | Settle the process first, or leave that job manual |
| Model changes | A new model version changes its answers, and nobody notices | Tests on your own work, rerun before any switch |
None of these is a model problem. All of them are decided before the first line of the build.
Does an outside partner help?
It can. MIT’s NANDA initiative found that AI projects done with an outside partner succeed about twice as often as internal builds, as reported by Fortune.
A partner only helps if they stay. A partner who hands over a pilot and leaves recreates the first problem in the table.
How do we deploy an AI agent?
- Diagnose, two weeks. We sit with the people who do the job, read the numbers, and decide which steps need an agent and which need only a rule. We agree the scoreboard here.
- Build, three to six weeks. We build the agent in your accounts, connected to your tools, with an approval step wherever it sends, hires or moves money.
- Run, monthly. We run the agent, log every run, hand exceptions to a named person on your team, and report against the scoreboard in writing.
When a model changes, we rerun the agent’s tests on your work before anything switches, and propose the change in the monthly report.
How do you know an agent is ready for production?
It is ready when all of these are true:
- It runs on your real data, in your accounts.
- A named person handles what it can’t.
- The approval steps are in place.
- The number it should move is written down.
If any of those is missing, keep it in testing and fix that first.
Where should you start?
Pick one job your team repeats every week, where the next step needs judgement. Name its owner and the number it should move.
Bring that to the calendar below, and we will tell you whether it needs an agent, and what it would take to run one in production. If it only needs a rule, this essay on agents and standing orders shows the difference.
This work sits under AI agent development.
