Your Agent Isn't Real Until It Survives Its Own Maintenance Bill

Full episode transcript

Your agent isn't real until it survives its own maintenance bill. That is the whole test, and it has nothing to do with the demo. What makes something real rather than a demo is not how good it looks on the day you show it off to the room. It is what is left over after you subtract everything it costs you to keep the thing alive. Most of what is getting called an agent right now has never had that subtraction run on it once. So let me say what I mean by real, because that word is doing a lot of work for a lot of people right now. A real agent runs by itself with little downtime, and it creates a net productive gain after maintenance, review, confusion, mistakes, and recovery are counted. Both halves have to be true. It runs on its own, and the gain still stands once you count what it costs to have it. Start with the first half, because it is the cheaper thing to check. If you are the reason it ran today, it is not running. Somebody opening a tab, pasting the input, kicking off the job, then checking that it finished is a person doing the work with extra steps in front of it. Little downtime is not a nice feature sitting on top of the real thing. It is most of what separates a system from a very impressive script that you personally operate. And I want to be fair here, because a lot of good work looks exactly like that early on, and that is fine. Everything starts supervised. The question is whether you are still standing over it in month three. If nobody has taken their hands off the wheel by then, you did not build an agent. You built yourself a job that you now have to keep showing up for. Now the second half, which is where almost all of them die. The gain is not what the thing produces. The gain is what the thing produces minus what it costs you to have it. And the cost people quote is the subscription, which is the cheapest line on the page and nowhere near the real number. The expensive part is always people. So here is the bill. Maintenance. Somebody keeps it working when the inputs change, and the inputs always change. Review. Somebody re-reads everything it produced, because nobody trusts it yet, and that reading is not free. Confusion. Somebody explains to a person in the building why it did that. Mistakes. It gets one wrong and the wrong one goes out the door. Recovery. Somebody finds it, undoes it, and apologizes for it. Every one of those is a real cost, and every one of them comes out of the gain. Not out of some other budget. Out of the gain. People account for these like they happen somewhere else, like the review hour belongs to the reviewer's day instead of to the agent's ledger. It does not. If the thing you built created that hour, that hour is charged to the thing you built. The short version of this is the one I keep coming back to. Saving five minutes after spending all day maintaining it is an experiment, not business value. And when you run that subtraction honestly, a lot of deployments go underwater right away. It saves a few minutes on each run. Then somebody spends most of a week fixing it when the input format shifts, re-reading everything it produced because nobody trusts it yet, and walking back the one thing it got wrong in front of a customer. You did not gain time. You bought yourself a part-time job with a very good demo attached to it. That is not an agent. That is an experiment you are paying for. And I want to be clear that I am not against experiments. I run them constantly. Everybody building anything real runs them. Half of what eventually works started out costing more than it returned for a while, and that is how you find the shape of the thing. Because the problem is not running the experiment. The problem is reporting the experiment as return. Somebody stands up in a meeting and says we automated intake, and what is actually true is that intake now takes about the same total human time, distributed differently, plus a tool bill. Everybody nods. Nobody in the room asks who is still touching it. That is how a company ends up with nine of these running and no capacity back. So call the thing what it is. If it is an experiment, say experiment. Say we are learning where this breaks, here is what it costs us right now, here is what has to be true before it counts as a win. Nobody is going to fire you for that. What gets you in trouble is the other version, where the number in the deck does not survive five minutes with the people who actually run the process. The test I would actually apply is boring. A week goes by. You did not think about it. Nobody escalated it. Nobody asked you why it did that. And the work still got done. If that is true for a full week, you have something real. If you cannot get through one week without touching it, you have a promising experiment, and you should keep going, but it does not go in the ROI column yet. And people are not failing this bar because they are bad at it. They are failing it because nobody told them there was a subtraction at the end. The demo culture around this stuff only ever shows the top line. It shows the thing working once, on a clean input, with the person who built it sitting right there. That is the easiest condition it will ever face, and it is the one condition that never happens again. So before you tell anybody you have an agent, run the bill. Maintenance, review, confusion, mistakes, recovery. Subtract all of it. If what is left is still a real gain, you built something, and you should be loud about it, because it is rarer than the feed makes it look. If what is left is five minutes, be honest about that too. The honest version is the only one you can actually build on.