In my last post I wrote about agents making subtle mistakes that are hard to detect unless you already know the system. Since then I’ve been thinking about the other side of that coin: sometimes the mistakes are exactly what we need.
I work with a spec-driven development flow (check spec-kit for an example of a well-defined toolkit). Before any code is written, I iterate on a spec with the agent: I write, it reviews and asks questions, I answer and update. And again. And again.
And it works, up to a point.
What I’ve found is that the spec never fully converges. There are always open issues, sometimes ones I thought we had already closed. And in the worst case there are no questions at all, because the agent quietly filled the gaps with its own assumptions. Reasonable assumptions, mostly. But still… assumptions.
Some things just can’t be solved on paper. The spec is a model of the system, and like every model, it leaves things out (the OPM person in me is nodding vigorously here 😄). You don’t discover what was left out by reviewing the model one more time. You discover it when reality pushes back.
So now, when the spec is “good enough”, I stop iterating and tell the agent to implement. And I let it fail. A wrong implementation shows me which assumption the agent made that I didn’t notice. A design that collapses halfway through tells me I was missing a constraint. Each failure goes back into the spec, and the next iteration is much better than another round of review would have been.
It reminds me of raising kids. You can explain how to ride a bicycle a hundred times, but none of it matters until they get on the bike and fall. Your job as a parent isn’t to prevent every fall. It’s to make sure the falls happen on the grass and not on the highway.
Same with agents. There is no perfect spec that prevents every mistake. So we need to create an environment where the agent can fail safely. Strong separation between dev, stage, and prod (what breaks in dev, stays in dev). Deployment guardrails on production that the agent can’t talk its way around: gated pipelines, required approvals, automatic rollbacks. A human in the loop for dangerous operations with checks that live outside the agent, not in a prompt asking it nicely to be careful. If the only thing between the agent and production is its own judgment, you don’t have a guardrail, you have a hope.
With that in place, failure becomes cheap. The falls happen on the grass, and every failure is a lesson instead of an incident.