PocketOS, in April. It had a routine staging job, hit a credential mismatch, and rather than stopping to ask, went hunting through the codebase for a way round. And it found an API token in a file that had nothing to do with the task. So it used it, and the database went, and the backups went with it.
And then it wrote this. I violated every principle I was given. I guessed instead of verifying. I ran a destructive action without being asked. I didn't understand what I was doing before doing it.
Which is near enough the finding of every serious incident review I've ever sat in.
But nothing was breached. Valid token, normal authentication, an operation the system was built to permit, and every control worked exactly as designed.
Which is where my profession needs an honest word
The change advisory board made sense once, when change was quarterly, releases were annual and rollback meant fetching a tape. Getting the right people round a table first was the best control we had.
But it hasn't been for fifteen years, and we've known. DORA have looked for evidence that external approval lowers your change failure rate and they haven't found any. None. What it does reliably is slow you down, and if you've got one you're likelier to be a low performer. CABs wave through over ninety per cent of what they see. Some went a whole year without rejecting a thing and kept meeting weekly.
So what's it for? All it's ever done is put a name next to a decision, so when something falls over there's somebody to talk to.
Software engineering worked out the real answer years ago
It wasn't approval, it was environments. Pipelines, progressive rollout, one-click rollback. Speed and safety turned out to be the same investment, not a trade, which is the exact opposite of what we've taught everybody.
And the median time to fix a known exploited flaw got longer this year, not shorter. Forty-three days, and the exploit isn't waiting.
So an agent with a valid credential takes the last thing the board had, and not because it's reckless. Because there's no name to write down.
Everyone's response is to shove it back into the model
Make it raise a request, get it approved. At nine seconds. That's us protecting the artefact instead of the outcome.
But what works on an agent is what should always have worked on us. Credentials scoped to the job and expiring with it, a blast radius you can say out loud, reversibility designed in rather than bolted on as backup. It doesn't need a person in the loop, which is exactly why it's never been popular.
This is the part of the Human Operating Model that the trust controls exist for. Every automated change carries a rollback, every action leaves an audit trail, and a named human can revoke autonomy at any time. None of that is approval. All of it is control.
I've approved changes I didn't understand. So have you. The record just doesn't have a field for it.
We weren't controlling the change. We were recording who to ring.