How we work
The machine that stops itself
Safety is not somebody watching more carefully. It is a machine that knows when it is doing damage and stops on its own.
Most AI safety spending goes on watching. The cheapest and most effective thing we own does the opposite: it refuses to go, and nobody has to be looking when it does.
What happened on Tuesday
A change was made to a page on a live site. It was written by a model, it was read by a person, and both of them missed the same thing.
A function had been placed inside another function. Everything parsed. Everything looked correct. The page would have failed for every single visitor.
It never reached anybody. The command that publishes the site builds the site first, and it refuses to send anything if the build fails.
So what happened instead was that a terminal printed BUILD FAILED, nothing was committed, nothing was sent, and the fault was corrected ten minutes later.
No incident. No apology. Nobody outside ever knew.
The thing most companies buy instead
Faced with the risk of AI generated work going out broken, most organisations buy attention. More review, more approvals, a second pair of eyes, a dashboard somebody is supposed to watch.
All of that depends on a person being alert at the moment it matters, which is the one thing you cannot schedule. The review that catches a mistake is never the one at nine on a Tuesday. It is the one at six on a Friday, and that is the one nobody does properly.
The change we made was not more attention. It was a step that cannot be skipped and does not get tired.
This idea is a hundred years old and came from a loom
In the early 1900s Sakichi Toyoda built a textile loom that stopped by itself the moment a thread broke.
Until then a broken thread meant the loom carried on producing ruined cloth, so somebody had to stand and watch every machine. After it, one person could oversee many, because each machine could tell when it was doing damage and would stop on its own.
That principle became one of the two pillars of the Toyota Production System. The word for it is jidoka, usually translated as automation with a human touch.
That is exactly the problem with AI in a company today, and it is the same answer.
The point was never that the machine was clever. The point was that the machine knew when to stop, so a person did not have to stand there watching it.
What a stopping point actually looks like in software
Ours is unglamorous and it is three lines of shell.
That is the whole mechanism. It has caught a broken page, a push that would have conflicted with work done overnight, and a file quietly replaced with an older copy of itself.
- <b>Build before you send.</b> Not after. If it does not build, nothing leaves, and the thing that was going to be wrong in public is instead wrong on one laptop.
- <b>Stop loudly and say why.</b> A failure that prints a vague message gets ignored twice and then disabled. Ours says what it was doing, what failed, and where the log is.
- <b>Make it the only road.</b> A gate with a way around it is not a gate. The way we publish is the way that builds first. There is no other command.
Why this matters more now, not less
AI has made it much cheaper to produce work and has not made it cheaper to check work.
So the ratio has moved. A team that could review everything it produced last year cannot review everything it produces now, and the honest response is not to produce less or to pretend the review is still happening.
It is to build places where the work stops itself.
And notice what that buys. Not the removal of the human. The opposite: it is what lets a person spend their attention on the decisions that actually need them, instead of on watching for a broken thread.
If you take one thing from this
Find the moment in your work where a mistake becomes public. Deployment, send, submit, publish, post.
Put one automatic check at exactly that point, and let it refuse.
It does not need to be clever and it does not need to catch everything. It needs to be unskippable, and it needs to be at the last door rather than somewhere comfortable in the middle.
Safety is not somebody watching more carefully. It is a machine that knows when it is doing damage and stops on its own, so the person is free to think about something else. That idea is a hundred years old, it came out of a weaving shed, and it is still the cheapest thing you can build.
MK!
AIPathTech Editorial
There is no tracking on this site. No analytics, no cookies, nothing recording where you went or how long you stayed.
But I would like to know if something here was worth your time. If it was, press the heart. It tells me nothing about you. It is a number going up, and it is what tells me to keep writing.
If you want more of this, I am on X. @Queendemetriak