I run a multi-agent pipeline that ships software on its own. Build, QA, merge. No human in the middle.
It mostly works. “Mostly” is doing a lot of work in that sentence.
Every week it finds a new way to break. Then this week taught me something bigger: the failures aren’t new. They’re the same five, wearing different coats.
Same bug, five times
Tuesday was a bad day. Five separate lanes froze. Five separate tickets, five separate components, five separate “investigations.”
One root cause.
An agent claimed a ticket and never released it. No staleness rule. No release. The lane deadlocked. It happened to the coder lane, the QA lane, the dispatch layer, twice over — five instances of the same class in a single day.
We used to fix each one as its own fire. Investigate from scratch. Patch the symptom. Wait for the next lane to freeze.
Now we classify first. Blocker arrives → match it against the known classes → apply the playbook. No new investigation. No new heroics.
The log became the product
Every failure I’ve ever logged is now three things at once: a warning signal, a fix recipe, and an entry in a knowledge base.
A new pattern still gets fixed the old way — diagnose, fix, write it down. But the instant it’s written down, it’s no longer a fire. It’s a known case. It becomes a feature, not a fire. The backlog of lessons is the actual asset.
The failure set is finite
That’s the thing I keep circling. The failure set is finite, not infinite.
A claim without a release is a deadlock. A scanner reading keywords instead of structure will eventually match its own reflection. A verdict hidden behind an old comment list gets missed. A route pointing at the wrong port kills every lane on it. These aren’t a thousand bugs. They’re maybe five classes, recurring across every component I build.
Once you treat them as classes, the whack-a-mole feeling dies. Each component used to carry its own private copy of the same bug. Now there’s one rule, shared everywhere: old + no activity = release.
What I’m not telling you
The framework design. The state machines. The exact playbooks. The metrics. The product name. That stays in the lab.
What I will say: it’s turning into something sellable. An evidence-backed control plane for people who run agents in production. The incidents are the training data. The register is the brain.
Every one of those failures became a rule. Not a lesson. A rule.
One more thing
I’ve started sketching what this looks like when you watch it live — lanes as slowly turning rings, each one carrying its own history. When a ring stops turning, you look. When it jolts, you act. When a new mark appears on it, that’s a new class — and a new feature on the roadmap.
Most people watch dashboards. I’m building one that feels like a machine you can hear breathe.
Either way, it’s going to be interesting.
Leave a Reply