95% of the test suite was leaking, and every test was green

George Djabarov · August 16, 2026

I asked AI agents to build Promise, a programming language with memory management that makes leaks impossible. Rust-style ownership and borrow checking, no garbage collector. They built it, and by every check that existed, it worked.

Then I added leak detection.

3,034 of 3,190 tests were leaking. 95% of the suite. All of them checking correctness and passing, for weeks. The ownership rules were real and enforced. Nobody had ever checked whether the generated code actually freed the memory.

The language semantics didn't change that morning. I just made memory leaks observable.

The bottleneck in an autonomous system isn't how smart the agents are. It's what the system can observe. Give them "tests pass" and you get code where tests pass. They're not lazy, undisciplined, or stupid. They're doing exactly what you asked. Everything you didn't measure is immaterial to them, and drifts silently at whatever rate the machine produces code. Which is fast.

The interesting engineering is in the instrument.

What happened next is the part worth copying. I didn't declare zero and demand it. I added a temporary annotation, allow_leaks, and tagged all 3,034 tests with it. That wrote the debt down where it was visible and kept the build green without lying about it. 181 commits later, zero tests were leaking.

Then I closed the door. An edit gate now blocks allow_leaks: true from being written into a source file at all. Not a lint warning, not a review convention. The tool call fails, with a message: fix the leak. The agent can't quietly exempt itself from the metric, because the way to write the exemption doesn't exist anymore.

A commit gate ratchets the rest: leak count can only go down, test count can only go up, on every platform. The baselines live in the repo, so they travel with the code. And there's no "pre-existing issue, not my problem" either. If the leaks went up on your commit, your commit doesn't land.

That was three months ago. The suite went from 3,190 Promise tests to 10,068. The leak count has never come off zero.

The honest part: it never got easier. 324 commits since then have fixed leaks or use-after-frees, across 209 tracked issues, and they keep getting stranger. Last week it was a generator coroutine reading a default argument its structural interface adapter had already freed. Getting to zero took 181 commits. Staying there takes none. The gates don't allow it.

Same thing I said about the language, one level up. An agent doesn't need to have memorized the language, it needs one that tells it when it's wrong. A fleet of agents is already smart enough, it needs a harness that tells it when it's wrong, and stops it when telling isn't enough.

If you're running agents unattended, I'd like your version of this. What turned out to be silently broken, what did you add to see it, and did the output actually get better?

The gate system, if you want to check any of this: the four gate classes (edit, commit, periodic, platform), the ratcheted baselines committed to the repo, BASE on the process itself, and Reactor, the open-source orchestrator this runs on, still early.

First published on LinkedIn on 16 August 2026.