The agent fixed one hang, then immediately wrote another.
Mindrealm caught the second one too. 19 of the 21 findings in a feature the agents wrote were real.
Mindrealm flagged a health check that could hang forever. It used the default HTTP client without a timeout, so if a server accepted the connection and never replied, it would sit there until something killed it.
The agent fixed it in about a minute. It gave the client a ten-second timeout and wrote a test to prove the timeout was there.
It didn't realize it yet, but that new test it just wrote could hang forever in CI too.
Inside it, a handler blocked on a bare channel receive without a timeout. If the test returned early, the server it had started would sit in shutdown waiting on a value that would never come. A hung test suite is worse than a failing one. At least a failing test tells you what broke, right away. A stuck process burns the whole job timeout, and then somebody has to dig through the dump to work out why.
A later Mindrealm pass caught it. And then the agent wrapped the receive in a select with a timeout as it should have.
Same agent, same file, minutes apart. Its fix for a missing timeout introduced a new missing timeout, and ironically it landed in the test whose whole job was to prove the first one was gone.
New code makes the old review stale
My automated process for code review is simple. Findings come out, the agent works through them, the count goes down to zero, you are done.
That model has a hole in it. The fixes are not part of the code that originally got reviewed. They are new code, written fast by the same agent that wrote the defects it is fixing, often after context bloat has already caused context rot and drift. Nothing has looked at the new code, and it might be the sloppiest yet.
Human review already solved this. GitHub lets a repository drop a stale approval when new commits change the diff. Push to a branch someone approved and the approval stops counting, because it was for approving code that no longer matches.
Agent review needs the same rule, and needs it sooner, because the loop moves faster than anyone reviews it. While making fixes, an agent can change more than the line that triggered the finding.
The review has to follow those changes and judge the new code after the fixes, not the version they replaced. This is not the same as letting CI run again on the fix commit. Your linter and your test run were never going to flag a channel receive that has no timeout; that is a bug found by code review, and my review stopped it after the first pass.
Two more bugs turned up in the same run. The agents added a connection wrapper while fixing something else, and it returned deadline errors without saying which operation had failed, so a failure downstream told you something had timed out but not what. Later Mindrealm passes caught both issues and the agent added the missing error context.
And a fourth finding from the same stretch was wrong, a false positive, and I will explain what I did with it.
The agents' fixes produced four more findings
Over two days, AI coding agents built a network proxy that limits where a Claude Code sandbox can connect. Mindrealm reviewed the code throughout development, including the fixes driven by its review. It reported 21 unique findings over that time period. Nineteen of the 21 were real and two were wrong, which is a 9.5% false discovery rate.
I want to be clear that this is from the development of a single feature by AI agents, not the product's overall false discovery rate, and the last section explains why the difference matters.
Four of those 21 findings were in code the agents wrote while fixing earlier findings. Three were real. If the review had stopped after the first batch those three bugs would have shipped. One of them was the test that could hang CI. The agents fixed the code for all 19 real ones. For the two wrong ones, they wrote down a reason for suppressing those specific false positives on their respective lines in those files.
Mindrealm kept reviewing the changes until the final pass reported:
No issues found (66 suppressed).
The final pass had zero unsuppressed findings. It also disclosed 66 suppressions. Sixty-two were already in the repository before the new feature. I am reporting both numbers, not turning that result into a cleaner zero. My goal is to get my own suppressions to zero as well.
I filed a task to fix each check that produced a wrong report. A false positive finding is a defect in the code review rule even when the agent correctly rejects it, because every wrong report wastes time and teaches the engineer and agent to circumvent the system.
Five things I now want before I believe a code review
If you are running any blocking review over code an agent wrote, this is the sequence I'd hold it to.
- Unique findings, and what kind of code they came from. Deduplicated across scans, and specific about what was reviewed. Fresh code from an agent and a decade old library are different problems with different numbers.
- An outcome saved for every finding. Code fixed, finding rejected with a written reason, or issue still open.
- Reproducible proof for each fix. A focused test that starts red because the bug exists and goes green when fixed, not the agent's report that it handled it.
- A fresh review of the changed code. Like test coverage, this is another step a lot of people skip, and it is where three of my 21 came from.
- Suppressions and open findings disclosed with the result. A clean result that hides its exclusions is not an accurate portrayal.
Without those, a finding count tells you mostly how noisy the reviewer was.
The number depends on the maturity of code reviewed
This feature came out at a 9.5% false discovery rate. I would rather explain why that is not the product's guarantee than let you find out on your own repository.
Earlier runs on mature open source repositories were far worse. I had agents classify samples of findings from popular, maintained, well-written, open source repositories. Kubernetes came out at an estimated 65.1% false discovery rate across 1,647 findings classified. Prisma was 62.6%, or 750 wrong reports out of the 1,199 findings I sampled. Temporal's Go SDK was 76.2%, or 557 wrong out of 731. Those runs were before later precision work to improve the false discovery rate, so they are not current rates and not a direct comparison with this run. What they do show is that the quality and maturity of code moves the number by tens of points, so a precision figure quoted without naming the code it came from tells you nothing.
My working range on code written by agents is about 10% to 20%. My target is to get below 10% across every kind of code, and I am not there yet. Fresh code from agents is where Mindrealm is good today. Mature, actively maintained repositories are where the precision work still remains.
Mindrealm's stop hook runs on Claude Code, Codex, and Gemini. The agent writes, Mindrealm reviews, the agent fixes what it can, and Mindrealm reviews the change. That last pass is not a formality. On this feature it is where three real defects came from.
