A test nobody runs is a claim. A test that cannot fail is worse — it looks like proof.
This week I pointed two different models (Opus and Terra) at each other to evaluate the fleet's test coverage. One model (Terra) had spent the time writing tests. A second (Opus) got a single instruction: review what the first built, and believe nothing you haven't re-run.
What the reviewer found first
My home fleet had accumulated 204 test files across 33 repos — 56 added in August alone. Good tests, mostly: injected clocks, offline fixtures, assertions that pin the failure direction ("a null never renders as 0"). The reviewer ran 34 suites by hand; 33 were green.
The biggest finding wasn't a bad test. It was that nothing ran any of them. No scheduler, no CI on the private repos. One suite had been red for days with nowhere to surface it. Seven thousand lines of assertions, zero executors. Coverage isn't observation; a suite nobody schedules is documentation.
Then it attacked the tests themselves
Instead of reading them, it deleted the thing under test and re-ran the suite. The function that strips untrusted text out of memory retrieval — my guard against a fetched web page reshaping what the agent remembers — had exactly one test: `fence("hello") == "hello"`. A fixture with nothing to fence. Delete the guard entirely: still green.
The mail gate had the same shape one level up. Its test proved the gate refuses unauthorized recipients. Nothing proved any send path still calls the gate. The test checked the lock, never the door.
The loop
Builder fixed, reviewer re-measured — five passes in one morning. Their only channel was me pasting each one's report to the other, and the reviewer treated every report as a hypothesis. When the builder reported the watcher discovering all nine suites, the reviewer ran it: discovery worked, execution didn't. A doubled path in its loop failed every suite in 0.041 seconds. It surfaced in a single run because it failed loud instead of reporting green.
Nearly every pass caught one new defect inside a fix for the previous finding. The repair that unblocked a drill quietly gave my memory store an environment override — the precise hole a comment four lines below the change warns about, written after a sandbox once published test fixtures as my live memory. The author of a fix is the person least likely to catch its flaw; two contexts with opposite mandates catch what one context reviewing its own work will not.
The reviewer had to eat its own corrections too. Twice it declared something missing that existed, and said so plainly the next pass. The symmetry is what keeps the loop honest.
Day one paid for it
Within an hour of going live, the watcher caught a real failure: one test out of 658 in my ionosphere project (it's a radio thing) passed on my machine and failed under the scheduler, because it silently reached the live database instead of a fixture. Exactly the class of defect an independent runner exists to find, and it had been invisible for as long as the suite had.
The fleet's tests now run every hour and record to a ledger, and the freshness monitor watches the watcher — even the checker's death makes noise. What I gained is smaller and better than a green dashboard: when a test breaks now, something other than luck tells me.
The next real catch will tell me whether the loop holds up. I'll let you know what it finds.