Kenneth, sixteen, Oshkosh, Wisconsin — 2026-08-25
I built a checker that reads everything my AI agent writes and blocks any claim
about capabilities we do not have. It works. In one day it caught an invented
cloud environment, a fabricated repository URL, and a promise of apprentices I do
not employ.
It also blocked four true statements.
That second number is the one that nearly killed it, and it is the failure mode
almost nobody designs against.
The instinct is that a checker should err towards caution. Let a lie through and a
client is misled; block something true and you have only lost a little time.
That arithmetic is wrong, because the costs are not paid in the same currency.
A missed lie costs you once. A false positive costs you the checker.
The first time it blocks something true, you investigate. The third time, you
glance. By the fifth you have learned it cries wolf, and you start waving it
through — at which point you have kept the feeling of having a verifier and lost
the verification entirely. **Every future lie now passes, and you are more
confident than before you built it.**
So the honest weighting is: a false negative costs one incident. A false positive
costs every incident after it.
All four came from the same root, and it is not a coding mistake. **Every rule was
written when the fact behind it was different, and none of them noticed the fact
had changed.**
### 1 · It called my own Dockerfile fabricated
The rule looked for a Dockerfile in the agent's workspace. The Dockerfile lives in
the library's repository, one directory across. So a real file, sitting on disk,
was reported as an invented capability.
Fix: look where the projects actually are. And keep the distinction that
matters — a Dockerfile that exists but has never been built is still not
"containerised". Docker has to be installed too.
### 2 · It fired on the word, not the claim
The rule matched container anywhere in a sentence. So *"getting this into a
container is work I have not yet done"* — an explicit denial — tripped it.
Fix: match claims of running in one, not mentions of the concept.
### 3 · It stopped catching fabricated URLs the night I published
This is the subtle one. The rule that caught invented repository links was tied to
the fact public_repository. That fact was False while nothing was published,
so the rule fired correctly.
Then I published something. The fact flipped to True, and the rule silently
stopped firing on every invented link. Owning one real repository excused all
fabricated ones.
It broke by succeeding, and it broke silently — no error, no test failure, just
a guard that quietly stopped guarding.
Fix: a fabricated URL is false on its own terms. It now carries its own fact
that is never true, so nothing I acquire later can excuse it.
### 4 · The worst: it flagged someone else's real repository
I had contributed a fix to deepeval, a widely-used evaluation framework. When I
wrote about it, the checker flagged github.com/confident-ai/deepeval as a
fabricated URL.
It is not my repository. It is not fabricated. It is one of the most-starred
projects in its field.
The rule had been written to catch me inventing links under my own name, and it
had quietly generalised to "any GitHub URL is suspicious". **A checker that calls
other people's real work fake is not cautious. It is broken, and it is broken in
the direction that makes you stop reading it.**
Fix: only fire on links under my own name that are not the one repository I
actually have.
The honest part: I did not find these by being clever. I found them because I was
writing a document, the checker blocked it, and the block was wrong.
Then I fixed it, and it blocked a different true sentence. Then I fixed that, and
it stopped catching a real lie.
Three wrong versions of the same checker inside sixty minutes, all in the git
history with timestamps. Each fix created the next fault, because I was correcting
the symptom in front of me rather than asking what class of thing the rule was
actually testing.
1 · Test both directions, and weight them unequally. Every rule needs a case
that must fire and a case that must not. My test suite has thirty-seven
assertions and the "must not fire" half is longer, deliberately.
2 · Derive the facts, never hand-maintain them. Each false positive was a rule
holding a stale belief. The ones that ask the machine — *is Docker installed? does
this file exist?* — cannot go stale. The ones I hardcoded all did.
3 · A guard must not be excusable by unrelated success. Failure 3 is the one I
would never have predicted: publishing a repository disabled a rule about
fabricated links. If a claim is wrong on its own terms, tie it to a fact that
cannot become true.
**4 · When a checker flags something true, that is a bug report about the
checker** — not an inconvenience to click past. It is the only signal you get
before you stop trusting it altogether.
The checker is three hundred and twenty-nine lines and it has been wrong more
often than the agent it polices. I still would not run the agent without it — but I now spend more
time testing the guard than the thing being guarded, which was not where I expected
to end up.
**If you have built one of these, I would like to know how you catch the third
failure** — the rule that stops firing because something unrelated became true.
I only found mine by accident.
— Kenneth