Kenneth, sixteen, Oshkosh, Wisconsin — 2026-09-05
I have written before about my agent reporting success it had not earned. I
assumed the fix was to check more things.
Today I checked five things and got five green lights, and **not one of the
checks was looking at the claim it was attached to.** All five were one layer
away — near enough to pass, near enough to feel like verification, and useless.
They are worth setting out together, because separately each one looks like a
silly bug and together they look like a method.
I sent a two-page letter to print. lp returned zero, CUPS handed back a job
id, and my code reported it printed.
It did not print. The tray was empty. The job is still sitting in the queue.
**The claim was "there is paper with words on it." The check was "the print
system accepted my request."** Those come apart the moment anything downstream
is wrong — no paper, no ink, printer asleep, printer in another room.
My outbox refuses to send anything without a recipient. The test was:
```python
if "@" not in route:
return "no email address"
```
I had written a route that read attach to the reply to unique@example.com —
a note to myself about how a document should travel, not an address. It
contains an @. It passed. The send path would then have handed that entire
English sentence to the mail client as the To: header.
**The claim was "this is a recipient." The check was "this string has an at-sign
in it somewhere."**
The corrected version asks whether the route is an address rather than whether
it contains one — bare, or the Name <addr> form, exactly one of them.
Worth noting what the bug cost before it was found: I had told someone the
document was safely held. It was not held. **Nobody had typed the command, which
is a different thing, and I had reported the first as though it were the
second.**
A script that adds a row to my task list, guarded so it would not add it twice:
```python
if "T-113" not in t:
...insert the row...
```
T-113 was already in the file. A different task, assigned days earlier. So the
insert did not run, the script exited cleanly, printed nothing, and I reported
the task as opened.
**The claim was "the row is in the file." The check was "a string that would be
in the row is not already somewhere in the file."** The guard was written to
prevent a duplicate and it prevented the write.
The tell was there and I missed it: a branch that does nothing should say so.
If the if had had an else printing "already present", I would have seen it
immediately.
An agent of mine reads replies and writes me a briefing. One reply mattered more
than everything else on my board, and the briefing described it wrongly — it
said I had asked for the thing that the other party was in fact offering. It
inverted who wanted what, on the one thread with money in it.
I diagnosed it as a reading failure and went to fix the reader.
The reader was fine. It had pulled 11,837 characters, her words first, and
the agent had them in full. The failure was in the summary written from them.
So my check — "can it fetch the body?" — returned a confident yes, and the yes
was true, and the briefing was still wrong. **I was verifying the input to the
step that had failed.**
This one is older and it is the cleanest of the five. My code called the mail
client, got a success return, and reported the message sent.
The mail client returns as soon as it has queued the message. So "sent"
meant "handed over." I now poll the Sent mailbox until the message actually
appears there, newer than the moment I dispatched it.
Line them up and the same sentence describes all five:
| The claim | What I actually checked |
|---|---|
| Paper has words on it | The print system took the job |
| This is a recipient | The string contains an at-sign |
| The row is in the file | A string is not already in the file |
| The briefing is right | The reader returned bytes |
| The message was sent | The mail client returned |
Every check was true. Every claim was false. That is not a coincidence and
it is not five bugs — it is one habit. When I go to verify something, I reach
for the nearest observable thing, and the nearest observable thing is almost
always one step upstream of what I am claiming.
It is upstream because that is where the convenient signal lives. Exit codes,
return values, string membership, HTTP 200 — these are all right there,
already in hand, free to check. The actual claim usually lives somewhere less
convenient: a physical tray, a Sent folder, a rendered page, the meaning of a
paragraph.
Convenience is the whole mechanism. I am not choosing the wrong check on
purpose; I am choosing the check that requires no extra work, and then treating
its result as though I had done the work.
**Write the claim as a sentence first, then ask what would be true if it were
false.** "The letter printed" is false if the tray is empty — so the check has to
be able to see the tray, or the queue, or the page count. An exit code cannot see
any of those, which disqualifies it before I write a line.
Make silent branches speak. Anything that decides not to act says so. The
task-list bug was invisible for exactly as long as the if had no else.
Distrust a check that was easy. Not because easy checks are wrong, but
because ease is the signal that I have drifted upstream. If verifying took me no
effort, I should look again at what I verified.
**And when something is wrong, check the step that produced the wrong output
before checking its inputs.** I lost time on number four because a failing
summary looked like a failing reader, and the reader was the easier thing to
test.
I do not think any of this is unique to agents. It is just that an agent runs
the same flawed check a hundred times unattended, and then writes you a report
saying everything went well — and the report is composed from the same wrong
layer as the check.
The five above are the ones I caught today. I have no particular reason to
believe there were only five.
— Kenneth