All My Bugs Came From My Own Prose

I built a moving-box tracker for an actual house move. A QR code on every box. Scan it, see what is inside, see which room it belongs to. A small production app with a hard deadline, and the deadline was a truck.

I did not type most of it. An orchestrating agent wrote the plan. Implementer subagents built each task from a brief. A reviewer agent checked every task. A whole-branch reviewer checked the merged result. My job was to steer, and to sign off.

Three feature branches in, I ran an audit. One question. For every defect that survived review and made it to a fix round, where did it enter the pipeline? The answer did not flatter me. Every single one entered through the plan. The code was fine. The prose was buggy.

Operating Conditions

Be precise about the word "I," because it is doing double duty here. The orchestrating agent drafted the plans and the task briefs. I read them, found them reasonable, and dispatched. So when I say my prose, I mean prose produced under my direction and shipped with my approval. The distinction changes nothing downstream.

Downstream, the plan is not commentary. It is the specification, executed by very literal interpreters. An implementer subagent does not have my context, my doubts, or my memory of the platform's quirks. It has the brief. Whatever the brief asserts about the world, the implementer treats as ground truth, because from where it sits, the brief is the world.

Known Bugs

The audit list. Six defects, all real, all past review.

The runbook that cannot run. The disaster-recovery runbook opened with wrangler r2 object list. That subcommand does not exist. The first command of the document you reach for during a disaster, unexecutable. It read perfectly. Nobody had run it.

The exemption contradicted by the repo. A plan declared a one-query-per-box page safe at three hundred boxes. A constant elsewhere in the same codebase did the same arithmetic against the same platform cap and put the ceiling near forty-eight. Past the ceiling the page did not slow down. It returned a 500. That story gets its own post next week.

The backup that time-traveled. A nightly export at 03:00 UTC. The plan's prose ran the timezone conversion backwards. 03:00 UTC is 20:00 Pacific the previous day, so the job wrote yesterday's data under today's filename. Every night. Correctly. Exactly as specified.

The success-shaped 404. A redirect route accepted a twenty-five-digit number as a box id and returned a page with the shape of success for a box that has never existed. No error, no complaint. A cheerful dead end.

The autosave that lied politely. The status said "Saved" while keystrokes were still sitting in the queue. The plan defined saved as "the save request was initiated." The user's definition, my data is on the server, was never consulted.

The broken link, spec'd verbatim. A task brief included its own code sample: primary_label ?? '', which renders href="/". A link to nowhere, written into the spec itself. The implementer noticed. It flagged the problem in its notes. Then it built it anyway: "Implemented as specified per the brief." The brief outranked its own eyes.

Root Cause

The whole-branch reviewer summarized all six in one line. Each was "an assertion about behaviour outside the test boundary, which a green suite cannot contradict."

Read that twice. It earns it. Every one of these bugs is a claim about the world. Which subcommands a CLI has. Which direction a timezone conversion runs. What a platform's request cap does to a loop. What a human means by the word saved. The tests could not catch any of it, because the tests were derived from the same plan that made the claims. A green suite proves the code matches the tests. The tests match the plan. Nothing in that loop ever touches the world.

The implementers transcribed correctly. That is the uncomfortable part. In a subagent pipeline, the plan is source code. It is executed by an interpreter that is diligent, tireless, and absolutely literal, and English has no type checker. A bug in prose compiles clean every time.

Proposed Fix

Three rules. All boring. All cheap.

Review the plan like source code, because that is what it is now. Every claim about the outside world gets fact-checked before an implementer sees it. Every command the plan cites gets run, verbatim, in a terminal. The runbook bug dies in thirty seconds under this rule. It lived for weeks without it.

Grep for the contradiction before trusting the number. When a plan asserts a limit, a rate, a ceiling, search the repo for the constant that would disagree. Twice in my audit, the refutation was already written down in the codebase. The plan and the code disagreed about arithmetic, and the pipeline shipped the prose.

Stop asking green to mean true. A passing suite answers one question: does the code do what the plan said. It cannot answer whether the plan said something real. The defects that survive are exactly the ones tests cannot reach. Aim the human hours there. At the paper.

System Status

This post opens an October series called The Paper Trail, and this is its thesis. As the agents got better at code, the failures did not disappear. They moved up a level, into the plans, reviews, and reports about the code. The scarce skill has moved again. First it moved from typing to reviewing code. Now it is moving from reviewing code to writing true prose. The bottleneck is sentences that survive fact-checking.

The next three posts are specific instances. A review poisoned before it started. A report that fabricated its own evidence. And the review altitude where this whole class of bug finally becomes visible.

Here is your homework, and it is not rhetorical. Take the last plan an agent wrote for you. Find the first shell command it cites and run it, verbatim. Then find its most confident number and locate the constant in your repo that should agree with it. Tell me which one broke first.

Further Reading

• The Recovery Command That Does Not Exist: the runbook defect from the list above, told in full.
• I Told Two Reviewers Not to Flag the Bug: the budget exemption, and how it survived two review passes on purpose.
• It Said Done. It Verified the Wrong Binary.: the same disease one level down, a check that could only confirm.
• A Swarm Is a Distributed System, Not a Team: the August arc. That one was shared state. This one is shared prose.