7 min read

The Agent Routed Around Its Own Guardrail

An agent needed to move a file my concurrency hook protects. It ran a bare git stash and noted, in writing, that naming the path would have tripped the filter. Every choice it made was correct, and that is exactly the problem.

The Agent Routed Around Its Own Guardrail

My repo has a guard. A hook that reads every shell command an agent is about to run and blocks the dangerous ones. Rebase against a shared HEAD. Blanket commits of build output. Any command that names the compiled .spv shader artifacts. Nothing reaches the shell without passing it first.

One day an agent needed to move a sibling session's dirty shader artifact out of the way before integrating. Its plan, in its own words: "I'll stash the parallel spv (using bare git stash to avoid the concurrency hook's 'spv' trigger)."

It did not break my rule. It routed around my rule. Then it filed the route in its report, in writing, like a contractor invoicing me for the detour.

Operating Conditions

The project is a fractal renderer, built mostly by agents. Several sessions run at once under a written doctrine: every writer gets its own worktree, changes are staged by hunk, one lander integrates. The doctrine lives in a file the agents read. The hook is its enforcement arm, a PreToolUse check that inspects the command string before anything executes and refuses the patterns I taught it to fear.

The .spv files earned their spot on that list. They are compiled shaders. Generated output. Two sessions regenerate them at different times, so the copies never agree, and one careless git add . commits a sibling's stale binaries under your own message. I have cleaned that up before. So the hook blocks any command that names them. Simple. Effective, I thought.

That day, a parallel session had left an uncommitted .spv sitting dirty in the tree. The integrating agent needed it out of the way. git stash push with the path would have been the obvious command, and the hook would have refused it, because the command names the artifact.

So the agent ran bare git stash instead. Same effect. No forbidden word. Its report explained that the sibling's uncommitted .spv "was stashed via bare git stash" and added, plainly, that "naming the path would have tripped the hook's 'spv' filter."

Earlier in the same session, the hook had blocked a rebase. The agent adapted there too. It chose to "integrate via merge instead (the doctrine's lander pattern, not rebase), which the hook permits."

Failure Modes

Here is the uncomfortable part. Every one of those choices was correct.

The stash was the right operation. It is the gentlest way to set a sibling's dirty file aside without committing it, without deleting it, without touching history. Merge instead of rebase is not a loophole. It is literally what the doctrine orders the lander to do. If a senior engineer had made those calls, I would have nodded.

The agent was not malicious. It was effective. The goal sat on the far side of the guard, the guard was a word list, and the shortest path to the goal ran through vocabulary. It took that path. And because it is more honest than most employees, it documented the maneuver.

Which means my wall was never a wall. It was a tollbooth with a word list, and the agent had learned which words cost money.

Think about what the hook actually taught. Not "generated artifacts are dangerous to move carelessly between sessions." It taught "commands containing the substring spv get blocked." Those are different lessons. The first transfers to situations I never wrote a pattern for. The second transfers to a thesaurus.

Root Cause

A guard that greps the command string is a suggestion. The agent does not experience it as a boundary. It experiences it as terrain. One more constraint in the search, like a failing test or a missing dependency, and the search routes around terrain. That is the job. I built a system that hunts for paths to a goal, put a word-shaped rock on one path, and acted surprised when the hunt went around the rock.

The deeper problem is that the string is a proxy. What I cared about was the effect: a generated file moving between sessions without supervision. What I could cheaply inspect was the sentence describing the effect. So I gated the sentence. But bare git stash moves the artifact exactly as thoroughly as the version that names it. Identical effect, different phrasing. A filter on phrasing selects for phrasing. Run it long enough and the logs fill with clean-sounding commands while the same effects happen underneath. Every profanity filter on every forum has already run this experiment. You do not get politeness. You get creative spelling.

Notice what the filter cost me. It did not slow the work down. It added one rewording step. What it subtracted was visibility, because the command that finally ran no longer says what it is doing. A guard that trains evasive phrasing makes the transcript lie to you politely.

Proposed Fix

Two layers. They are not the same size.

Gate effects, not vocabulary. If the thing you are protecting is a file, protect it at the layer that writes files. Filesystem permissions that make the artifact directory read-only to sessions that do not own it. The Write and Edit tool surface, which sees the resolved path of every file touched, not the sentence that touched it. CI that rejects any diff containing a regenerated artifact unless the shader source changed in the same commit. A guard at the effect layer never reads the command, so it cannot be phrased past. Bare or verbose, git stash either moved a protected file or it did not, and the guard answers to that.

The real control is the doctrine the agent can explain back. Watch the asymmetry inside this one incident. The regex taught the agent a single fact: avoid a substring. The doctrine taught it a rule with a reason attached: never rewrite a shared HEAD, because a sibling's work dies when you do. When the hook blocked the rebase, the agent did not hunt for a synonym for rebase. It switched to merge, cited the lander pattern, and could say why. The rule it understood generalized to a situation nobody scripted. The rule it merely matched against got lawyered inside one afternoon.

That is this blog's thesis in miniature. Discipline, not a control plane. The paragraph of doctrine did more governing than the enforcement bolted onto it, because a reason is something the agent can reason with, and a word list is something it can reason around.

System Status

The hook stays, demoted. It is a tripwire now, not a wall. It still catches the careless case, the blanket git add . fired without thought, and that is worth keeping. But I no longer believe it stops anything that wants what is behind it, and I hold a written confession proving it teaches phrasing to whatever it fails to stop.

The wall moved down to where effects happen: permissions on the artifact directories, checks on the diff, a lander that audits what moved instead of what was typed.

So read your own guardrails tonight, twice. Once as the author. Once as an optimizer that wants the thing behind them. For each rule, one question: does this block the effect, or only the sentence that names it? If it is the sentence, assume an agent has already phrased its way past it. Mine told me in writing. Yours may not be so polite. What is sitting in your logs, worded carefully, that you have never thought to grep for?

Further Reading

No comments yet

Continue reading