Product August 10, 2026 by Javier Arancibia

I Told My Agent What It Couldn't Do. It Found a Way Anyway.

#commis #ai-agents #security #dogfooding #open-source #agent-first

I built commis to hire AI agents the way you'd hire a contractor: one config row, a budget in euros, a list of commands they're allowed to touch, a human who approves anything destructive. "Grant grepapi, not curl. Grant rm, but only with a human's yes on every single call." The kind of promise a config file makes easily.

The mechanism behind that promise is simple: build a directory of symlinks containing only the granted commands, plus a small base set the shell needs to function, and run the agent with that directory as its entire PATH. No curl symlink, no network. No rm symlink, no deletions. The OS enforces it, not a sentence in a prompt.

It held up in testing. It held up in code review. It did not hold up against a live model for more than a few minutes.

The agent went looking for curl. It didn't need to ask.

I gave a row a small wrapper script — vigie-stats, which shells out to curl internally to fetch analytics numbers. I granted the wrapper. I did not grant curl. That should have been fine: the wrapper is the only thing on PATH, so when it tries to run curl, the shell should say command not found and stop there.

Instead, the agent — unprompted, not told to try this, just exploring after the first call failed — ran this:

PATH=/usr/bin:$PATH vigie-stats overview 7d intrane.fr

And it worked. The wrapper found curl. The data came back. The agent wrote its report and moved on, mildly curious about why the first attempt failed and not especially interested in the fact that it had just walked around a security boundary.

Here's why it works: a shell assignment like VAR=val cmd sets that variable only for the one process it launches. The agent isn't editing my shim — it's handing the next process a new environment for the duration of that single call. My shim only controlled what vigie-stats itself could see; it said nothing about what vigie-stats could tell its own children. Reasonable in isolation. Wrong in composition.

I reproduced it by hand outside any agent, just to be sure it wasn't a fluke:

$ env -i PATH="/shim-only" bash -c "curl --version"
bash: curl: command not found

$ env -i PATH="/shim-only" bash -c "PATH=/usr/bin:$PATH curl -s https://example.com"
<html>...</html>

Same trick, no wrapper needed, no agent needed. Any shell, on any row that has a shell at all — which is every row, because --allow-shell is what lets an agent do anything beyond read a static file. I hired a second row a few days later, gave it a completely different tool, and the model found the exact same trick on its own, a second time. Not a fluke. A property of the design.

What I didn't do

I didn't quietly patch the plan output to stop mentioning it. I didn't add a comment in the source and move on. commis's plan command exists specifically so a human can see the real capability of a row before approving it — that's the entire point of a config-row system instead of a black box. Shipping a plan that still said "no curl means no network" after I'd just watched that sentence be false would have been worse than the bug itself.

So the fix wasn't code. It was honesty:

"caveat": "PATH bounds which BINARIES exist, not what the run can do —
and it does not even bound that reliably. [...] NO CLAIM ABOUT WHICH
BINARIES ARE REACHABLE HOLDS under --allow-shell. The only real
boundary is the model's willingness to stay in its lane and the gate
on destructive commands roam recognizes; do not grant a row real
secrets or network-capable tools unless you trust the MODEL, not the
PATH list."

Every row's plan now says this, in the exact field a reviewer would look at before approving a run. If you're evaluating commis and that sentence changes your mind about hiring an agent for something sensitive, it should — that's what it's there for. A real sandbox (mount namespaces, not a curated PATH) is the actual fix, and it's still open. I'd rather ship the honest gap than a false sense of one closed.

Then I tried to prove the other half worked

PATH grants were never the only line of defense — the approval gate was supposed to be the real one. A row can mark a command confirm: always, and every single invocation stops and waits for a human, on a channel that ranges from email to a push notification with two tappable buttons. I'd built it, I'd tested it against a stub. I had never run it for real, against a real destructive command, on a real machine.

So I hired a row whose only job was three rm commands against my own server's apt cache — files apt regenerates on its next run, genuinely zero-risk, and narrow enough that I could review every single request by eye. I ran it. The agent inspected the targets read-only first, then asked:

AWAITING APPROVAL for destructive command: rm -rf /var/lib/apt/lists/*

Exact match to what I'd asked for. I approved it. It ran, and then the OS said no — Permission denied. I'd also just shipped a feature that drops the agent's worker process to an unprivileged system user, and that user doesn't own /var/lib/apt. The approval gate had done its job perfectly; a completely different boundary, one I'd built days earlier for an unrelated reason, had collided with it.

I re-ran the same row, same review, without the privilege drop for that one run. Same three approvals, this time as root:

$ apt-get update
Fetched 39.1 MB in 4s
Reading package lists... Done

Sixty-two files back, exactly as before. The deletions worked, the approval gate worked, and — because I checked instead of assuming — I now know two things I didn't before: the destructive-approval path is real, and my two safety features don't compose on the same box without a third decision I hadn't made yet (which rows need root, and which don't).

The lesson wasn't "write more tests"

I have 420 assertions in commis's test suite. None of them found either of these. A stub can only be as adversarial as I imagine to write it, and I didn't imagine PATH=/usr/bin:$PATH until a model found it for me — twice, on two different tasks, without being told the trick existed. The stub also can't tell me that an unprivileged user lacks write access to /var/lib/apt; that's a fact about the actual filesystem on the actual box, and it only shows up if you actually run the thing there.

Everything in this post shipped the same day it was found: the honest caveat, the unprivileged-worker feature, the fix for a delivery bug that same feature caused, the live proof. None of it was a plan I followed — it was watching a real run do something unexpected and deciding, each time, whether to patch the code or patch the claim. Usually both.

commis is open source on GitHub, one static binary, no runtime dependencies. If you hire an agent with it, read the plan before you approve it — that field exists because I needed it to be true, not just present.