Running coding agents unattended: an instruction is not a security boundary

Coding agents are good at recurring judgement work: reading a repo for bugs, checking that the docs still match the machines, going back over last week’s sessions for mistakes. Running that work on a timer is harder than doing it at a keyboard, because nobody is there to approve a command, notice a loop or watch the meter. claude-fleet is the control plane I built for it. It syncs one agent config to every machine, shows every session on every host in one place, and runs scheduled headless jobs inside declared permission envelopes, with results pushed to my phone. The meter problem was real: before I measured anything, the fleet’s agent jobs were on course for about 47.6 billion tokens a year, roughly 87% of it from one hourly job.

What it does #

One config everywhere. Agent settings, instructions and hooks live in git and are symlinked into place on each host. An hourly sync pulls the repos, relinks the config and converges the timers. The pull is the deployment.
Session visibility. fleet-sessions lists every agent session on every host in one table, or writes an HTML dashboard.
Scheduled jobs. Deterministic collectors (host health, config drift) don’t need a model. Agent jobs handle the work that needs judgement. repo-review reads all my repos and reports bugs and security problems without ever fixing them. homelab-audit compares the docs with the live machines and opens a pull request. ai-incidents reads my own agent transcripts for incidents, and has its own write-up.
Push notifications and a dead-man’s switch. Jobs report to my phone through ntfy, with a priority set by the outcome. Every job also writes a heartbeat into Prometheus, so a job that silently stops running raises an alert.

How it works #

Flowchart. A git forge holding the three fleet repos feeds an hourly fleet-sync on every host, which links the shared agent config and installs systemd timers that declare their host. A timer runs a deterministic gate. If there is no work, the gate writes a heartbeat. If there is, it picks a model and starts fleet-agent, which reads the job's envelope and runs a headless agent with everything else denied. The agent can use read-only wrappers and one publish script, which writes to the job's own repo. fleet-agent writes the heartbeat, which feeds a Prometheus dead-man's switch, and pushes a notification to the phone. — The model only runs in the middle of this chain. Code decides whether it runs at all, and code does its only write.

The fleet is three repos. fleet-core is the harness: config, sync, session tools, notifier and heartbeat. fleet-ops holds the deterministic jobs. claude-fleet holds only the agent jobs.

Every job is a systemd user timer with an extra [X-Fleet] Host= section. It names the host that runs the job, or an alias such as @primary. Each host installs only its own timers. The sync refuses to run if any timer leaves the line out, because an earlier job had quietly ended up scheduled nowhere.

An agent job is a Markdown file, with the prompt in the body and the permission envelope in the front matter. fleet-agent runs it through claude -p headless. A headless agent can’t ask for permission, so any tool call outside the shared allowlist and the job’s own allow: list is denied automatically. The agent notes the denial and carries on. Widening an envelope is a reviewed git change, not a runtime flag, and the runner rejects some envelopes outright:

case "$r" in
  Bash|"Bash(*)"|"Bash(*:*)")
    echo "fleet-agent: blanket '$r' is refused — scope it, e.g. Bash(df:*)" >&2
    exit 5;;
  "mcp__"*"__*")
    echo "fleet-agent: whole-server '$r' is refused — name individual tools" >&2
    exit 5;;
esac

It also rejects bypassPermissions and loads only the MCP servers a job declares.

The envelope has to be the only door #

The first version ran agents with permission checks bypassed and an unrestricted shell. I replaced it the same day with plain scripts, and agents came back the next day inside envelopes. The envelopes then leaked in three ways that no prompt could have fixed.

Grants that were really blank cheques. The review job’s prompt said never to modify a reviewed repo, but it held Bash(git:*): git push --force to every repo the host’s SSH key reaches. The public-records job, the one agent that reads untrusted input from the web, held a network client, an unrestricted interpreter and git, which is every link of an exfiltration chain. Now the review job gets ro-git, which allows read verbs only. Its one write is a publish script that pushes to its own report repo and checks the remote first. ro-git has to parse past git’s global flags, because Bash(git -C:*) also matches git -C /repo push --force. Even so, its first version allowed git -c protocol.ext.allow=always clone 'ext::sh -c whoami', which runs a shell command through a “read” verb. It now refuses -c and ext:: outright.

A config layer nobody managed. The jobs run from the home directory, so an unmanaged local settings file there loaded as project settings on every run. It granted Bash(git *). The review job’s log showed raw git running ten times, and the wrappers built to be its only git ran zero times. The whole write-isolation design had been inert. The fix was one flag, --setting-sources user,project, confirmed with a live probe.

A permission for a tool that didn’t exist. For twelve nights the docs audit pushed a branch and was then denied the pull request, because its envelope named the PR tool wrongly. The right name would have failed too, since the MCP server deliberately holds a read-only token. Now the publish script opens the PR with a write token the model never sees.

The lesson from all three: an instruction is not a security boundary. A capability the agent doesn’t hold is.

Don’t pay a model to find out there’s nothing to do #

An agent is billed for context times turns, and context accumulates, so anything read on turn 3 is paid for again on every later turn. The hourly podcast-processing job measured about 5 million tokens a run, some runs took up to 234 turns, and 11 of 27 runs in a week did nothing at all. Because the account is a subscription, that cost came out of quota headroom rather than an invoice. The session that measured it priced it at API rates anyway and reported a $48,000-a-year bill that didn’t exist, and that mistake went into the incident ledger too.

The fix is a gate: the timer runs a bash script, not the agent, and the script answers the deterministic question first. In the docs audit, if the checker finds the docs clean, no model is loaded. Otherwise mechanical drift goes to Haiku, a false sentence to Sonnet, and anything that would mislead me during an incident to Opus. The incident sweep’s gate saves about 2.1 million tokens per quiet run, the measured cost of loading an agent to run one command and find nothing. The repo-review gate enforces a monthly budget capped at about $17 at API-equivalent rates, in bash, where a prompt can’t argue with it.

Loading fewer tools also helped, because tool definitions are re-read every turn. One Sonnet turn with all 30 built-in tool definitions cost 29,348 tokens. With only Bash and Write it cost 15,778.

A dead-man’s switch that believed the job #

On 13 July the incident sweep hit a spend limit. It threw away its candidates, sent the error text to my phone as its result, and recorded exit code 0. In 39 samples over 19 hours, the exit code never left 0, and both alerts stayed green through a job that was dead for a day. claude -p exits 0 while printing the limit message, and the heartbeat was written before anything read the output.

The runner now judges the output, not the exit status. A usage limit gets its own exit code. The “last run failed” alert ignores it, but it doesn’t count as a success either, so a limit that never lifts still ages into “overdue”. The runner also reads the reset time from the provider’s message and schedules a one-shot systemd retry for five minutes after it. Before this, two weekly jobs had each stayed red for a week over limits that lifted within hours. Gates follow the same rule from the other side: a skipped run writes its own success heartbeat, or the switch fires because the job is healthy.

Status #

claude-fleet is private. The three repos have about 650 commits between them since 8 July 2026, and fleet-core is at 0.7.0. It’s Bash and Python on systemd user timers, with Prometheus and Grafana for alerting and a self-hosted ntfy.

The honest limits:

It’s less of a fleet than it was. It started with agents on three machines, and for a while the scheduler had a node of its own. Now the agents run on the dev VM. Folding the scheduler into it was a trade: rebuild the dev VM and you rebuild the scheduler with it.
Most agent jobs are pinned to the dev VM because that’s where the checkouts are. On a host with fewer clones, repo review would check a fraction of my repos and still report green.
One job keeps arbitrary code execution by design, scoped to its own project, because there the model’s output is the product. The risk is written down, not papered over.

What I’d do differently #

I’d start with the gate and the wrapper, not the prompt. Nearly every fix here moved a decision out of the model and into code: whether there is work, which model to use, and what can be written where. I’d also test every envelope on day one with a call that should be denied.

Links #

The repo is private. The parts that generalize are published separately:

opencode-broker: model routing for OpenCode, with quota circuits and budget balancing. It combines what used to be opencode-router, opencode-gateway and opencode-hud. The router and the HUD started in this fleet’s harness.
opencode-guard: permission modes, credential guards and secret redaction for OpenCode. It was formerly opencode-guardrails.
ai-incidents: the transcript sweep, as a standalone tool.