Every agent session leases its model from one broker

I run many opencode sessions at once, against three cloud subscriptions and two local GPUs. Out of the box, each session picks one model and keeps it. The architecture decision and the variable rename draw on one budget, a session that hits a plan limit stops, and no session knows what the others spend. opencode-broker is a small daemon that fixes this. Every session, subagent and OpenAI-style client I point at it asks for a model lease, turn by turn, and it answers from one shared ledger. The router at its core reached v0.51 in three weeks and 137 commits. It and its two companions, opencode-gateway and opencode-hud, become one project for release.

What it does #

An agent names a tier, not a model. The tiers are ranked by cost: worker, review, fast-build, build, smart and deep, plus a separate classifier lane. The test runner sits on worker, reviewers on review and implementers on build. The researcher sits on smart, because its output is the facts that design gets built on. general inherits its parent’s tier, so delegated design work never lands on a worker model by accident. A deterministic content match on medical, legal or electrical questions raises the floor to smart or deep, whatever the agent.

A session also has a profile. auto routes by tier and manual locks to your own pick. The rest, such as local, private and vision, are lanes of specific local models that stay on the LAN unless the config says otherwise. F11 in the HUD, a plugin for opencode’s terminal UI, picks one per session. The HUD also shows a FALLBACK:<target> badge when the broker has degraded you, and how much of each provider’s window is used.

Services that only speak OPENAI_BASE_URL + key + model, such as Open WebUI and Home Assistant’s voice assistant, point at the gateway. Each request gets a fresh lease, the same circuits and the same accounting.

There is no built-in model. With no config, every lease fails with a clear error, because a router that ships a provider’s name in its code will quietly spend a metered account.

Each piece exists somewhere else: LiteLLM, claude-code-router, OmniRoute and oh-my-opencode each cover part of this. What I didn’t find was a single broker that every concurrent session leases from, per turn, against one ledger.

How it works #

The broker is a Node daemon that speaks JSON over a Unix socket and never proxies an opencode session. The router plugin hooks chat.message, the last hook that can still change the model before the provider call. It leases with the session’s profile, tier and context size, binds the turn to the leased model, reports usage and failures, and releases the lease when the session goes idle. The broker handles one request at a time, so two sessions can’t both take the last local slot, and it logs every decision with its reasons.

Selection drops anything with an open circuit, a quarantined provider, too small a context window, no free slot or an unloaded model. It then spends where the most budget headroom is left, breaking ties by model fit, provider weight, discounts, load and round-robin.

Privacy lanes fail loudly instead of degrading. Offline profiles run shell commands in a sandbox with no network. A profile’s fallback rungs are filtered to local targets unless the profile is named in profileCloudEgress, a separate top-level list, so pasting a tier’s cloud-heavy rungs into a profile can’t grant egress by accident.

Hard problems #

An unreadable quota looks like an empty one #

Where a provider reports its own usage, the broker reads those figures. Otherwise it falls back to its own ledger, which only sees this machine’s requests and so can only undercount. On 2026-09-04 Alibaba’s usage CLI had lost its token. That subscription read 2.75% used against Anthropic’s 32% and OpenAI’s 18%, and every build lease went to it. The lane I could account for least was the one being drained fastest.

The fix floats an unobserved provider up to the mean of the observed ones:

const observed = new Map();
const unobserved = [];
for (const target of targets) {
  if (target.kind !== "cloud") continue;           // local has no quota
  if (isObserved(target.providerID)) {
    if (!observed.has(target.providerID))
      observed.set(target.providerID, utilizations.get(target.id) ?? 0);
  } else unobserved.push(target);
}
if (!observed.size || !unobserved.length) return null;
const balance = [...observed.values()].reduce((a, b) => a + b, 0) / observed.size;
for (const t of unobserved)
  utilizations.set(t.id, Math.max(utilizations.get(t.id) ?? 0, balance));

The mean is per provider, not per target, because Anthropic fielding two models had dragged it to 0.2733 when the real mean of 0.32 and 0.18 is 0.25. Readings also survive a restart now: with an empty cache nothing was observed, the correction stood down, and Alibaba took six build leases in a row.

Budgets calibrate themselves too. Providers don’t publish plan limits, so a rate-limit complaint sets the shortest window’s ceiling to what the ledger had spent at that moment, and spending past a learned ceiling without complaint raises it.

Whose fault is a failure? #

A quota error covers the whole subscription, so it opens a provider-wide circuit that holds until the exact reset. The reset comes from the error, else the provider’s usage API, else a 15-minute re-probe. A lease also waits for a due usage refresh, so the first request after a five-hour window fills can’t still get Opus.

A failure the broker doesn’t recognize fences one model for five minutes and counts against its provider. Two different failing models within 15 minutes quarantine the provider, and 30 minutes later probation admits exactly one lease. That sounds cautious, but deterministic failures replay across sibling models and manufacture exactly that evidence:

• An expired OAuth token quarantined a healthy OpenAI lane for a day.
• A transcript whose tool calls had lost their results failed over to Anthropic. Opus 5 rejected it, Opus 4.8 rejected the same bytes seven seconds later, and the provider serving every build and smart lane was out with no sick models at all.
• The router’s own “routed model mismatch” guard took three providers dark in one day.

So failure classification grew to ten kinds, each with its own scope:

quota     provider, until the reported reset
connect   provider, 60 s
rate      one model, 5 min
overload  one model, 15 s
auth      one model, 1 min
model     one model, no expiry
context, payload, noop   release the lease only
other     one model, 5 min, plus evidence against the provider

Only the last line counts against a provider, and every new failure shape reached it first.

Local GPUs are slots, and models move #

A local target declares how many requests it serves at once and a per-slot context window. llama.cpp’s router mode lists every configured model whether it’s loaded or not, so a model is eligible only when its status reads loaded.

Several models share two cards, and some can’t be loaded together. When a lease fails because a model isn’t resident, the broker runs that target’s prepareCommand, as an argument list with no shell, and doesn’t wait for it. A 27B swap took 2 minutes 40 seconds, and the lease call sits inside a turn. The refusal carries a machine-readable target-preparing code. Before waits were split from refusals, 650 of 1,500 logged decisions read “refused”, and 617 of those were a model loading: a 43% failure rate on a router that was working.

Cost decides which swaps happen. A 70B model spans both cards and evicts three others, including the one Home Assistant’s voice assistant uses. It has its own profile rather than being a second target in another lane, because the broker picks among a lane’s local targets by round-robin, and a cost like that should be chosen on purpose, never won on a tie-break.

Status #

The router is at v0.51.0, the gateway at 0.5.0 and the HUD at 0.14.0: 202 commits between 31 August and 21 September 2026, with no tagged releases yet. It’s about 10,000 lines of JavaScript plus about as many lines of tests, and the broker and gateway have no npm dependencies. It runs on my dev machine, in front of three subscriptions and llama.cpp on two V100 GPUs.

It’s built against opencode 1.18.22 and relies on observed wire shapes, not documented API. Limitations:

• No plugin hook fires on a provider error, so recovery happens on the next turn, through a synthetic “continue” prompt capped at two per ten minutes.
• Stock opencode has no clean way to rebind a turn’s model, and the workaround does nothing on a subagent’s first turn. A small engine patch fixes it, and I’d rather that patch went away than grew.
• The gateway can’t fail over after the first streamed byte.
• It serves one host over a Unix socket.
• The risk floor is a keyword match, so coding vocabulary like “circuit breaker” can trip it. Narrowing it is in progress.

What’s next #

First, merging the three repositories. After that, I may fold in the child-session cleanup I’ve written about before as a session janitor: abort, prove the loop can’t write, delete, then read the row back.

If I rebuilt the failure handling, I’d invert its default: an unrecognized failure would fence only the one model, and a provider would be indicted only on positive evidence. I’d keep deterministic escalation. A content match escalates every time, while a model asked whether to escalate answers differently on different days.

Links #

• Repository: github.com/sudolulo/opencode-broker. The socket protocol and the on-disk state contract are documented in its docs/ directory.
• Companion project: opencode-guard, whose command classifier leases through the broker’s classifier lane.