Supervisor Advice and Priced Cache Keepalive: Octomind 0.68.0

When an agent repeats itself, the agent supervisor steers it with a note. Until now that note was the same fixed sentence on every alarm. And while you read a reply, or a long tool runs, the prompt cache counts down, so the next request may pay to write the conversation into the cache again.

Octomind 0.68.0 lets the supervisor write a note that cites the tool result the agent missed, prices every cache keepalive ping against the rewrite it saves, and adds read-only WebSocket observers. This post covers 0.67.1 through 0.68.0 since the 0.67.0 post. The open-source update is available today.

Supervisor advice when the agent loops

The loop stop from the 0.61.0 post ends a doom loop. Before it does, the agent supervisor's loop and recovery alarms steer the agent with a note.

That note was fixed text, so all it could say was "you repeated this". The new note cites the receipt, the recorded tool result, that shows what the agent missed. It is off by default:

[supervisor.advise]
enabled = true  # default: false

When it runs:

• It fires only on a loop or recovery alarm. Never on no-progress, never periodically, and not before 4 actions are recorded in the turn.
• One shared supervisor-model call writes it. The writer reads your request, the specialist's standing instructions, the live plan, the last tool receipts and their outputs, recent commands, mutated paths, advice already given this turn, and up to 2 past experience records keyed to the same stuck pattern. It treats tool output as untrusted data.

What a note may say:

• A note is an observation, never an order. It cites a receipt (#N) and the question that receipt raises. It never contains a command, code or a next action.
• Bad notes are dropped. A note that cites a missing receipt, runs over 120 words, or contains a code fence or a $ command line is dropped. The writer may also skip, and a skip or a drop falls back to the plain note.

How the effect is measured:

• 1 in 4 eligible alarms gets the plain note. That control arm is how the effect of advice gets measured before the trigger is tuned.
• Every alarm is logged to advise.jsonl in the data directory: its arm, signature and receipt, whether the call-set changed afterwards, and the verify-gate verdict that followed.
• /info counts it with advise_calls, advised, advise_skipped and advise_control.
• A Noul scores usefulness, observe-only. With [decision] supervisor = true, one Noul per treated alarm scores the note, and the score is logged beside the advice record. It acts on nothing yet, and when the call fails nothing is recorded.

The supervisor guide lists every input the writer reads and every drop rule.

Cache keepalive, priced instead of timed

The conversation cache lives 5 minutes. Until 0.67.3, keepalive stopped after a fixed idle time, 1800 seconds by default. Now every wait gets a budget priced from your model's cache rates and from the session's own waits (0.67.3), with the cold-start, longest-wait and rate-limit rules below added in 0.68.0:

# A root field: put it before any [table] header.
cache_keepalive_enabled = true  # default: false

A ping is a cache pre-warm: it sends max_tokens: 0, reads the cache and produces no output. Pings go out every 270 s, 0.9 × the TTL of the newest cache marker on the 5m conversation cache, during replies, background work (tap runs, shell and agent jobs) and long tool runs.

How many pings a wait gets:

• Each ping has to pay for itself. A ping costs one cache read of the prompt, while an expired cache costs a rewrite of the conversation tail. Ping n goes out only while the chance that the wait ends in the next interval beats ping cost divided by the rewrite it saves.
• The budget has a hard bound. It never exceeds the ski-rental bound (cache_write - cache_read) / cache_read × tail share pings. Tools and the system prompt stay warm on their own 1h cache, so only the tail counts.
• Cold start: 2 pings. Until 3 waits of a kind are measured, a wait gets at most 2 pings.
• Never past the longest wait seen. The first wait that runs longer pays one rewrite and extends the record.
• Three kinds of wait, learned apart. Replies, background work and tool runs each keep their own history, recent waits weigh most, and waits cut short by exit or disconnect count too.
• A known scheduled wake is bridged when the pings to reach it are affordable.

When it stops, and what it costs:

• It stops when a ping can't help. No ping goes out when the next request compresses or trims the conversation anyway. A ping that misses the cache ends the keepalive, and one the provider rate-limits with HTTP 429 ends it for that wait.
• Pings show up in cost, nowhere else. Each is billed to the session cost only, not counted as an API call or as prompt tokens. /info shows the pings sent, their cost and the last budget with its basis: cold start, learned, known wake or prefix replaced.
• Through octohub 0.16.3 or later, the hub pre-warms only the lane that served the conversation, when that lane's provider has a policy. A lane that cannot be pre-warmed answers with zero read and zero cost, which ends the keepalive.

cache_keepalive_max_idle_seconds is gone from the config, the docs and validation. It defaulted to 1800, 0 meant until the session ends, and the maximum was 86400. There is no time cutoff now: the budget comes from model pricing.

The compression guide has the full keepalive rules, and prompt caching explained covers why a cache read is cheaper than a rewrite.

Watch a session without touching it: WebSocket observers

A connection that binds a session takes part in it: it can own the output and answer the session's questions. An observer only reads. Send one observe frame and the connection sees every session on the server, including turns that run while no client is bound:

{
	"type": "observe",
	"sessions": [
		{ "session_id": "my-session", "stream": "3f9a0c6e2b7d41e8a5c09d6f1b2e7a34", "after": 41 }
	]
}

sessions is optional. Each entry is a mark from an earlier observation, so the replay picks up after frame after.

The connection then receives four kinds of frame, in this order:

1. observing, always first, with the capabilities a bind ack lists, observe_v1 included:

   json

   { "type": "observing", "capabilities": ["message_attachments_v1", "…", "observe_v1"] }

2. stream_open for every open stream, then that stream's replay:

   json

   {
       "type": "stream_open",
       "session_id": "my-session",
       "stream": "3f9a0c6e2b7d41e8a5c09d6f1b2e7a34",
       "last_seq": 41,
       "gap": false
   }

3. observed for every numbered frame, replayed or live. frame is the exact frame a bound client reads, stream_seq included:

   json

   {
       "type": "observed",
       "session_id": "my-session",
       "stream": "3f9a0c6e2b7d41e8a5c09d6f1b2e7a34",
       "frame": {
           "type": "assistant",
           "content": "Done.",
           "session_id": "my-session",
           "stream_seq": 42
       }
   }

4. stream_closed when the session unloads with close or /exit.

What an observer can and cannot do:

• Observers are read-only. Any other frame gets {"type":"error","message":"observer connections are read-only"} and the connection stays open. An observer never owns a session's output and never answers elicitations.
• observe comes before any bind. After a bind it is refused with observe must come before any session bind on a connection.
• Any number of observers can connect, and each gets everything.
• A slow observer is disconnected and reconnects with its marks. ack, run_log and bind statuses answer one connection's request and are never observed.

Turns with no client bound come from the session monitor. After a session is loaded, by a bind or by a message or command for a session on disk, one background monitor runs until the session unloads with close or /exit. It watches the inbox: schedules, background agents, webhooks and delegated results.

The monitor belongs to the session, not to a connection. A timed or idle schedule, or a delegated result that lands after every client disconnected, still runs. Its frames go on the session stream, where a later resume bind reads them and an observer sees them live. Background turns stream to the connection that owns the output, and are only held when none does.

An observer can record daemon sessions from outside without binding any of them. The WebSocket server guide has every frame and replay rule.

Other changes since 0.67.0

Replay now runs on three budgets (0.67.2):

• The hold used to be bigger. Before 0.67.2 it kept 4096 frames or 8 MiB, covering the running turn and the one before.
• A resume replays only when everything is there. The stream must match, every frame after after must still be held, and those frames must fit the replay budget, half the connection's outbound limits. A single held frame larger than that is never replayed.
• An observer's lost frames stay lost. Frames left out of a replay are never sent later, so record that range as lost. A single frame larger than the whole budget is never replayed to an observer.
• Observer marks decide the replay. A mark on the same stream replays frames with stream_seq above after, with gap true when the oldest held frame is above after + 1. No mark, or a different stream, replays every held frame, with gap true when the oldest held stream_seq is above 1.
• GitHub taps refresh in the background (0.67.1). Missing default taps are still cloned before loading returns on first run. A recent or running refresh suppresses a duplicate one, judged from FETCH_HEAD plus a refresh stamp, and there is no setting.
• MCP tools with a non-object input schema are skipped (0.68.0).

Before you update

• Your config migrates to version 27 on first start. The migration adds the [supervisor.advise] section with enabled = false. It removes nothing.
• Delete cache_keepalive_max_idle_seconds if your config has it. The key is gone from the config and from validation, and the keepalive budget now comes from model pricing.
• WebSocket clients must handle "resumed": false. A resume bind whose frames no longer fit the hold or the replay budget replays nothing, and the client reloads the session from its own store.
• Keepalive through octohub: needs octohub 0.16.3 or later. Update a self-hosted hub before you set cache_keepalive_enabled = true on an octohub: model.

Update to Octomind 0.68.0

On macOS and Linux:

curl -fsSL https://octomind.run/install.sh | bash -s -- --version 0.68.0
octomind --version

On Windows, in PowerShell:

$env:OCTOMIND_VERSION = "0.68.0"
irm https://raw.githubusercontent.com/muvon/octomind/master/install.ps1 | iex

Or run scoop update octomind. See the installation guide for other methods, the full changelog, and the 0.67.0 notes if you're coming from 0.66 or earlier.

Then try it: set enabled = true under [supervisor.advise], start a task that loops, and run /info to see advise_calls and advised count up.

— DK