Skip to content

hyp status names recent clients from gateway-tracked entrypoints (#421 item 5) - #502

Merged
philcunliffe merged 3 commits into
masterfrom
fix/issue-421
Jul 31, 2026
Merged

hyp status names recent clients from gateway-tracked entrypoints (#421 item 5)#502
philcunliffe merged 3 commits into
masterfrom
fix/issue-421

Conversation

@philcunliffe

@philcunliffe philcunliffe commented Jul 31, 2026

Copy link
Copy Markdown
Contributor

What the thread decided

Items 1-4 of #421 shipped in #425 (merged 2026-07-28, Refs #421). The only
open item was item 5, and the issue thread carries a maintainer decision
that this PR follows: option 4 of the four candidates LLP 0141 left open -
"the gateway tracks last-seen entrypoints into status.json as traffic flows,
and hyp status reports them ... without any cache read"
, plus "fix the
claude-desktop verify.js line as noted"
. That comment is the spec here.

So this is not the documentation-and-acceptance shape the rest of #421
had. Item 5 is behavioural, and it has a real failing-then-passing test.

What I established about today's behaviour, and how

Executed, not reasoned:

  • hyp status genuinely cannot see a row. decideBootProfile
    (src/core/cli/dispatch.js) returns { activate: [] } for status, so no
    dataset registry and no cache read. Confirmed by running bin/hypaware.js status against a temp HYP_HOME on origin/master: no client-surface
    information in either the text or --json output.
  • status.json source details were frozen at boot. startConfiguredSources
    writes them once; runTick refreshed only status.sinks. Proved with a
    fixture plugin whose status() details change per call - on origin/master
    the on-disk snapshot stayed at the boot value for the daemon's whole life
    (probes=1 after 20s of ticking).
  • The row-level attribution already exists. gateway_codex_capture already
    drives a Desktop-shaped exchange (originator: Codex Desktop) and already
    asserts entrypoint === 'Codex Desktop' on the projected rows. Nothing about
    capture needed changing - only the path from a captured row to hyp status.
  • The ~/Library/Application Support/Codex boundary is already handled
    by Make Codex Desktop capture explicit and acceptance-tested #425: codex/src/backfill.js flags it as codex_desktop_app and carries
    covered_by. Left untouched.

What changed

  • ai-gateway/src/entrypoint_activity.js (new). Folds each committed
    batch of projected rows' entrypoint / client_name into an in-memory map:
    last-seen + row count per distinct entrypoint. Bounded at 32 with
    least-recently-seen eviction (entrypoint is a client-supplied header
    string). Rows with no entrypoint are skipped rather than bucketed under a
    placeholder no query could reproduce. It interprets nothing, which is what
    keeps client-specific data out of core (LLP 0130, LLP 0003).
  • ai-gateway/src/source.js. Tracker lives on liveState so a config
    reload does not erase it; exposed as recent_entrypoints in status()
    details.
  • src/core/daemon/runtime.js. New refreshSourceDetails(), called on
    every tick and once at shutdown before sources are torn down. Name, plugin,
    and state are untouched - liveness stays the lifecycle's business.
  • src/core/daemon/status.js. recentEntrypointsFromSources(): find the
    gateway snapshot, validate shape, order by time, drop malformed entries.
    Deliberately not liveness-gated, unlike
    resolveLiveGatewayEndpointFromStatus: a bound port is a claim about now,
    "last seen at T" stays true after the daemon exits.
  • src/core/commands/status.js. recent clients: text block (only when
    non-empty, so a never-captured install keeps the V1 surface) and a
    recent_entrypoints JSON array. Coarse ages (just now / 5m / 3h /
    2d), never negative.
  • claude-desktop/src/verify.js. Step 3's "confirm capture via
    hyp status" was aspirational; it is now achievable, and the wording points
    at the line that answers it. Claude Desktop's local-agent entrypoint shows
    up with no Claude-specific code.
  • Docs. New LLP 0164; LLP 0141's open consequence resolved in place; LLP
    0086's "captured at boot" note corrected; docs/ACCEPTANCE.md's
    codex_desktop_capture step 4 gains the query-free check and two
    expected-negative cases (restarted daemon, null originator).

Actual output, run against a temp HYP_HOME:

  recent clients:
    - Codex Desktop  (codex)  last seen 7m ago, 6 rows
    - codex-tui  (codex)  last seen 5h ago, 40 rows

The ground-truth gate

Fails before, passes after. Verbatim, on a origin/master@73b4618 worktree:

not ok 1 - the daemon refreshes source details on every tick, so accruing details reach status.json
  error: 'source details never refreshed (probes=1)'
not ok 2 - a daemon that never reached a tick still refreshes source details before it stops
  + actual - expected
  +   probes: 1, seen: []
  -   probes: 2, seen: [ 'late-arrival' ]
# tests 4 / pass 0 / fail 4

(The other two files fail at module load on master - formatEntrypointAge and
entrypoint_activity.js do not exist there. The daemon test is the
substantive behavioural reproduction.)

On this branch the same three files are # tests 16 / pass 16 / fail 0.

The hermetic smoke is the end-to-end half. gateway_codex_capture now asserts
that the Desktop originator the projector wrote into a committed row is
readable from status.json after the daemon exits. Copied onto origin/master
it fails with recentEntrypointsFromSources is not a function; here it passes.
This proves the plumbing, not a real app - the fixture's originator is
synthetic, exactly as that file's header block already says.

Executed vs reasoned

Executed: everything above, plus npm test, npm run typecheck, and the
hermetic smokes gateway_codex_capture, daemon_foreground_start_stop,
status_diagnostics, gateway_claude_capture, ai_gateway_passthrough,
activation_lifecycle, sink_export_driver (all ok).

Reasoned, not executed: the docs/ACCEPTANCE.md steps that need a Mac with
Codex Desktop installed and signed in. I have no macOS host and no Codex
Desktop, so I did not run steps 1-7 of codex_desktop_capture. What I did
validate is that the new step 4 addition uses only commands and output shapes I
ran here (hyp status, hyp status --json | grep -A 6 recent_entrypoints, and
the exact recent clients: line above). The literal Desktop originator
string remains unpinned by design (LLP 0141) - the procedure still tells the
human to record what their machine actually reported.

Checks

  • npm test: 3053 tests, 3044 pass, 8 fail. The 8 are the pre-existing
    test/core/leave-command.test.js failures, identical by name to my own
    origin/master baseline (3037/3028/8).
  • npm run typecheck: clean.
  • @refs: every annotation added resolves; LLP 0164's three anchors
    (#gateway-tracks-what-core-cannot-name, #status-reads-it-from-the-status-file,
    #not-liveness-gated) all exist.
  • Pre-existing smoke failure, on master too: walkthrough_to_first_query
    (ai_gateway_messages has exactly one row, value=0). Not touched by this PR.

In-flight collisions

None. No file here appears in #495, #493, #491, #462, #456, #347, #361, #360,
#279, or #221. In particular I did not touch
ai-gateway/src/types.d.ts (in flight on #456) - the tracker's types are
inline JSDoc in the plugin, and the shared RecentEntrypoint interface went
into src/core/daemon/types.d.ts, which is not in flight.

What this does not do

  • The tracker is in-memory and daemon-scoped, so a restart clears the list.
    The cache stays the durable record and the query stays the pass condition;
    documented in LLP 0164's consequences and in the acceptance failure notes.
  • Backfill writes through the materializer, not the live gateway, so importing
    a Desktop rollout does not put Desktop in recent clients. Honest for a line
    that means "traffic arrived here", but the two LLP 0141 routes are not
    symmetric on this surface.
  • hyp query overview is untouched (option 3 would have moved that block's
    calibrated window budget, LLP 0135#window).

Fixes #421

test and others added 2 commits July 31, 2026 01:26
…item 5)

Issue #421 items 1-4 shipped in #425. Item 5 - "make `hyp status` or the
verification surface identify recent Codex Desktop traffic when `entrypoint`
or `originator` proves it" - was left open because closing it needed an
architecture choice, recorded as four candidates in LLP 0141's consequences.
The maintainer picked **option 4** on the issue thread: the gateway tracks
last-seen entrypoints into `status.json` as traffic flows, and `hyp status`
reports them with no cache read.

- The AI-gateway source folds each committed batch of projected rows'
  `entrypoint` / `client_name` into an in-memory, bounded, LRU-evicting map
  (new `entrypoint_activity.js`), and exposes it as `recent_entrypoints` in
  its `status()` details. It counts and timestamps; it interprets nothing, so
  no client-specific string enters core (LLP 0130, LLP 0003).
- The daemon re-reads started sources' `status()` details on every sink tick
  and once at shutdown. Boot captured them exactly once, which was enough for
  details fixed at bind time (host, port, LLP 0114 fallback marker) and is not
  enough for a detail that accrues.
- `hyp status` renders a `recent clients:` block and `--json` a
  `recent_entrypoints` array, read from `status.json`. It is deliberately not
  liveness-gated: unlike a bound port, "last seen at T" stays true after the
  daemon exits, and the rendered age carries the staleness.
- `claude-desktop verify` step 3 told users to "confirm capture via
  `hyp status`", which was not achievable. It now is, and the wording points
  at the line that answers it.

Docs: new LLP 0164; LLP 0141's open consequence resolved; LLP 0086's
status.json note corrected; `docs/ACCEPTANCE.md`'s `codex_desktop_capture`
step 4 gains the query-free check plus its two expected-negative cases.

Co-Authored-By: Claude <noreply@anthropic.com>
…erminal

`entrypoint` reaches the tracker by three routes, not the two the design
assumed. Codex's `originator` header and the User-Agent product are both
bounded by Node's HTTP parser (no C0 control bytes, ~16KB). The third is
not: for Claude the *live* projector copies `entrypoint` off a transcript
`.jsonl` line on disk (`applyTranscriptMatch` -> `assignTranscriptIdentity`,
transcripts.js:517), and that is an ordinary JSON string of any length
holding any byte, which then wins over the exchange-level value at
message_projector.js:715.

That value was stored verbatim, written to `status.json` on every tick, and
printed verbatim by `hyp status`. Executed against the real binary: a 30KB
transcript entrypoint carrying ESC and newline produced a 30375-byte
status.json and terminal output containing a raw escape sequence and a
forged `daemon:   FORGED ALL GOOD` line. The 32-entry cap bounds the number
of entries, not their size, so it did not prevent this.

Adds `sanitizeLabel` to core util (strip C0/C1 and the Unicode line
separators, clamp to 120 chars) and applies it in two places:

- at the gateway tracker's `record`, so the map key itself is clean and the
  eviction cap cannot be diluted by values differing only in invisible bytes
- at core's `recentEntrypointsFromSources`, because `status.json` is a file
  and core must not assume the daemon that wrote it was this build; this is
  the last point before render

Same input after the fix: status.json 444 bytes, output 1139 bytes, no
escape byte, no forged line, surface still named.

Also corrects LLP 0164's "Bounded" rationale, which named only the header
route while its own motivating example (`local-agent`) is the transcript
one, and pins the default cap in a test (raising it to 100000 previously
went uncaught, since every eviction test passes an explicit `max`).

Co-Authored-By: Claude <noreply@anthropic.com>
@philcunliffe

Copy link
Copy Markdown
Contributor Author

Verdict

Findings - one substantive (fixed on the branch), three minor (two fixed, one
left for you as a design call). The PR implements the maintainer's option-4
decision correctly
, including the load-bearing no-cache-read constraint, and
the reproduction/test story is honest and holds up. I did not revisit option 4.

Reviewed at head 7d1b111. Fix pushed as be9ce7f.

Note on tooling: the code-review skill is not installed in this clone and
codex is not on PATH, so this was a direct audit. There is also no
llp-ref-hygiene npm script (ref hygiene here is the prompt-based /ref-check
skill), so I ran a mechanical equivalent over the changed files instead.


Does it implement the decision?

1. No cache read - holds. Established by execution. [executed]

I did not take this from the code. Three independent executions against a temp
HYP_HOME:

  • Syscall-level. Ran bin/hypaware.js status under a preload that wraps
    fs/fs.promises open*/readFile*/createReadStream/readdir/stat and
    records every path. With recent clients: rendering correctly, the only
    access under <state>/cache/ was readdir on three directories and one
    stat - the pre-existing V1 cache-size/dataset-count walk. Zero content
    reads. The answer came from a single fs.readFileSync of
    <state>/run/status.json.
  • Poisoned cache. chmod 000 on the dataset parquet: hyp status still
    printed the correct recent clients: block and exited 0. A query would have
    failed.
  • Whole cache unreadable. chmod 000 on <state>/cache/datasets itself:
    still correct, still exit 0.

Re-ran the syscall + unreadable-cache checks against the pushed remote SHA
be9ce7f from a fresh checkout - same result.

This is what respects decideBootProfile, and it genuinely does.

2. Client-agnostic - holds. [executed + reasoned]

Nothing hardcodes Codex. The tracker keys on whatever string is in the
entrypoint column and never interprets it; recentEntrypointsFromSources
validates shape and orders by time only. I confirmed by execution that a
local-agent entrypoint renders through the identical path with no
Claude-specific code.

Worth recording because it is load-bearing and non-obvious: Claude Desktop is
covered because the live Claude projector enriches the projected message
from the transcript (applyTranscriptMatch -> assignTranscriptIdentity,
transcripts.js:517) before appendRows, so the tracker sees it. I initially
suspected this only happened in flush-time settlement (settle.js), which would
have meant Claude Desktop was silently not covered and the verify.js change
was still false. It is not the case - the enrichment is on the live path.
Settlement does it too, but the live path is what matters here.

3. The verify.js line is now true - yes, genuinely. [executed]

Not merely reworded. hyp status really does print a recent clients: block
naming the entrypoint, so "run hyp status and look for entrypoint
'local-agent' under 'recent clients'" is a check a human can now actually run.


Findings

F1 [Medium] Unbounded, unsanitized client-supplied labels reached status.json and the terminal - FIXED

entrypoint_activity.js:49-53 and status.js:156-164 (at 7d1b111) stored and
returned entrypoint / client_name verbatim; commands/status.js:390 prints
them verbatim.

The design assumed two provenances, both parser-bounded. There is a third that
is not. I verified the bounds by execution against Node's HTTP server parser:

Route Bounded?
Codex originator header Yes - HPE_INVALID_HEADER_TOKEN on ESC/BEL, HPE_HEADER_OVERFLOW past ~16KB
Codex User-Agent product regex Yes
Claude transcript .jsonl on disk (transcripts.js:688 -> :517 -> message_projector.js:715) No. Ordinary JSON string, any length, any byte - and it wins over the exchange-level value

Executed end-to-end through the real binary, using the real
assignTranscriptIdentity and the real tracker: a 30KB transcript entrypoint
carrying ESC and newline produced a 30375-byte status.json and 31063
bytes of terminal output containing a raw escape sequence and a forged
daemon: FORGED ALL GOOD line
.

The 32-entry cap bounds the number of entries, not their size - and
status.json is rewritten on every tick, so this is sustained write
amplification as well as a display-integrity problem on an operator trust
surface.

Reachability, stated honestly: transcripts are vendor-written, so a hostile
value needs a malicious/compromised client or local write access to the projects
dir. The length half needs no hostility at all.

Fix (be9ce7f): added sanitizeLabel to core util (strip C0/C1 + U+2028/9,
clamp to 120 chars) and applied it in two places:

  • at the tracker's record, so the map key is clean and the eviction cap
    cannot be diluted by values differing only in invisible bytes;
  • at core's recentEntrypointsFromSources, because status.json is a file and
    core must not assume the daemon that wrote it was this build. This is the last
    point before render.

Same input after the fix: status.json 444 bytes, output 1139 bytes, no escape
byte, no forged line
, surface still named. Verified from a fresh checkout of
the pushed remote SHA, not my working tree.

F2 [Low] LLP 0164's "Bounded" rationale named the wrong threat model - FIXED

It justified the cap with "entrypoint is a client-supplied string riding in
from request headers" - the one route that is parser-bounded - while the
document's own motivating example (local-agent) is the transcript route that
is not. Same claim in the code comment at entrypoint_activity.js:7. Corrected
both, and noted the one deliberate exception to the "byte-identical to what a
query filters on" claim.

F3 [Low] Test gap: the default cap was not pinned - FIXED

Mutation-testing caught this: raising MAX_TRACKED_ENTRYPOINTS from 32 to
100000 left all 16 tests green, because every eviction test passes an explicit
max. Added a test that pins the shipped default, plus tests for the F1
behaviour.

F4 [Low] A throwing source status() is now swallowed silently forever - NOT fixed, your call

refreshSourceDetails calls safeStatus per tick, whose catch {}
(runtime.js:1046) discards the error with no log. Before this PR a throwing
status() was swallowed once at boot; now it is swallowed every 60s for the
daemon's life, invisibly. That sits awkwardly with CLAUDE.md's
log-driven-development rule.

I did not fix it: a sensible fix needs dedup/rate-limiting so a persistently
failing probe does not spam the log every tick, and that is a design decision
about shared code I should not make unilaterally in review. Flagging it for you.

Blast radius today is nil [reasoned]: I read all five in-tree source
status() implementations (ai-gateway, otel, gascity, hermes, vector-search) and
every one is a pure in-memory field read - no I/O, no mutation, nothing that can
throw. The exposure is third-party plugins only.

Design notes (no action, recorded only)

  • The backfill/live asymmetry (importing a Desktop rollout does not put Desktop
    in recent clients) is real, but you disclosed it in the PR body and in LLP
    0164's consequences. Correct call - the line means "traffic arrived here".
  • LLP 0164 uses <a id="..."> anchors where much of the corpus uses {#anchor}.
    Both are used in roughly equal numbers (33 vs 38), so this is not a deviation.

Checks

All run by me, with my own baselines.

Check Baseline origin/master@73b4618 PR head 7d1b111 After fix be9ce7f
npm test 3037 / 3028 pass / 8 fail 3053 / 3044 / 8 fail 3059 / 3050 / 8 fail
npm run typecheck - clean clean
gateway_codex_capture n/a ok ok

The 8 failures are identical by name across all three columns - the
pre-existing test/core/leave-command.test.js set. Confirmed against my own
baseline, not assumed.

Also ok after the fix: gateway_claude_capture, daemon_foreground_start_stop,
status_diagnostics, ai_gateway_passthrough.

walkthrough_to_first_query: confirmed also failing on pristine master
(ai_gateway_messages has exactly one row, value=0). Not attributable here.

The ground-truth gate - reproduced [executed]

  • The daemon test on a pristine origin/master worktree fails with exactly the
    quoted error: source details never refreshed (probes=1). The frozen-source-
    details bug is real and the reproduction is honest.
  • The smoke assertion copied onto master fails with
    recentEntrypointsFromSources is not a function. Verbatim as claimed.

Tests are non-vacuous [executed]

Mutation-tested. Caught: dropping the per-tick refresh; dropping the shutdown
refresh; negative/raw-seconds age formatting; no longer dropping malformed
last_seen; bucketing a missing entrypoint as "unknown". The gateway wiring
(liveState.entrypoints.record) is not covered by any unit test, but removing
it fails the smoke - so it is covered end to end. Only genuine gap was F3.

Time handling [executed]

formatEntrypointAge on future (+1h and +10y) -> just now; unparseable and
empty -> at an unreadable time; 1970 -> 20664d ago. No NaN, no negative
age. Sound.

@ref hygiene [executed]

56 @refs across the changed files; zero genuinely broken. All three LLP
0164 anchors (#gateway-tracks-what-core-cannot-name,
#status-reads-it-from-the-status-file, #not-liveness-gated) resolve as
explicit <a id> targets. (My first pass reported 11 breaks; that was my
slugifier not handling {#anchor} headings. The 4 residual LLP 0045 hits are
pre-existing on master and resolve fine - my slugifier does not strip em dashes
the way GitHub does.)

LLP 0164's number is unclaimed. Checked every one of the 26 origin/* refs:
fix/issue-421 is the sole claimant. No collision with #469's duplicate-number
problem.

LLP 0086 and 0141 edits are accurate. 0141's "still cannot say ..." is correctly
retired rather than left stale, and 0086's note correctly narrows the impact
("changes nothing here, since host and port are fixed at bind time").


Documentation honesty - passes

I checked this specifically because a document asserting unverified behaviour is
worse than no document.

Every step added to docs/ACCEPTANCE.md is one whose commands and output
shapes I executed myself:

  • hyp status -> produces the recent clients: block in the documented shape.
  • hyp status --json | grep -A 6 recent_entrypoints -> I ran this exact pipeline
    and it emits exactly one complete entry, as the doc implies.
  • Both expected-negative cases (restarted daemon shows nothing; null originator
    means no entry) follow from behaviour I verified.

The hardware requirement is already explicitly marked at section level
("Requires: macOS with Codex Desktop installed and signed in"), and the
author disclosed in the PR body that they could not run steps 1-7. That
disclosure is correct practice and I am not faulting it - the added material is
exactly the part that does not need a Mac.


Executed vs reasoned

Executed: the no-cache-read proof (syscall trace + two poisoned-cache runs,
on both 7d1b111 and be9ce7f); the HTTP parser bounds; the full F1 attack
chain through the real binary before and after the fix; both master
reproductions; mutation testing; time-formatting edge cases; all test/typecheck/
smoke runs and both baselines; ref and LLP-number checks; the ACCEPTANCE.md
commands.

Reasoned: F4's blast radius (read all five in-tree status() bodies rather
than executing a throwing plugin); the reachability characterisation of F1.

…k loop

Round-2 review of #502. Three gaps, all executed against the real tracker,
the real core reader, and a real daemon.

1. `sanitizeLabel` stripped control bytes but not the two other classes that
   make a label render as something other than what it stores. Bidi overrides
   and isolates survived to the terminal, and an unterminated one reorders
   past the end of the label into the rest of the status line. Zero-width and
   default-ignorable characters survived to the map *key*, which is exactly
   what the tracker's own comment claimed could not happen: 500 variants of
   `codex-tui` that differ only in zero-width spaces filled all 32 slots and
   evicted every real surface. Both classes are now stripped, so those 500
   variants collapse to one entry. Confusables stay out of scope on purpose.

   Also: the clamp appended its marker past the ceiling (120 chars in, 123
   out) and could cut between the halves of an astral character, putting a
   lone surrogate into `status.json`. The marker is now inside the ceiling
   and a stranded high surrogate is dropped.

2. Core sanitized each label it read back but capped only their length, not
   their number, though the comment justifying the read-side sanitize is that
   this build may not have written the file. A status file with 5000 entries
   printed 5000 lines. Capped at 32 after the sort, matching the writer.

3. The per-tick refresh put plugin code on the tick loop's critical path with
   no bound. A source whose `status()` boots fine and then hangs froze
   `persist()`, so *every* field in `status.json` stopped updating while the
   daemon went on reporting itself healthy, and it hung `hyp daemon stop`
   outright (shutdown refreshes too). Probes now run under a timeout, a
   source with an outstanding probe is skipped rather than piling up an
   unclosed span per tick, and failures are logged once per transition
   (`daemon.source_status_failed` / `_recovered`) rather than never, which
   was the old `safeStatus` behaviour, or every tick forever.

Verified by execution before and after: status.json keeps being written and
the daemon still stops in ms with a permanently hung probe, and ~500 ticks
produce at most 3 log lines, not 500.

Co-Authored-By: Claude <noreply@anthropic.com>
@philcunliffe

Copy link
Copy Markdown
Contributor Author

Verdict

Findings. This is round 2 of 2 - the last round before the ladder routes
this PR to triage, so I fixed everything actionable rather than leaving it.
Reviewed at head be9ce7f; fixes pushed as 2487861.

Three new findings, all fixed and all verified against a fresh checkout of the
pushed remote SHA rather than my working tree. F4 from round 1 is settled: it
was real, it was understated, and it is now fixed
- see below.

I did not revisit the maintainer's option-4 decision.

Tooling note, same as round 1: the code-review skill is not installed in this
clone and codex is not on PATH, so this was a direct audit. There is no
llp-ref-hygiene npm script either (ref hygiene here is the prompt-based
/ref-check skill), so I ran a mechanical equivalent.


Round 1's fixes: verified, and the delta is sound

7d1b111..be9ce7f is one commit touching 7 files. sanitizeLabel is applied
at both points round 1 claimed - entrypoint_activity.js:64,66 (tracker
record) and status.js:163,169 (core's reader) - and I confirmed that record
is the map's only writer and recentEntrypointsFromSources the only reader that
reaches a terminal, so there is no third path around either. [executed]

Its stated scope holds: driving 13 hostile payloads through the real tracker and
the real core reader, C0/C1/DEL/LS/PS are stripped everywhere, a 30KB
entrypoint lands as 123 chars, and 13 hostile entries produce a 3863-byte
status.json (was 30375 bytes for a single entry at 7d1b111). [executed]

I did not redo the no-cache-read proof; round 1 established it three ways and the
delta does not touch that path.


F1 continued [Medium] The sanitizer was partial - src/core/util/json_util.js:40

A partial sanitizer is worse than an obvious hole because it reads as solved, so
I attacked the character class directly rather than trusting it. Two of the three
classes that make a label render as something other than what it stores were not
covered.

Class At be9ce7f Now
C0/C1/DEL/LS/PS stripped stripped
Bidi overrides + isolates (U+202A-202E, U+2066-2069) SURVIVES stripped
Zero-width / default-ignorable (ZWSP/ZWNJ/ZWJ, U+2060-2064, BOM, SHY, U+180E, variation selectors) SURVIVES stripped

Executed, through the real tracker and the real reader:

  • Bidi. CONTROL_CHARS was scoped to "move the cursor, erase a line, open an
    escape sequence, or split into a second line". A bidi override does none of
    those - it prints nothing and reverses what follows, and an unterminated
    U+202E keeps reordering past the end of the label into the rest of the rendered
    status line. Same operator trust surface F1 was about; the other half of it.
  • Zero-width, and this one falsifies a comment in the PR.
    entrypoint_activity.js:54 states the eviction cap "cannot be diluted by values
    that differ only in bytes no reader will ever see." It could. Feeding
    'codex-tui' + ZWSP.repeat(i) for i in 0..499 produced 32 distinct map
    entries that all render identically
    , filling every slot and evicting every
    real surface. After the fix the same input produces 1 entry. That comment is
    now true rather than aspirational.

Also fixed in the same function, both executed:

  • The clamp appended its marker past the ceiling: MAX_LABEL_CHARS = 120 in,
    123 out (json_util.js:66). The marker is now inside the ceiling.
  • slice(0, max) counts UTF-16 code units, so a cut could land between the halves
    of an astral character and put a lone surrogate (isWellFormed() === false)
    into status.json. No corruption resulted - JSON.stringify escapes it, so the
    file stays valid JSON - but a non-JS consumer of hyp status --json has no
    reason to tolerate it. A stranded high surrogate is now dropped.

Confusables are deliberately out of scope and I say so in the code and the
LLP: this bounds what a label does, not what it looks like. Two labels built
from different but similar-looking real letters must stay distinct.

Reachability is unchanged from round 1's honest characterisation: the transcript
route needs a malicious or compromised client, or local write access to the
projects dir. The length half still needs no hostility at all.

F5 [Low] Core capped label length but not label count - src/core/daemon/status.js:150-175

The comment justifying the read-side sanitize is that core "must not assume the
daemon that wrote it was this version, was this build, or was well behaved."
Under exactly that reasoning the count needs a cap too, and there wasn't one:
the 32-entry cap lived only in the gateway. A status.json carrying 5000 entries
returned 5000, and hyp status prints one line each. [executed]

Now capped at 32, applied after the sort so what survives is the most recently
seen. Same number as the writer, for the same reason, at the last point before
render.

F4 [now Medium, was Low] - SETTLED, and it was worse than "an error is swallowed"

Verdict: not nil-risk. Fixed. Round 1 declined this as a design call it should
not make unilaterally, which was the right call for round 1. I am the last round,
and the reason to act is not the swallowed error - it is what I found when I
executed the case round 1 reasoned about.

First, a correction to round 1's enumeration. Round 1 read "all five in-tree
source status() implementations". There are eight: it missed
hermes/src/source.js:198 (the disabled-mode source) and both
context-graph-enrich sources (batch.js:372, propose.js:447). Its
conclusion survives - I read all eight and every one is a pure in-memory field
read, so the in-tree blast radius really is nil - but the count was wrong, so the
"reasoned" label was carrying more weight than it had earned. [executed - I read
all eight]

Second, a correction to "invisibly". src/core/registry/sources.js:174 wraps
the probe in withSpan('source.status'), which calls span.recordException and
sets error_kind. So a throwing status() is not invisible to tracing. It is
invisible to the daemon log, which is what an operator actually has on a default
install with no trace exporter. Executed: over ~150 ticks with a throwing probe,
daemon.log contained zero lines mentioning it, and the daemon kept
reporting itself healthy.

Third, and this is why it needed fixing. F4 asked about a probe that
throws. I also executed a probe that hangs, which the kernel contract
equally permits, and it is a different animal:

Behaviour (third-party status()) At be9ce7f [executed]
throws every tick daemon survives, ticks persist, no log line ever
hangs after booting cleanly status.json mtime frozen at 4007ms and climbing - the tick loop stopped writing the file at all
... and then stop() STOP TIMED OUT after 8s - hyp daemon stop hangs

persist() is downstream of refreshSourceDetails() (runtime.js:529), so a
single hung probe freezes every field in status.json, not just that
source's - sink tick times, sink successes, daemon state, warnings - while the
daemon goes on reporting itself healthy. And the shutdown path added by this PR
(runtime.js:611) refreshes too, so the daemon becomes unstoppable and must be
SIGKILLed. Before this PR a probe was awaited only at boot, where a hang is at
least loud: the daemon does not start.

This is squarely new in this PR, so it is this PR's to answer.

Fix (2487861, src/core/daemon/runtime.js):

  1. Each probe runs under a 5s timeout, well under the tick floor.
  2. A source with an outstanding probe is skipped rather than starting a fresh
    call each tick - otherwise a permanently hung probe would hold open a new
    unclosed source.status span every tick for the daemon's life.
  3. Failures are logged once per transition: daemon.source_status_failed when
    a source's failure changes, daemon.source_status_recovered when it clears.
    That is the dedup round 1 correctly declined to invent on its own, and it
    satisfies CLAUDE.md's log-driven-development rule without trading silence for
    a line every 60s forever.

Same three cases after the fix, executed on a fresh checkout of 2487861:

Behaviour After
throws ticks persist (mtime age 4ms), stop in 2ms, 1 log line over ~500 ticks
hangs after boot ticks persist (mtime age 16ms), stop in 2ms, 3 log lines over ~500 ticks
healthy unchanged - 156 probes in 4s, zero spurious log lines

Details keep their last good value in every failure case, which is the documented
best-effort contract.


Scope items with nothing to report

Unbounded growth, count half. Eviction cannot be defeated. The re-insert on
hit keeps Map iteration order least-recently-seen-first, which is what the
while (seen.size > max) loop relies on, and that loop only runs in the branch
that can grow the map. With the F1 fix the cap is no longer dilutable by
invisible bytes either. Worst case status.json is now bounded at roughly 16KB
(32 entries x 2 labels x 120 UTF-16 units, up to 4 bytes each) - note the cap
counts code units, not bytes, which I documented rather than changed.
[executed + reasoned]

Nothing expensive runs 60x more often. The only new per-tick work is the eight
status() bodies, all pure in-memory reads, plus one source.status span per
source per tick from the withSpan wrapper. The runtime.js delta against
master is exactly the new function plus two await refreshSourceDetails() calls -
nothing else moved. A throwing probe cannot crash the loop: safeStatus caught
it before, probeSourceDetails catches it now, and runTick's withSpan chain
already has its own .catch. [executed]

LLP 0164. Number is unclaimed: re-checked after a fresh fetch against all
26 origin/* refs, fix/issue-421 is the sole claimant. No collision with
the #469 duplicate-number problem. All three anchors
(#gateway-tracks-what-core-cannot-name,
#status-reads-it-from-the-status-file, #not-liveness-gated) resolve as
explicit <a id> targets. The LLP 0086 and 0141 edits describe what the code
does. I updated 0164 in the same commit as the code (living-docs rule): the
Bounded section now describes the full character class and why the
zero-width group is load-bearing rather than cosmetic, the tick section documents
the probe timeout and the transition-only log, and core's paragraph documents the
count cap. [executed]

@ref hygiene. 226 @refs across 51 files, zero genuinely broken. The 14
my checker flags are all LLP 0045/0044/0042/0074/0007 and are my slugifier not
reproducing GitHub's handling of / in headings (## Module / seam breakdown (independently-mergeable tasks) -> module--seam-breakdown-..., two hyphens).
None is on a line this PR added. Same conclusion round 1 reached. [executed]


Checks

All run by me, both suites serially, against my own baseline.

Check Baseline origin/master@bd7dd0b After fix 2487861
npm test 3087 / 3078 pass / 8 fail 3064 / 3055 pass / 8 fail
npm run typecheck - clean
npm run smoke -- gateway_codex_capture - ok

The 8 failures are identical by name in both columns - the pre-existing
test/core/leave-command.test.js set (leave after join removes the seed ...
through leave self-heals an org attach whose plugin is gone ...). Confirmed
against my own baseline run, not assumed.

Also ok on the remote SHA: daemon_foreground_start_stop, status_diagnostics,
gateway_claude_capture.

walkthrough_to_first_query: confirmed also failing on pristine master - I
ran it on both trees myself, identical failure (ai_gateway_messages has exactly one row for this dev_run_id, value=0). Not attributable here.

Tests are non-vacuous [executed]

I added 5 tests and mutation-checked them by stashing only the src/ changes
and re-running: all 5 fail without the fix, 22/27 pass. They pin the bidi
strip, the cap-dilution collapse, the well-formed clamp, the reader-side count
cap, and the hung-probe case (status file keeps advancing, shutdown completes,
failure logged at least once and at most 4 times).

Verified against the committed remote tree, not a green suite

Per LLP 0002 I re-ran the full attack script and both daemon probes from a
fresh worktree checked out at the pushed remote SHA 2487861, plus
typecheck, the 27 targeted tests, and gateway_codex_capture. Every result in
the "After" columns above is from that tree.


Design notes (no action, recorded only)

  • The 5s probe timeout and the transition-only log shape are my calls, made
    because I am the last round and the alternative was handing triage an
    unstoppable-daemon vector. Both are documented in LLP 0164. If you prefer a
    different bound or a different event shape, they are one constant and one
    function.
  • The backfill/live asymmetry remains disclosed and correct, as round 1 noted.

Executed vs reasoned

Executed: the full sanitizer attack (13 payload classes through the real
tracker and real reader, before and after, on both my tree and the remote SHA);
the cap-dilution reproduction; the reader-side count reproduction; the lone
surrogate; all three daemon behaviours (ok / throw / hang-after-boot)
against a real runDaemon with a real third-party plugin fixture, before and
after, including log-line counts over ~500 ticks; reading all eight in-tree
status() bodies; mutation-testing my own 5 tests; both test baselines;
typecheck; five smokes; the walkthrough_to_first_query master comparison; the
LLP-number scan over 26 refs; the @ref scan.

Reasoned: that the in-tree blast radius of a throwing or hanging status()
is nil (I read all eight bodies rather than executing each plugin); the worst-case
status.json size arithmetic; the reachability characterisation of F1.

@philcunliffe

Copy link
Copy Markdown
Contributor Author

TRIAGE verdict: safe to ship

The review fix-loop exhausted its 2 rounds, so this landed on triage
(LLP 0017). I independently re-checked every finding from both rounds
against head 2487861 rather than trusting the review comments at face
value:

  • Read the actual fix code: sanitizeLabel in src/core/util/json_util.js
    (strips C0/C1, bidi overrides/isolates, and zero-width/default-ignorable
    characters; clamps to 120 chars without a stranded surrogate), the
    read-side MAX_RECENT_ENTRYPOINTS = 32 cap in src/core/daemon/status.js,
    and the 5s probe timeout + in-flight skip + transition-only failure
    logging in src/core/daemon/runtime.js.
  • Ran npm test on a fresh worktree at 2487861: 3055/3055 non-pre-existing
    tests pass, with the same 8 pre-existing test/core/leave-command.test.js
    failures both review rounds already attributed to master.
  • Ran npm run typecheck (clean) and npm run smoke -- gateway_codex_capture
    (ok).
  • Confirmed the dedicated hung-probe test
    (test/core/daemon-source-details-refresh.test.js) is non-vacuous: it
    stages a real plugin whose status() never resolves, and asserts
    status.json keeps advancing, hyp daemon stop still completes, and the
    failure is logged (bounded, not once per tick).

Result: every finding from both review rounds (F1-F5: unsanitized/
unbounded client-supplied labels, a partial sanitizer missing bidi/
zero-width characters, an uncapped read-side entry count, and a hung/
throwing plugin probe that could freeze status.json and make the daemon
unstoppable) was genuinely fixed within the PR itself. No TRUE BLOCKER is
open at the current head.

One non-blocking item remains: the round-2 reviewer chose the 5s probe
timeout and the log-event shape (daemon.source_status_failed/
_recovered) unilaterally while fixing the PR forward, and said as much in
their comment. The author hasn't explicitly signed off on those specific
values. Deferred as a follow-up rather than blocking merge, since the
current values are safe and tested: #511.

@philcunliffe philcunliffe added the neutral:approved neutral reviewed this and holds it for a maintainer merge (own or adopted PR; LLP 0025/0030) label Jul 31, 2026
philcunliffe pushed a commit that referenced this pull request Jul 31, 2026
PR #502 (fix/issue-421) already mints llp/0164-status-names-recent-clients-
from-gateway-entrypoints.decision.md and is held+approved awaiting a human
merge, so it has the prior claim on 0164. This branch's decision doc and
every @ref pointing at it move to 0165, which is free across master and all
open PR branches.

No content change: the file is renamed and the number updated at all 13
reference sites (5 source, 3 test, 3 in LLP 0151's Extended-by forward
refs, the doc title, and the smoke flow). Both anchors still resolve.
philcunliffe pushed a commit that referenced this pull request Jul 31, 2026
Three open branches independently minted 0164 off the same master
high-water mark, none able to see the others:

  fix/issue-421 (PR #502)  status-names-recent-clients-from-gateway-entrypoints
  fix/issue-473 (PR #517)  codex-flat-pair-needs-a-namespace-signal -> 0165
  fix/issue-451 (PR #520)  session-control-plane-states-its-guarantees -> 0166

PR #502 is held and approved awaiting a human merge, so it keeps 0164.
This branch takes 0166; 0165 went to PR #517.

No content change: the file is renamed and the number updated at every
reference site (3 in session_command.js, 1 in the new responder-trust
test, the Extended-by forward-ref in LLP 0067, and the doc title). The
#stated-not-proved and #membership-not-grain anchors still resolve.
@philcunliffe
philcunliffe merged commit e35c307 into master Jul 31, 2026
9 checks passed
@philcunliffe
philcunliffe deleted the fix/issue-421 branch July 31, 2026 04:51
philcunliffe added a commit that referenced this pull request Jul 31, 2026
…451) (#520)

* hyp session states that the control responder is never authenticated (#451)

`validateControlResponse` (PR #439) proves the responder echoed our token;
it cannot prove the responder IS the gateway. A local process that binds the
resolved port and echoes the token back still yields `exit 0, ignored: true`,
so the privacy control tells a user "you are opted out" when nothing recorded
the decision.

Per the maintainer's direction on #451, this is accept-and-document: the
attack presupposes local code execution as this user, a gateway-written secret
is readable by whoever can bind the port, and peer-process identity has no
portable form (three platform implementations plus a native dependency). So
the guarantee is stated rather than proved.

- human output gains a `trust:` note beside every confirmed answer, on
  `status` and on both mutation verbs, naming the endpoint that was trusted
- `--json` gains `endpoint_authenticated: false`, constant by contract rather
  than by outcome, so a consumer never infers authenticity from silence
- the disclosure is unconditional and rides the `daemon_status` path too,
  which said nothing at all before: a live daemon's status.json is evidence
  about a past bind, not about who answers now
- `endpoint_source` and every existing fail-closed refusal are unchanged

LLP 0164 records the decision and carries the companion #460 contract
statement (membership is not a match) so the control-plane guarantees read as
one story; LLP 0067 §cli-response-check gets the forward-ref.

test/plugins/ai-gateway-session-responder-trust.test.js stands up an impostor
that echoes the token, pins that its answer is still believed (the accepted
residual), and asserts the disclosure on every surface.

Fixes #451

Co-Authored-By: Claude <noreply@anthropic.com>

* Renumber LLP 0164 -> 0166 to resolve a cross-branch number collision

Three open branches independently minted 0164 off the same master
high-water mark, none able to see the others:

  fix/issue-421 (PR #502)  status-names-recent-clients-from-gateway-entrypoints
  fix/issue-473 (PR #517)  codex-flat-pair-needs-a-namespace-signal -> 0165
  fix/issue-451 (PR #520)  session-control-plane-states-its-guarantees -> 0166

PR #502 is held and approved awaiting a human merge, so it keeps 0164.
This branch takes 0166; 0165 went to PR #517.

No content change: the file is renamed and the number updated at every
reference site (3 in session_command.js, 1 in the new responder-trust
test, the Extended-by forward-ref in LLP 0067, and the doc title). The
#stated-not-proved and #membership-not-grain anchors still resolve.

* review: document endpoint_authenticated where the report shape lives

Three review fixes on top of #451's disclosure work, no behaviour change:

- `types.d.ts`: `SessionStatusReport` bills itself as "what `hyp session
  status` reports, in `--json` field order", and a maintainer adding a report
  shape reads it. It said nothing about `endpoint_authenticated`, so the
  constant-by-contract rule lived only inside `writeStatus`. State the
  envelope, the by-contract choice, and the "a real check needs a new field"
  consequence there, with the LLP 0166 ref.
- LLP 0166 named `RESPONDER_TRUST_NOTE`, a symbol that does not exist; the
  code has `responderTrustNote(endpoint)`, and the reason it is a function
  (it names the endpoint) is the point.
- Drop the unused `@import { IncomingMessage, ServerResponse }` copied into
  the new test from its sibling suite.

Co-Authored-By: Claude <noreply@anthropic.com>

* review: point the LLP 0067 link at the section its label names

The new `responderTrustNote` docblock links `[LLP 0067 §cli-response-check]`
at the file with no fragment, so a reader who follows the label lands at the
top of a 600-line design doc. Every other fragment-bearing LLP link in the
tree carries its anchor (`sync.js`, `verb_codec.js`, `remote_commands.js`,
`first_sync_hold.js`); this one was the outlier.

Co-Authored-By: Claude <noreply@anthropic.com>

---------

Co-authored-by: test <test@test.com>
Co-authored-by: Claude <noreply@anthropic.com>
Co-authored-by: test <test@example.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

neutral:approved neutral reviewed this and holds it for a maintainer merge (own or adopted PR; LLP 0025/0030)

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Make Codex Desktop capture explicit and acceptance-tested

1 participant