Repository navigation
feat(freshclaude): real-CLI ground truth, CLI simulator, Stop without a banner or a lost bell, and structured sidecar logs (core PR-A) - #863
Merged
Conversation
Core runtime design (identity, turn accounting, Stop, lifecycle, rollback, fork, stuck detection, test realism) plus the transcript/live-rendering, client alerts/UX, and agent API/hosted/ops workstream plans. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Add test/integration/providers/claude-cli-contract.test.ts, one named test
per fact T1-T19 of the freshclaude core-runtime design, run against the
pinned Claude Agent SDK 0.3.237 and its bundled CLI 2.1.237. The CLI only
ever talks to a loopback fake Anthropic endpoint, with a throwaway HOME and
CLAUDE_CONFIG_DIR, a dummy key and a dead proxy for any other egress.
- claude-local-provider.mjs: the fake endpoint extracted from the
checkpoint harness, plus the pinned-CLI resolver and the isolated
fixture. Prompt markers choose the model's behavior: text, SLOW, a Bash
tool (TOOL:sleep=N[,timeout=MS]), APIERR:<status>, HANG, a hung stdio MCP
tool (MCP:hang, claude-hang-mcp-server.mjs) and a background Agent
(AGENT:async). The marker grammar lives in
test/fixtures/claude-cli-sim/prompt-markers.mjs so the CLI simulator can
share it. Runnable standalone (--serve) for non-JS lanes.
- claude-checkpoint-uuid-replay.mjs now imports the provider, resolver and
fixture; its test is unchanged.
- claude-cli-scenarios.mjs drives one scenario per process and prints a
trace; claude-cli-trace.ts runs it and labels every uuid.
- Cold-Stop facts assert the outcome (runs to success without a re-send,
aborted with one), never the receipt contents.
- A missing pinned CLI makes every test fail (PinnedClaudeMissingError);
nothing is skipped. The suite runs in the default Vitest config, and the
Cloud Run image already ships the pinned CLI (root node_modules plus the
sidecar's node_modules).
Deviation from the design's T16 wording: after /clear the CLI's
conversation_reset carries a new_conversation_id that is NOT the new
session id. The new session id arrives in the next system/init and the
/clear result; its transcript exists at once and `resume` on it boots, while
`resume` on new_conversation_id fails ("No conversation found"). The
contract pins the observed behavior; C24's re-key must take the new id from
the init that follows conversation_reset.
Other facts confirmed beyond the design text: an API error result also ends
the command with lifecycle `cancelled`; setModel before the first prompt
replays a /model command under a CLI-minted uuid (the "uuids we never
sent" of T8); tool heartbeats carry parent_tool_use_id = the tool_use id
and tool_use_id = "<id>-heartbeat-<n>"; BASH_MAX_TIMEOUT_MS caps a
foreground Bash call even when the model asks for 600000 ms.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Add test/fixtures/claude-cli-sim/query.mjs, a drop-in query() for the sidecar's FRESHELL_CLAUDE_SDK_QUERY_MODULE seam that behaves like the pinned SDK 0.3.237 driving CLI 2.1.237, and claude-cli-conformance.test.ts, which runs every scenario of claude-cli-scenarios.mjs on the real CLI and on the simulator and requires equal normalized traces (claude-cli-normalize.ts). The simulator implements T1-T19: boot without a prompt and initializationResult() from a recorded catalog (catalog.json), typed boot refusals and the "already in use" exit; a stand-in CLI process (stand-in-cli.mjs) whose pid and death behave like the real CLI's (SIGKILL ends the stream with the SDK's termination error); per-turn init, command_lifecycle, session_state_changed, replays (including uuids the CLI mints itself); sends absorbed during a tool versus queued during text; interrupts with receipts, aborted_streaming/aborted_tools results and cancelled lifecycles, and both cold-Stop receipt shapes (FRESHELL_CLAUDE_SIM_COLD_STOP_RECEIPT); lazy JSONL under CLAUDE_CONFIG_DIR for fresh, resumed and forked sessions (transcript.mjs); /compact, /clear and local slash commands (local-commands.mjs); tool heartbeats, Bash task promotion and the BASH_MAX_TIMEOUT_MS cap on a virtual clock (FRESHELL_CLAUDE_SIM_TIME_SCALE), background agents and the CLI-initiated turn they start, API errors, hangs, canUseTool with the CLI's suggestion shapes and ExitPlanMode (tools.mjs); a stdin stall for the host process (FRESHELL_CLAUDE_SIM_STALL_STDIN); and an options ledger (FRESHELL_CLAUDE_SIM_LEDGER). The model's side of a turn comes from the prompt markers shared with the loopback provider, or from a program in the e2e fixtures' rule language (FRESHELL_CLAUDE_SIM_PROGRAM, _FILE, or the transcript plan's FRESHELL_CLAUDE_SDK_SCRIPT alias; triggers "start" and "msg:prompt"; emissions text, tool, approval, question, exit_plan, completion, api_error, hang, crash, stall_stdin, cli_turn, marker). claude-cli-sim.test.ts covers the program language, the ledger, and the real index.mjs running over the simulator through the seam. Scenario and harness changes made for parity: - New conformance scenario `permission` (canUseTool allow/deny shapes, requires_action, ExitPlanMode in plan mode) with the TOOL:touch and PLAN:exit markers. - The compact scenario now runs a successful /compact, a turn, then a failed one. Deviation: after a FAILED compaction the real CLI re-yields the failed command's synthetic error message (same uuid) when the next compaction succeeds (its "preserved segment"); the simulator does not model that quirk, so the scenario no longer runs a success right after a failure. T15's facts are unchanged. - The crash observation counts messages after the throw instead of the absolute frame count. - The scenario runner waits for its stdout to flush before exiting (large results on a pipe were truncated by process.exit). - Normalization keeps the CLI process exit out of the ordered frames: Node can report a child's exit before its last stdout line. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
A Cloud Run run of the conformance suite failed sim/heartbeat with "ENOTEMPTY: directory not empty, rmdir .../.claude/projects/...": the scenario closes two sessions while their tools run (a Bash sleep and a hung MCP tool) and then deletes the fixture. Closing aborted the turn, and the aborted tool kept unwinding after the stream had ended: abortTool appended the rejected tool_result and the interruption notice to the transcript, and the 100 ms flush timer wrote them while the fixture's rm was walking the directory. The real CLI is killed by close() and writes nothing more. Transcript gains close(): it writes what is buffered, then ignores every later flush or schedule. The simulator closes the transcript wherever its stream ends (terminate for a close or a CLI death, endWhenIdle for the end of input), instead of only flushing it. claude-cli-sim.test.ts: a closed simulator leaves CLAUDE_CONFIG_DIR unchanged after the close, for a Bash call and for a hung MCP tool. Both cases failed before the fix with the late tool_result, token reminder and "[Request interrupted by user for tool use]" lines. (Two near-duplicate red tests from an interrupted earlier attempt were merged into this one parametrized test.) Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…ke the real CLI
Probing the pinned CLI 2.1.237 for which settings calls it refuses (needed to
port the sidecar's rejected-configure tests onto the simulator in C03):
setModel and applyFlagSettings accept any name (setModel('no-such-model')
succeeds and replays its /model command; the next init reports the model),
and only setPermissionMode with an unknown mode is rejected, with
"Cannot set permission mode: must be one of acceptEdits, auto,
bypassPermissions, default, dontAsk, plan". The session keeps working.
- New conformance scenario `settings` (claude-cli-scenarios.mjs) and a named
contract fact extending T7.
- The simulator's setPermissionMode now rejects an unknown mode with the
CLI's message (PERMISSION_MODES in sdk-shapes.mjs); it accepted anything.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Move the checkpoint-attempt parsing, the bounded pending set and the replay-binding rule out of index.mjs into checkpoint-identity.mjs, with no behavior change. index.mjs now calls checkpointAttemptOf, rememberCheckpointAttempt and takeCheckpointIdentity. Why now: C03 moves user-message-replay.test.ts off its scripted fake SDK and onto the real index.mjs over the CLI simulator. The fake could hand the sidecar a NON-replay user row carrying the attempt's own uuid (its "ordinary" mode), a shape the real CLI never produces and a faithful simulator cannot. checkpoint-identity.test.ts pins that guard (and the mismatch, role, uuid-shape, bound and refresh rules) on the pure function, so the port loses no coverage. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…lator Add test/unit/claude-sidecar/harness.ts: startRealSidecar() spawns the production index.mjs with the CLI simulator in the FRESHELL_CLAUDE_SDK_QUERY_MODULE seam, in a throwaway HOME / CLAUDE_CONFIG_DIR / workspace with a clean environment, and kills and removes it when the test finishes. It offers frame waits, create, a one-chunk sendLines, a stdin barrier, and reads what the SDK side received from the simulator's ledger. Ported to it (interrupt-idle-gate, user-message-replay, permission-channel, rollback-quiesce, session-settings); their scripted query modules are deleted (fake-checkpoint-sdk, fake-query-module, interrupt-gate-query-module, settings-query-module) and the retirement ledger row that named fake-query-module now names the harness. Pure-function cases are untouched. Simulator additions the harness needs (tested in claude-cli-sim.test.ts): - ledger rows `boot`, `prompt` (each command as it starts, with the model, effort and permission mode it runs under) and `can_use_tool` (each answer), every row tagged with its query() ordinal; effort is tracked from the query option and applyFlagSettings; - programs may choose a tool call's tool_use id (`id`) and the whole AskUserQuestion input (`input`), both of which are the model's side. Where the old fixtures differed from the real CLI, the ports follow the CLI (deviations from "unchanged assertions"): - No init at create (T1): tests that waited for init after create now wait for the simulated boot or for the init that opens the turn. - The fakes' probe.* stdout frames are read from the ledger instead (probe.prompt -> waitForPrompt, probe.resolved -> can_use_tool rows, probe.hosted-mcp / probe.auth-env -> the query row's options and env key names). The auth check is now key presence; the ledger never records values. - A send pushed while a tool runs is absorbed into that turn (T10), so the permission case waits for each turn to end before the next prompt. - rollback-quiesce: the fake parked its input pump; the real SDK pump is eager, so the never-handed window is one stdin chunk (sendLines), the queued-pull case syncs on a stdin barrier, and a hung request (HANG) holds the CLI instead of timers. The unrelated-result case runs a finished turn and a hung turn ahead of the compact and interrupts the hung one. - user-message-replay: mismatch, absent and ordinary are streams the CLI produces (a CLI-minted /model replay, an unknown slash command that is never replayed, tool-result rows). The fake-only shape (a non-replay row with our uuid) is pinned on the pure module by the previous commit. - session-settings: the CLI accepts any model or effort name and refuses only an unknown permission mode (previous commit), so the two rejected- configure cases use that; the partial-failure case is renamed. The commands catalog is published after the first turn's init today, and its assertion now checks the real catalog (compact kept with its hint; the CLI's terminal-only doctor and color removed) instead of a fake list. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…decar module claude_sidecar_interrupt_dispatch.rs copies the real index.mjs and its helper modules into a temp dir next to a stub SDK. It listed the helpers by name, so the new checkpoint-identity.mjs (a0a5634) was missing and node failed to resolve the import: that test has been red since a0a5634. It now stages every *.mjs of crates/freshell-claude-sidecar, so the modules the coming commits add (boot.mjs, log.mjs, sdk-message-frames.mjs) cannot break it the same way. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Add crates/freshell-ws/tests/common/claude_rig.rs: a real freshell-ws server on an ephemeral loopback port whose claude sessions run the REAL sidecar (crates/freshell-claude-sidecar/index.mjs). - RigKind::Sim (RealSidecarSim): over the CLI simulator through the FRESHELL_CLAUDE_SDK_QUERY_MODULE seam; the simulator's ledger is readable (ledger_rows). - RigKind::Cli (RealSidecarCli): over the pinned SDK 0.3.237 and its bundled CLI 2.1.237, against the local fake endpoint (claude-local-provider.mjs --serve) with a dummy key and dead proxies for any other egress. Never the real API. Each rig has a throwaway HOME / CLAUDE_CONFIG_DIR / workspace. A missing node, simulator or pinned CLI panics with the reason (the pinned CLI is checked with the same resolver as the real-CLI contract suite); nothing is skipped. RigConnection drives one WS client (create, send, kill, event waits and counts over every frame read). New freshagent_claude_real_cli.rs with the smoke test on both rigs (sim::create_send_result_rings_once, cli::create_send_result_rings_once): today's plain turn gives a success result and exactly one freshAgent.turn.complete with a numeric `at`; the transcript lands in the rig's isolated config dir; a kill ends the run and adds no edge; on the simulator the ledger shows the one prompt. The `test-support` feature (how it is realized): freshell-freshagent gains the feature and, behind it, FreshClaudeState::with_sidecar_environment, which gives every sidecar that state spawns exactly the rig's environment (plus its ownership tag) instead of the server's. Production never sets it: the field is None and the sidecar inherits as before. create, attach-resume and rollback now spawn through one state method. freshell-ws enables the feature in its dev-dependencies, so the rigs run in the plain `cargo test --workspace` that rust-gate uses, need no process-global env vars and can run in parallel. Unit test: a_configured_sidecar_environment_replaces_the_inherited_one. The CLI rig hands its dummy key to the SDK child with FRESHELL_CLAUDE_ONECLI_API_KEY=1 (the sidecar otherwise strips ANTHROPIC_API_KEY), the same managed-key path production uses. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The browser sends every message with a checkpoint intent, and handle_send
fences a checkpointed send through the ownership coordinator. Two things
made that fence refuse a new freshclaude/kilroy pane's messages on the
real sidecar:
1. Before the CLI reports the session (system/init, which the real CLI
emits only once a prompt runs, T1) the coordinator holds no record for
the pane, so the first send was refused as SESSION_RESERVED ("A
lifecycle operation owns this session"). Nothing could ever run, so
init never came: every new pane's first message was lost.
2. After the report, the coordinator owns the session under its durable
id, but the fence was keyed on the pane's placeholder id, which stays
Vacant, so later sends were refused the same way.
The fakes hid both: they emit init at create, and the C04 rig left the
coordinator unwired. The live server (built 2026-09-20) predates the
fence (15598fb, 2026-10-16), so users have not hit it yet; origin/main
has it.
Fix: handle_send keys the fence on the session's durable id once the CLI
reported it (the rule handle_attach already follows: the map key is the
lane's own), and before the report, when the coordinator holds no record
for the pane, the send runs unfenced (there is no owner to fence; such a
session has no durable id to hand off, reattach or roll back). The
skipped fence is logged at debug as
freshagent.claude.send_before_session_report.
Tests: the claude rigs now wire one ownership coordinator into the WS
state and the claude lane, as main.rs does. New
freshagent_claude_real_cli.rs {sim,cli}::
first_checkpointed_send_of_a_new_pane_runs sends two messages the way
the browser does (checkpoint intent; the observed owner pair once a
session.runtimeOwner frame named the durable id): both run and ring once
each. Before the fix the first send was refused on both rigs, including
the pinned real CLI; with only the pre-report skip, the second was.
Deviation: this is outside the planned C01-C31 sequence. The C05 e2e Stop
lane on the real sidecar cannot send at all without it. C23 (durable id
minted at create, ownership claimed at create) supersedes the pre-report
branch.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The pinned CLI ends a stopped turn with error_during_execution and an
`[ede_diagnostic] ...` error (T11). The sidecar turned every non-success
result into sdk.error{turnFailure} BEFORE it asked the turn gate whether
the result was the user's own Stop, so every Stop showed a red "Agent
error: [ede_diagnostic] result_type=user ..." banner.
- New crates/freshell-claude-sidecar/sdk-message-frames.mjs (design §3.2,
§9.1 N6; transcript B1 has not landed, so core creates it): the pure
result mapping. resultFrames(sessionId, result, {userStopped}) gives
sdk.result, then sdk.error{turnFailure} only for a failed turn that is
not the user's Stop. resultErrorText (moved from session-settings.mjs,
where it was resultErrorMessage) never shows `[ede_diagnostic]` lines;
when nothing else is left it explains the result kind, and an aborted
turn nobody stopped reads "Claude stopped the turn before it
finished." (§5.3 copy).
- index.mjs `case 'result'`: the gate's verdict is taken first and passed
to resultFrames; frame order is unchanged (sdk.result, sdk.error?,
sdk.turn.complete).
Tests:
- test/unit/claude-sidecar/sdk-message-frames.test.ts (pure): diagnostics
stripped, per-kind copy, the abort copy, success and API-error results
have no text, and the frames for a user Stop, a failure and an abort
nobody stopped. The old resultErrorMessage case in
session-settings.test.ts moved here with its function.
- interrupt-idle-gate.test.ts (real index.mjs over the simulator): a Stop
during streamed text (HANG, aborted_streaming) and during a tool call
(aborted_tools) ends silently with no sdk.error. Red before: each
emitted sdk.error with the diagnostic text.
- session-settings.test.ts: the failing-turn program now puts a
diagnostic line next to the real error; the banner text is exactly the
real error. Red before.
- e2e fresh-agent-error-turn-attention.spec.ts: the Stop lane moved off
the fake sidecar to the production index.mjs over the CLI simulator,
for freshclaude and kilroy. A watched Stop (HANG; Stop is clicked once
the simulator's ledger shows the prompt started) rings nothing, marks
nothing, shows no "Agent error" banner, and no client frame carries
`ede_diagnostic`; the next turn, in the background, still rings with
attention. Red before on both flavours (the banner appeared). The
fake-based test keeps its other lanes, renumbered.
Deviations: the Stop lane needs no FRESHELL_FAKE wire audit; it reads
the simulator ledger instead. A new pane on the real sidecar is
`connected` (not `idle`) until its first turn, since the CLI reports
nothing before a prompt (T1); the lane waits for either. Kilroy needs
KILROY_ENABLED=1 to appear in the picker. The lane depends on 827d939
(a new pane's checkpointed sends were refused on the real sidecar).
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The turn-complete gate silenced whatever result came next after a Stop was armed. A Stop sent while the CLI is still starting can miss its turn: the receipt comes back ok (either shape, T13) and the turn then runs to success/completed, so its bell, tab mark and sidebar mark were swallowed while the user believed the turn had stopped. A mark armed for a send the CLI absorbed into an earlier turn (T10) likewise swallowed the next turn's bell after an idle Stop. - turn-complete-gate.mjs: resultEmitsAttention(result) takes the SDK result. The mark is consumed by the next result either way, and silences it only when the result was aborted (terminal_reason aborted_streaming/aborted_tools via sdk-message-frames.mjs isAbortedResult; error_during_execution on a CLI that reports no terminal_reason). Interim: the Rust Stop latch replaces it in C18. - index.mjs: the result case passes the result. consumeStream's finally no longer consults the gate: a result-less turn end is never the stopped turn's aborted result, so a pending mark does not silence it (matching the final design, §5.5: a death rings if work was owed). - The two e2e fixtures that import the real gate (fake-claude-sdk-sidecar, fake-kilroy-runtime) pass the CLI's stopped-result shape (T11) and their ordinary results; the fake's stream-error lane mirrors the real stream end. - AGENTS.md: the freshclaude/kilroy gate sentence states the new rule (C31 rewrites the section). Tests: - test/unit/sidecar/claude-turn-complete-gate.test.ts now passes results: a missed Stop's success/completed result rings and leaves no mark; a failed turn under a missed Stop rings; an aborted result nobody stopped rings; both aborted shapes are silenced under a mark; the older-CLI fallback; rejected interrupts and the no-reset-at-send ordering keep their cases. Three were red before. - interrupt-idle-gate.test.ts (real index.mjs over the simulator): a cold Stop the CLI never applied (send and Stop in one stdin chunk before the simulated CLI booted; receipt shapes `empty` and `queued`) completes with success and rings; a stale mark (absorbed send, then an idle Stop) never swallows the next turn's bell; a cold Stop followed by a CLI crash mid-turn (no result, T14) rings. All four were red before (each timed out waiting for the ring). Deviation: the result-less-end change (finally block) is part of "silences only aborted results"; the plan's C06 row names only the result case. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The claude sidecar's stderr used to be read into a buffer and thrown away, every line it wrote was stamped "warn" (even "ready"), and the attention edges the sidecar minted were forwarded without a server log line. Now: Sidecar (crates/freshell-claude-sidecar/log.mjs): log(severity, event, fields) writes one JSON record per stderr line with a real severity. sidecar.ready is info; configure.failed, interrupt.failed, respond.unknown/respond.invalid, request.unparseable (byte count only, since a request line can carry prompt text), request.unknown_type, commands.unavailable and create.failed are warn; stream.consume_failed is error; stream.ended is warn for an error end (with the error) and info otherwise; CLI stderr is cli.stderr at info, cut to 2 KiB, and by byte count only in hosted mode. The model catalog probe uses the same module. Rust (claude_sidecar_io.rs): spawn_stderr_logger replaces drain_reader. It re-emits each record as freshagent.claude.sidecar_log at the record's severity with ownership_id, pid, session_type and, once the sidecar has named its session, session_id. A non-JSON line is a warn with raw=true, its text cut to 4 KiB (byte count only when hosted). Lines are read up to 16 KiB (the rest is counted), and at most 50 lines a second are logged per sidecar; the dropped count is logged as freshagent.claude.sidecar_log_dropped when its window closes. Edges: both existing edge sites (the forwarded sidecar turn.complete / turn.waiting, and the consumer's death mint) go through one FreshClaudeState::emit_attention_edge shim (design I-1; C18's hub replaces it), which logs freshagent.turn.complete / freshagent.turn.waiting at info with provider, session_id, session_type, source and at, then broadcasts the unchanged frame. Tests: log.mjs unit tests and real-sidecar tests (ready at info, every stderr line a record, refusals and stream ends logged with ids, prompt text never logged); Rust logger tests (severities, labels, raw lines, cut lines, hosted redaction, rate limit and its summary); a spawned sidecar's stderr is logged and redacted when hosted; create, attach resume and rollback respawn bind the session; the shim logs and broadcasts each edge kind; WS on RealSidecarSim and RealSidecarCli: a sidecar record reaches the server log with the session labels, and a forwarded turn.complete (and, on the simulator, turn.waiting) is logged. Deviations from the design (§5.9): - The re-emitted event keeps one message (freshagent.claude.sidecar_log) and carries the sidecar's event as sidecar_event, because tracing field names are static. - boot.ok/boot.failed arrive with C10's boot handshake; until then the sidecar logs create.failed. stream.ended is logged now (C08 adds the sdk.stream.ended frame). - New freshagent.claude.sidecar_bound (info) logs the join between a sidecar's ownership id and its session, since the early records (sidecar.ready) come before the session exists. - Hosted means FRESHELL_HOSTED_FRESH_AGENT is set to anything (claude or kilroy); the sidecar used to redact only for "claude", so a hosted kilroy sidecar's CLI output would have reached the server log once Rust stopped discarding stderr. The sidecar's other hosted checks (exec-profile.mjs, hosted-mcp.mjs) still compare to "claude"; that pre-existing gap is left alone. - A forwarded edge without a numeric `at` is now dropped with a warning (freshagent.claude.attention_edge_dropped_malformed_at); the client already dropped such frames. - The catalog relocation test copies every sidecar module (the catalog helper now imports log.mjs). Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The Stop gate's fallback for CLIs that report no terminal_reason treated every error_during_execution result as a stopped turn. The pinned CLI's boot refusal (T5: missing resume, bad fork point, drops-turn guard) has that same shape, so a Stop sent while the CLI was still starting silenced the refusal: no bell for a turn that never ran, and C05's banner rule dropped the refusal text. The fallback now also requires num_turns > 0. A boot refusal reports num_turns 0; every aborted result the probes recorded reports 1 or more. The design's §5.3 verdict line carries the same rule for C18. Tests: a real-sidecar harness case (cold Stop during a refused resume boot rings and shows "No conversation found ..."), a pure gate case for the refusal shape, and the older-CLI fallback fixture now carries the num_turns a real stopped turn reports. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…ion_edge (F2) The stdout consumer routes sdk.turn.complete / sdk.turn.waiting to emit_attention_edge before it calls sdk_line_to_frame, so the two edge arms of normalize_sdk_type were unused in production, while three unit tests still asserted that old mapping and would keep passing if the real edge exit broke. normalize_sdk_type no longer maps the edges, so sdk_line_to_frame refuses them. The tests now pin that: the table test expects None, a rewritten edge test checks AttentionEdge::from_sidecar_type classifies both lines while the plain mapping refuses them, and the idle-death test models the consumer's per-line split. The frame shape stays pinned by emit_attention_edge_logs_each_edge_and_broadcasts_its_frame. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…aged key's value (F3) The C03 port of the settings tests checked less than the originals: - the refused setting (now permissionMode, which the real CLI refuses) was never checked to keep its previous value, so a sidecar that reported or stored 'no-such-mode' before setPermissionMode resolved would pass; - the managed OneCLI key was checked by key name only, so a sidecar that handed the SDK child a different or empty ANTHROPIC_API_KEY would pass. Both refusal cases now create with an explicit permission mode and assert it in the sdk.configured answer and in the next prompt's ledger row. The simulator's query ledger row now carries envDigests: a short sha256 digest of each allow-listed credential (ANTHROPIC_API_KEY, ANTHROPIC_AUTH_TOKEN, CLAUDE_CODE_OAUTH_TOKEN), never the value, and the managed-key test compares it to the digest of the configured values. Verified by mutation: storing the mode before setPermissionMode resolves fails both refusal tests; passing an empty managed key fails the OneCLI test. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…report (F4) 827d939 let a new pane's first checkpointed send run unfenced while the CLI had not reported the session and the coordinator held no record for it. That branch also skipped a send that DID carry an observed owner pair, running it with no ownership guard and only a debug trace, while the comment above the guard still said every supplied fence is honored. A supplied pair is now always checked: before the report it is checked against the pane's wire key, where the coordinator holds no record, so the pair is stale and the guard refuses it typed (SESSION_RESERVED). The unfenced exception is narrowed to what 827d939 needed: no report, no record, no pair (a new pane's first message). The rule that turns a session into its ownership key (the reported durable id, else the wire id, both through the alias walk) now lives in one helper, claude_owner_key, used by send and by the kill path's observed-owner fallback. C23 (durable id at create) removes its wire-id fallback. Test: a_fenced_send_before_the_session_report_is_checked_not_skipped (a fenced send before the report is refused and never reaches the sidecar; the unfenced first send still runs). Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…issions (F5)
A program `approval` step asked canUseTool in every permission mode,
although the simulator's own needsPermission rule says the pinned CLI
asks for no tool call under bypassPermissions (only AskUserQuestion asks
there). A kilroy test (kilroy runs in bypass) that got its approval card
or waiting edge from an `approval` step would pass on the simulator while
the real CLI never parks canUseTool there; design §11.6 uses a question
under bypass for kilroy for exactly that reason.
An `approval` step under bypassPermissions now fails the stream loudly
("Claude CLI simulator failure: ... bypassPermissions ..."), so the
mismatch cannot hide behind a green simulator run. The two simulator
tests that scripted an approval under bypass by default now run in
default mode; the program docs say what the step does under bypass.
Test: claude-cli-sim.test.ts "an approval step under bypassPermissions
fails loudly instead of asking".
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The sidecar had two disagreeing definitions of "hosted": log.mjs counted any FRESHELL_HOSTED_FRESH_AGENT value (claude or kilroy), while exec-profile.mjs and hosted-mcp.mjs compared it to 'claude' inline. Readers could not tell which rule was intended. hosted.mjs is now the one reader of the variable. It names the hosted provider and the two rules that read it, which differ on purpose: - isHostedSidecar: every hosted sidecar (freshclaude or kilroy) logs CLI output by size only; - usesHostedClaudeRuntime: the managed spawn check and the fixed hosted MCP recipe apply to hosted claude alone. log.mjs, exec-profile.mjs, hosted-mcp.mjs, index.mjs and model-catalog.mjs use them. Behavior is unchanged. Deviation from the review's suggestion (one predicate at all three sites): a single predicate would either stop redacting hosted kilroy's CLI output or silently give hosted kilroy the managed spawn and MCP recipe. Extending those to kilroy stays an explicit item for the api-hosted-ops workstream (its plan §2.4 already replaces hosted-mcp.mjs with mcp-servers.mjs); hosted.test.ts pins today's split so that change has to be made on purpose. The managed-provider qualification test (freshell-server, managed-provider-qualification feature) stages sidecar modules by hand; it now stages hosted.mjs too (compile-checked; it needs the provider sandbox to run). Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…s (F6) Records, in the api-hosted-ops plan §2.4, that hosted kilroy currently gets neither the managed spawn check nor the fixed hosted MCP recipe (hosted.mjs usesHostedClaudeRuntime is claude-only), and that the workstream decides and changes it red-first. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…everity (F7) C07 moved model-catalog.mjs onto log.mjs, so the catalog helper writes JSONL records with a severity, but the Rust probe still read its stderr as one blob: dropped on success (cli.stderr info records, catalog.close_failed warnings), and on failure logged as a single `stderr = <everything>` field of one warning, with no bounds. The probe now hands the helper's stderr to the session sidecar's spawn_stderr_logger, labelled ownership_id `claude-model-catalog` (no session type: the helper serves no session). Each record is re-emitted at its own severity with the logger's line cap, raw-line handling, hosted redaction and rate limit. The probe waits (bounded, 1 s) for the logger after the process ends, then logs one freshagent.claude.model_catalog_unavailable warning on failure or timeout, with the exit status or error and a pointer to those records. spawn_stderr_logger returns its task handle for that wait. Tests: claude_catalog_probe_re_emits_its_log_records_at_their_severity (cli.stderr at info and catalog.close_failed at warn, on success) and claude_catalog_probe_failure_keeps_its_records_apart_from_the_warning (catalog.unavailable at error; one warning with the status and no stderr field). The relocated-sidecar setup is shared by a helper. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The contract file and the conformance file each kept their own
in-memory scenarioCache('real'), and Vitest runs test files in separate
workers, so every real-CLI scenario (the 30 s+ heartbeat included) ran
twice per run, on two real CLIs at once against loopback providers. The
C07 cloud log showed 57.8 s and 70.2 s for the two files.
The conformance suite is now a module (claude-cli-conformance.ts,
describeSimulatorConformance) that claude-cli-contract.test.ts runs at
its end with its own real-trace cache. The T1-T19 `it` names and every
conformance `it` name are unchanged. Comments that named the old file
and the design's §11.4 now point at the module.
Deviation: the review's other option, a Vitest globalSetup that records
the traces once, would run in every cloud shard and every focused run,
whether or not it holds these files.
Verified (local backend, temporary run counter, not committed): one
file, 39 tests passed in 99 s, every real scenario ran exactly once
(the simulator's cold-stop runs twice: once per receipt knob, as
before).
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Brings PR-A (freshclaude core runtime: C01-C07 plus review fixes) up to main at fc1024c (#852-#862). Non-trivial conflict resolutions: 1. handle_send's ownership fence (main #854 vs PR-A 827d939/bdfe2658d). One implementation now serves both test suites. PR-A's `claude_owner_key` (shared by kill and send) and `send_owner_key` stay: a reported session is fenced on its durable id, and a supplied observed owner pair is always checked, even before the session is reported (F4). Main's `fence_owner_key` is dropped as a duplicate of that rule. From main's version, handle_send keeps the `self.ownership.is_some()` gate (a lane without a coordinator, the session host's, has nothing to fence and never logs `send_before_session_report`) and the `owner_key` field on the refusal warning. Main's three checkpointed_send tests and PR-A's a_fenced_send_before_the_session_report_is_checked_not_skipped all stay and pass. 2. Sidecar spawn (main #859 ClaudeNodeCommand/SidecarTreeGuard vs PR-A C04 with_sidecar_environment and C07 stderr logging). spawn_sidecar now builds through main's `sidecar_node_command` + `spawn_guarded`, and `SpawnedSidecar` carries main's `tree_guard` next to PR-A's `log_labels`. PR-A's test-support environment moves into the launch recipe as `ClaudeNodeCommand::with_complete_environment`: the configured environment replaces the inherited one, and the ownership tag, OneCLI key flag and managed secrets still layer over it, so each rig sidecar still gets exactly its own environment (new claude_spawn unit tests cover the layering and that nothing is inherited). PR-A's stderr logger replaces main's drain_reader. Two PR-A spawn tests now hold the tree guard until the child is reaped (dropping the guard kills the tree), and main's teardown test uses the SpawnedSidecar struct. 3. Model-catalog probe (main #859 tree guard + timeout field vs PR-A F7 structured logs). The probe spawns through main's catalog_node_command + spawn_guarded and kills the whole tree on timeout; its stderr goes through PR-A's sidecar logger. The timeout's tree confirmation is folded into the single `freshagent.claude.model_catalog_unavailable` warning instead of a second warning. Tests: one staging helper (`stage_catalog_sidecar`, copies every sidecar module, so sdk-env.mjs, log.mjs and hosted.mjs are all present) serves PR-A's F7 tests and main's OneCLI-key and timeout tests. 4. Sidecar JS (main #859 sdk-env.mjs + stdout-error shutdown vs PR-A log.mjs/hosted.mjs). index.mjs and model-catalog.mjs import all three modules; both use main's createClaudeSdkCleanEnv (so the probe now keeps the managed OneCLI key like the session sidecar). Main's stdout-error handler logs through PR-A's structured log as `stdout.closed` (warn, as before) since `logerr` no longer exists. 5. AGENTS.md Agent Status Indicators: main's ring-queue watchdog text and PR-A's Stop-gate (aborted results only) text are both kept. 6. claude_sidecar_interrupt_dispatch.rs: PR-A's stage-every-module helper is kept; it already covers main's added sdk-env.mjs. Main's new streaming-query-module.mjs fixture (stdout-loss test) is kept as is: it runs the real index.mjs through the SDK seam; moving it onto the CLI simulator belongs with C30. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…rn's idle The fork conformance case failed on cloud Vitest after the main merge: the real CLI's trace ended at the fork turn's result while the simulator's also had `lifecycle u3 completed` and `state idle`. The scenario closed the fork session right after the result (plus a transcript poll that can return at once), and the normalizer drops every frame that arrives after a close, so whether the turn's last two frames made it into the real trace depended on timing. The seed session had the same race on its second turn. Both sessions now wait for the turn's `session_state_changed idle` before closing, as the turns, interrupts and compact scenarios already do. The traces compared are unchanged when the race goes the slow way. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…ut reader goes away
The merge with main rewrote the lost-stdout handler's log line from main's
`logerr` to PR-A's structured `log('warn', 'stdout.closed', ...)`, but no test
read it: the stdout-loss test kept stderr only as a failure message. The
lost-stdout test now requires exactly one `stdout.closed` record at warn with
the EPIPE code and error text. Changing its severity to info makes it fail.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…nned-CLI helpers Main's sidecar shutdown contract (#859) and manual transcript capture tool (#860) each kept their own copy of the pinned SDK/CLI lookup that PR-A moved into claude-local-provider.mjs: a hard-coded 0.3.237/2.1.237 check and a linux-x64-only `.pnpm` store lookup. A pin bump would leave them failing separately, and on any other platform they threw a plain error instead of PinnedClaudeMissingError with its remediation hint. - claude-sidecar-shutdown.mjs now uses resolvePinnedClaude (which also runs the CLI's version check) and createClaudeFixture. Its sidecar gets the fixture's complete environment, including the dead HTTP(S)_PROXY that blocks any egress but the loopback provider, plus CLAUDE_CMD and the managed-key flag. The fixture now lives in the OS temp directory like the other contract fixtures, not in the repository root. The test checks the receipt against PINNED_SDK_VERSION/PINNED_CLI_VERSION. - claude-capture-fixtures.mjs uses resolvePinnedClaude for the SDK and CLI and gets the same dead proxy. It keeps its own `ws` workspace, because the committed fixtures and the durable-transcript e2e spec depend on the `/fixture/ws` path. Verified: the shutdown contract passes (both modes) before and after the change. A capture of tools-sync and api-error produces the same record type sequence as the committed fixtures, in both session.jsonl and sdk.jsonl. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Owner
Author
|
About the red Electron Build (windows-2022) job: it failed on one Rust test, This PR does not cause it:
That branch touches |
danshapiro
added a commit
that referenced
this pull request
Oct 23, 2026
…core-lifecycle PR-A moved two test commits past e128d37 before it landed as #863: the stdout-loss unit test now pins the `stdout.closed` warn record, and the pinned-CLI shutdown contract uses the shared resolvePinnedClaude / createClaudeFixture helpers and FAKE_MODEL. Conflict: test/integration/providers/claude-sidecar-shutdown.mjs, the create line. Resolved to PR-B's protocol-v2 create (protocol 2, a fresh boot) with PR-A's FAKE_MODEL. The sidecar's merged stdout-error handler logs `stdout.closed` once at warn before anything else, so the new record assertion holds unchanged. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
This was referenced Oct 23, 2026
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
This is core PR-A of the freshclaude core-runtime redesign (design:
docs/plans/2026-10-22-freshclaude-core-runtime.md, section 10). It fixes three bugs in how freshclaude and kilroy panes handle Stop. It also gives the Claude sidecar real log lines, and puts the tests on facts measured from the real Claude CLI instead of hand-written fakes.What users get
[ede_diagnostic] ...line. Every Stop used to show that as "Agent error: ...". A Stop now ends the turn quietly.What changed
test/integration/providers/claude-cli-contract.test.tshas one named test per CLI fact (T1-T19 in the design). It runs the pinned SDK 0.3.237 and its bundled CLI 2.1.237 against a local fake Anthropic endpoint (claude-local-provider.mjs), with a throwaway home folder, a dummy key and a dead proxy for any other traffic. The real Anthropic API is never contacted. A missing pinned CLI fails the suite; nothing is skipped.test/fixtures/claude-cli-sim/is a drop-in replacement for the SDK'squery()that behaves like the pinned CLI. The conformance tests run every scenario on both the real CLI and the simulator and require the same trace from each. The simulator also writes nothing after its stream ends, refuses an unknown permission mode the way the CLI does, and refuses an approval step underbypassPermissions.index.mjsharness.test/unit/claude-sidecar/harness.tsruns the real sidecar over the simulator. The interrupt, message-replay, permission, rollback and settings tests now use it, and four hand-written fake SDK modules are deleted. The checkpoint-identity logic moved into its own module (checkpoint-identity.mjs) so its edge cases are tested directly.crates/freshell-ws/tests/common/claude_rig.rsruns a real freshell-ws server whose Claude sessions use the real sidecar, in two versions: over the simulator (RealSidecarSim) or over the pinned real CLI with the local fake endpoint (RealSidecarCli). Each rig sidecar gets exactly its own environment through a test-only setting; production never sets it.sdk-message-frames.mjsmaps a CLI result to the messages sent to the browser. It checks whether the user stopped the turn first, and only a failed turn the user did not stop gets an error. Error text never shows[ede_diagnostic]lines.log.mjsin the sidecar writes one JSON record per stderr line, with severity, event name and ids. Prompt text is never logged; hosted sidecars log the CLI's stderr by size only. On the Rust side,claude_sidecar_io.rsre-emits each record at its own severity, with line-size limits and a 50-lines-a-second cap. The model-catalog probe uses the same logger. Attention edges go out through one function,emit_attention_edge, which logs each one.hosted.mjsmodule for "is this sidecar hosted"; the model-catalog log records keep their severity; each real-CLI scenario runs once per test run instead of twice.Merge with main
Main moved a lot while this branch was open (#852 through #862). The merge commit (
80cee6877) explains each conflict. In short:owner_keyfield on the refusal warning. Main's tests (freshagent_claude_checkpointed_send.rs) and this branch's tests all pass.ClaudeNodeCommand::with_complete_environment), with two new unit tests.sdk-env.mjsand main's stdout-loss shutdown. Main's shutdown handler now writes a structuredstdout.closedrecord at warn.After the merge:
e128d37c4: the real-CLIforkscenario now waits for the turn to go idle before closing, which fixes a timing-dependent cloud failure.a36b9a48e: a test now checks thestdout.closedwarn record.1691c4ec9: main's sidecar-shutdown test and its manual transcript-capture tool now use the shared pinned-CLI lookup instead of their own copies. The old copies worked only on linux-x64 and would have needed a separate edit at the next version bump. The shutdown test's sidecar also gets the dead proxy now.How the tests changed
index.mjsharness: sidecar unit tests run the production sidecar file instead of fake SDK scripts.Test results
All on the final head unless noted.
cargo fmt --check-D warnings(workspace, excluding freshell-tauri)cargo test -p freshell-freshagentfreshagent_claude_real_cli(freshell-ws)freshagent_claude_checkpointed_send(freshell-ws)cargo test -p freshell-session-hostpnpm run typechecktest/unit/claude-sidecar/,test/unit/sidecar/,test/integration/providers/The Rust rows were measured on the merged tree. Every commit after the merge changes only Vitest test files.
The e2e specs: fresh-agent-error-turn-attention, fresh-agent-control-rust, fresh-agent-rollback-rust, freshclaude-first-send-rust, freshclaude-identity-persistence-rust, freshclaude-restart-parity-rust, freshclaude-durable-transcript-rust, fresh-agent-mobile-attention, harness-03-provider-fixtures, harness-06-misc-fixtures.
Note for bisecting
Commits
a0a5634c8andef0527f8cfail one Rust test (claude_sidecar_interrupt_dispatch): it copied the sidecar's modules by name and missed the newcheckpoint-identity.mjs.254df0c02fixes it. Skip those two commits when bisecting.Known timing-dependent tests outside this PR
The pre-push hook's workspace run failed four times on four different tests in code this PR does not change (identical to
main). Each passed when run alone or with fewer threads:freshell-sessionssession_watcher::tests::metadata_late_subagent_named_mkdir_arms_via_deferred_recheck_without_a_discoverfreshell-supervisorprovider_accepted_completion_retries_after_registry_success_on_fresh_healthy_startupfreshell-servernet_bind::tests::rebind_serves_a_connection_queued_on_the_retiring_listenerfreshell-freshagentsession_handoff::tests::opencode_handoff_stop_waits_for_a_mid_dispatch_rest_drive_under_the_gateThey need their own fixes in a separate PR.
🤖 Generated with Claude Code