fix: background shells end in the chat, with their exit code (BRA-3094)

.claude/skills/verify-berthd/features/README.md

@@ -23,6 +23,7 @@ The maintained list of what a person can do in the app, and how to prove each on
 | Chat and board: no question card or answers but the options the agent's screen shows; the mode menu while turn check waits; a structured agent's form while its helpers report back, and a reply box that answers a card with nothing to pick | [chat-ask-card.md](chat-ask-card.md) | `chat-ask-card`, `chat-structured-ask-live` (a real box, `--agent`); `app/e2e/ask-card.spec.ts` on a stand-in box |
 | The board's cards for agents in chat: what they ask with its answers, their reply's last lines, the prompt they started with; never the host's frames or "stdio" | [chat-board-cards.md](chat-board-cards.md) | `chat-board-cards`; `app/e2e/dashboard-chat-cards.spec.ts` asserts it |
 | Worktrees changed with git outside Shipyard: added, committed in, switched, locked, removed; a new remote; a quiet box runs no git for them | [worktree-git.md](worktree-git.md) | `worktree-git-live` (a real box, `--agent`); `TestListingLocationsAgainRunsNoGitUntilSomethingChanges` |
+| Chat's background work: the chip counts only what still runs; the panel shows each shell's `Done · exit 0`, `Failed · exit N` or `Stopped` and a duration that stops growing, with its output | [chat-background-shells.md](chat-background-shells.md) | `chat-background-shells`; `chat-background-shells-live` (a real box, `--agent`); `app/src/lib/background-work.test.ts`, `TestABackgroundShellEndsWhenTheStreamSaysSo` |
 | Turn check's "working": an hourglass with why in the sidebar, the tab and the rail's card, not a live turn's spinner; a handoff reads as done | [turn-check-indicator.md](turn-check-indicator.md) | `turn-check-indicator`; `app/e2e/sidebar-turn-check.spec.ts` asserts it, `TestTurnCheckLive` asks the real provider |
 | Upgrading a box: sessions keep running, and nothing the old build started stays a zombie under berthd | [box-upgrade.md](box-upgrade.md) | `../scripts/upgrade-zombies.sh FROM TO` (a real box, Linux); `go test ./internal/childreap` |
 

.claude/skills/verify-berthd/features/chat-background-shells.md

@@ -0,0 +1,45 @@
+# Chat: work left running in the background
+
+Under the reply box, a chip says what the agent left running in the background (shells started with `run_in_background`, monitors): "1 shell running", "2 tasks running". It counts only what still runs. With nothing running it says how many ended in the last five minutes ("2 finished", a grey dot), then goes. Clicking it opens **Left running in the background by Claude**: a row per job with its label and command, a badge and how long it ran, newest first. Opening a row shows its output (the shell's `tasks/<id>.output`), followed while it runs.
+
+The badge is green **Running**, grey **Done · exit 0**, red **Failed · exit N** or **Stopped** (TaskStop, KillShell). Once a job ended, its duration stops growing. (`components/conversation/chat-controls.tsx` `BackgroundChip`, `JobRow`, `State`; `lib/background-work.ts`.)
+
+The box decides when a shell ended (`internal/transcript/signals.go`):
+
+- Claude's notification for it, in any of its shapes: a `queue-operation`, a `queued_command` attachment, a prompt (`<task-notification>…<status>…</status><summary>… (exit code N)</summary>`). A structured (chat) agent's log (berthd agent-host) says it in a `system`/`task_notification` line of its own, read as that notification (`host.go` `unwrapHost`). That log holds lines as Go's JSON writes them, with `<` escaped, so a check for the literal `<task-notification>` misses its prompts (`escapedNote`).
+- Else the shell's output file: Claude ends it with `[exited with code N]` as the last line when the shell exits. Each read of the conversation reads the end of each running shell's file (`settle`), so it shows within one poll (a few seconds) even when no transcript line comes. The time is the file's.
+
+A notification, when one comes, has the last word (a shell the file said ended can still read Stopped). A monitor is never read from a file.
+
+## Sub-features
+
+- `chip-counts-running`: the chip's count is of running jobs only; with none, "N finished" for five minutes.
+- `exit-badge`: `Done · exit 0`, `Failed · exit N`, `Stopped`; plain `Done`/`Failed` from a box too old to send `exit`.
+- `frozen-duration`: an ended job's duration stays put while the panel's clock ticks (every 15 s).
+- `structured-end`: a chat agent's shells end (they never did before: the stream's `task_notification` only reached the crew).
+- `output-file-end`: a shell ends from its output file's last line, before or without a notification.
+
+## How to get to it (user POV)
+
+- Open a worktree's agent in **Conversation** and ask it to run something in the background.
+- On the mock: `gpu/judge-v2` has the dev server running, the payment tests done (exit 0) and a lint that failed (exit 3).
+
+## Driving it with drive.mjs
+
+- On the mock: `scenarios/chat-background-shells.mjs` opens `gpu/judge-v2`, asserts the chip says "1 shell running", the three badges and durations, then waits past a clock tick and asserts the ended ones' durations didn't move.
+- On a real box: `scenarios/chat-background-shells-live.mjs` (`export const live = true`) on a dev box whose agent runs structured. Claude starts `sleep 20; echo hi` and `sleep 6; exit 3` in the background. It asserts Running, then `Failed · exit 3`, then `Done · exit 0` within ~45 s of the chip first showing both, the output `hi`, durations that stay put for 20 s, and "N finished" on the chip. Second view: the box's `transcript` signals (`state`, `exit`, `until`) through the agent (`call`). About 1.5 minutes.
+
+```sh
+VERIFY_BOX_DIR=/tmp/vb-$$ VERIFY_DRIVER=claude .claude/skills/verify-berthd/scripts/dev-box.sh up
+eval "$(VERIFY_BOX_DIR=/tmp/vb-$$ .claude/skills/verify-berthd/scripts/dev-box.sh env)"
+node .claude/skills/verify-berthd/scripts/drive.mjs chat-background-shells-live --port $PORT --agent http://127.0.0.1:9378
+VERIFY_BOX_DIR=/tmp/vb-$$ .claude/skills/verify-berthd/scripts/dev-box.sh down
+```
+
+For a before, `dev-box.sh restart <a main checkout>/bin/berthd` on a fresh dev box, then the same run: it fails at `Failed · exit 3` with both rows still Running.
+
+## Gotchas
+
+- Run each live pass on a fresh dev box (`down`, then `up`): the scenario finds rows by label, and a second pass in the same session has two of each.
+- A new structured agent opens on its first-task box ("What should Claude Code do?"), not the chat; the scenario types there.
+- From inside an agent session, unset every `BERTH_*` before `dev-box.sh`, or the session's own driver leaks into the dev box's agents.

.claude/skills/verify-berthd/scripts/scenarios/chat-background-shells-live.mjs

@@ -0,0 +1,100 @@
+// chat-background-shells-live: the chat's background-work chip and panel on
+// a real box (features/chat-background-shells.md). Claude Code starts two
+// shells in the background, one that says hi after 20 seconds and one that
+// exits with code 3. The panel shows the first Running, the second
+// `Failed · exit 3`, then the first `Done · exit 0` within seconds of its
+// exit, each with a duration that no longer grows, and the chip stops
+// saying "running". The box's own transcript signals say the same.
+//
+// On a dev box of this checkout, its agent structured (the way the app
+// starts agents, and where the bug was):
+//
+//   VERIFY_DRIVER=claude ../dev-box.sh up
+//   eval "$(../dev-box.sh env)"
+//   node drive.mjs chat-background-shells-live --port $PORT --agent http://127.0.0.1:9378
+//
+// Without VERIFY_DRIVER it drives Claude in its terminal, which reads its
+// own transcript file instead: the same assertions hold.
+
+export const live = true;
+export const params = { view: "conversation", sidebar: "list", labs: "0" };
+
+const WORKTREE = process.env.VERIFY_WORKTREE ?? "devq/acme-shop";
+const BOX = WORKTREE.split("/")[0];
+const SESSION = process.env.VERIFY_SESSION ?? "ask";
+const HI = "Say hi after 20 seconds";
+const BAD = "Exit with code 3";
+
+export async function run({ page, shot, step, expect, call }) {
+  const signals = async () => (await call(`/v1/boxes/${BOX}/api/sessions/${SESSION}/transcript`)).signals?.background ?? [];
+  const job = async (label) => (await signals()).find((j) => j.label === label) ?? {};
+  const pane = page.locator("[data-testid=pane]:visible").first();
+
+  step("open the agent's chat");
+  await page.locator(`[data-testid=worktree-row][data-worktree="${WORKTREE}"]`).click({ position: { x: 32, y: 12 } });
+  // A new agent opens on its first-task box; one with a conversation, on
+  // its chat.
+  let reply = pane.getByRole("textbox", { name: "What should Claude Code do?", exact: true });
+  try {
+    await expect(reply).toBeEditable({ timeout: 30_000 });
+  } catch {
+    reply = pane.getByTestId("composer").getByRole("textbox", { name: "Reply" });
+  }
+
+  step("ask for two shells in the background: one that says hi after 20 seconds, one that exits with code 3");
+  await reply.fill(
+    "In one message, call the Bash tool twice, both with run_in_background true: " +
+      `first command \`sleep 20; echo hi\` with description "${HI}", then command \`sleep 6; exit 3\` with description "${BAD}". ` +
+      "Don't wait for them and don't read their output: reply with the single word started and end your turn. When one finishes later, reply with one word.",
+  );
+  await reply.press("Enter");
+
+  step("the chip says both run");
+  const chip = pane.getByRole("button", { name: /^Background work: / });
+  await expect(chip).toHaveAccessibleName("Background work: 2 shells running", { timeout: 120_000 });
+  const started = Date.now();
+  await chip.click();
+  const row = (label) => page.getByRole("button", { name: new RegExp(label) });
+  const state = (label) => row(label).getByTestId("job-state");
+  const took = (label) => row(label).getByTestId("job-took");
+  await expect(state(HI)).toHaveText("Running");
+  await shot("1-both-running");
+
+  step("the one that exits with code 3 shows Failed · exit 3, the other still runs");
+  await expect(state(BAD)).toHaveText("Failed · exit 3", { timeout: 30_000 });
+  await expect(state(HI)).toHaveText("Running");
+  await expect(chip).toHaveAccessibleName("Background work: 1 shell running");
+  await shot("2-failed-exit-3");
+
+  step("within seconds of its exit, the other shows Done · exit 0");
+  await expect(state(HI)).toHaveText("Done · exit 0", { timeout: 60_000 });
+  const late = (Date.now() - started) / 1000;
+  step(`Done showed ${late.toFixed(0)}s after the chip first said both ran (the shell sleeps 20s)`);
+  if (late > 45) throw new Error(`Done came ${late.toFixed(0)}s after the shells started: more than ~10s after the exit`);
+  await expect(chip).toHaveAccessibleName(/^Background work: \d+ finished$/);
+  await shot("3-done-exit-0");
+
+  step("its output stays readable");
+  await row(HI).click();
+  await expect(page.locator("pre").filter({ hasText: /\bhi\b/ })).toBeVisible({ timeout: 15_000 });
+  await shot("4-done-output");
+
+  step("the durations no longer grow");
+  const before = [await took(HI).innerText(), await took(BAD).innerText()];
+  await page.waitForTimeout(20_000); // the chip's clock ticks every 15s
+  await expect(took(HI)).toHaveText(before[0]);
+  await expect(took(BAD)).toHaveText(before[1]);
+  step(`durations stay ${before.join(" and ")}`);
+
+  step("the box's transcript says the same (a second view)");
+  const hi = await job(HI);
+  const bad = await job(BAD);
+  if (hi.state !== "done" || hi.exit !== 0 || !hi.until) throw new Error(`box signals for "${HI}": ${JSON.stringify(hi)}`);
+  if (bad.state !== "failed" || bad.exit !== 3 || !bad.until) throw new Error(`box signals for "${BAD}": ${JSON.stringify(bad)}`);
+  step(`box: ${HI} ${hi.state} exit ${hi.exit}, ${BAD} ${bad.state} exit ${bad.exit}`);
+
+  step("closed, the chip no longer says running");
+  await page.keyboard.press("Escape");
+  await expect(chip).not.toHaveAccessibleName(/running/);
+  await shot("5-chip-finished");
+}

.claude/skills/verify-berthd/scripts/scenarios/chat-background-shells.mjs

@@ -0,0 +1,41 @@
+// The chat's background-work chip and its panel on the mock
+// (features/chat-background-shells.md):
+//
+//   node drive.mjs chat-background-shells --port $PORT
+//
+// gpu/judge-v2's agent left three shells: the dev server still running,
+// the payment tests done (exit 0) and a lint that failed (exit 3). The chip
+// counts only the running one; the panel shows each ended shell's exit code
+// and a duration that stays put.
+
+export async function run({ page, shot, step, expect }) {
+  await page.goto(page.url().replace(/\?.*/, "?mock=1&sidebar=list&view=conversation"));
+  await expect(page.getByTestId("nav-home")).toBeVisible();
+  const pane = page.locator("[data-testid=pane]:visible").first();
+
+  step("open gpu/judge-v2's chat");
+  await page.locator('[data-testid=worktree-row][data-worktree="gpu/judge-v2"]').click({ position: { x: 32, y: 12 } });
+  await expect(pane.getByTestId("chat")).toBeVisible();
+
+  step("the chip counts only the shell still running");
+  const chip = pane.getByRole("button", { name: /^Background work: / });
+  await expect(chip).toHaveAccessibleName("Background work: 1 shell running");
+  await chip.click();
+  const row = (label) => page.getByRole("button", { name: new RegExp(label) });
+  const ended = [
+    ["Watch the payment tests", "Done · exit 0", "3m"],
+    ["Lint the checkout package", "Failed · exit 3", "30s"],
+  ];
+  const durations = async () => {
+    for (const [label, , took] of ended) await expect(row(label).getByTestId("job-took")).toHaveText(took);
+  };
+  await expect(row("Start the dev server").getByTestId("job-state")).toHaveText("Running");
+  for (const [label, state] of ended) await expect(row(label).getByTestId("job-state")).toHaveText(state);
+  await durations();
+  await shot("panel");
+
+  step("the ended ones' durations stay put while the clock ticks");
+  await page.waitForTimeout(16_000); // the chip's clock ticks every 15s
+  await durations();
+  await shot("panel-later");
+}

app/src/components/conversation/chat-controls.tsx

@@ -31,6 +31,7 @@ import { toastManager } from "@/components/ui/toast";
 import { isMock } from "@/hooks/use-berth-connection";
 import { agentPresets } from "@/lib/actions";
 import type { LaunchFailure } from "@/lib/api";
+import { backgroundLabel, isRunning, jobBadge, jobTook } from "@/lib/background-work";
 import {
   type ChatJob,
   type ChatSignals,
@@ -494,28 +495,22 @@ function Ring({ pct }: { pct: number }) {
 
 // --- Background work ---
 
-const RECENT = 5 * 60_000;
-
 function BackgroundChip({ box, session, jobs, who }: { box: string; session: string; jobs?: ChatJob[]; who: string }) {
   const [now, setNow] = useState(Date.now());
   useEffect(() => {
     const t = window.setInterval(() => setNow(Date.now()), 15_000);
     return () => window.clearInterval(t);
   }, []);
   const list = jobs ?? [];
-  const running = list.filter((j) => j.state === "running" || j.state === "starting");
-  const recent = list.filter((j) => j.until && now - j.until < RECENT);
-  if (!running.length && !recent.length) return null;
-  const shells = running.filter((j) => j.kind === "shell").length;
-  const label = running.length
-    ? `${running.length} ${shells === running.length ? (running.length === 1 ? "shell" : "shells") : running.length === 1 ? "task" : "tasks"} running`
-    : `${recent.length} finished`;
+  const label = backgroundLabel(list, now);
+  if (!label) return null;
+  const running = list.some(isRunning);
   return (
     <Popover>
-      <PopoverTrigger render={<button type="button" className={cn(chip, running.length && "text-foreground/85")} aria-label={`Background work: ${label}`} />}>
+      <PopoverTrigger render={<button type="button" className={cn(chip, running && "text-foreground/85")} aria-label={`Background work: ${label}`} />}>
         <span className="relative flex size-2 items-center justify-center" aria-hidden>
-          {running.length > 0 && <span className="absolute inline-flex size-full animate-ping rounded-full bg-success/50" />}
-          <span className={cn("relative inline-flex size-1.5 rounded-full", running.length ? "bg-success" : "bg-muted-foreground/50")} />
+          {running && <span className="absolute inline-flex size-full animate-ping rounded-full bg-success/50" />}
+          <span className={cn("relative inline-flex size-1.5 rounded-full", running ? "bg-success" : "bg-muted-foreground/50")} />
         </span>
         {label}
       </PopoverTrigger>
@@ -535,7 +530,7 @@ function JobRow({ box, session, job, now }: { box: string; session: string; job:
   const [open, setOpen] = useState(false);
   const [detail, setDetail] = useState<ToolDetail>();
   const [error, setError] = useState<string>();
-  const live = job.state === "running" || job.state === "starting";
+  const live = isRunning(job);
   useEffect(() => {
     if (!open) return;
     let alive = true;
@@ -555,7 +550,7 @@ function JobRow({ box, session, job, now }: { box: string; session: string; job:
     if (pre.current) pre.current.scrollTop = pre.current.scrollHeight;
   }, [detail?.output]);
   const Icon = job.kind === "monitor" ? EyeIcon : TerminalIcon;
-  const took = fmtDur((job.until ?? now) - job.since);
+  const took = jobTook(job, now);
   return (
     <div className="rounded-md">
       <button type="button" aria-expanded={open} onClick={() => setOpen((o) => !o)} className="flex w-full min-w-0 items-center gap-2 rounded-md px-1.5 py-1.5 text-left outline-none hover:bg-accent focus-visible:ring-2 focus-visible:ring-ring">
@@ -565,7 +560,9 @@ function JobRow({ box, session, job, now }: { box: string; session: string; job:
           {job.label && <span className="block truncate font-mono text-mono text-muted-foreground">{job.command}</span>}
         </span>
         <State job={job} />
-        <span className="w-12 shrink-0 text-right text-muted-foreground text-meta tabular-nums">{took}</span>
+        <span className="w-12 shrink-0 text-right text-muted-foreground text-meta tabular-nums" data-testid="job-took">
+          {took}
+        </span>
         <ChevronDownIcon className={cn("size-3.5 shrink-0 text-muted-foreground transition-transform", open && "rotate-180")} />
       </button>
       {open && (
@@ -588,29 +585,19 @@ function JobRow({ box, session, job, now }: { box: string; session: string; job:
   );
 }
 
-function State({ job }: { job: ChatJob }) {
-  switch (job.state) {
-    case "running":
-    case "starting":
-      return <span className="shrink-0 rounded-full bg-success/12 px-1.5 py-px text-tag text-success-foreground">Running</span>;
-    case "done":
-      return (
-        <span className="flex shrink-0 items-center gap-1 text-tag text-muted-foreground">
-          <CheckIcon className="size-3 text-success" />
-          Done
-        </span>
-      );
-    case "failed":
-      return <span className="shrink-0 text-tag text-destructive-foreground">Failed</span>;
-    default:
-      return <span className="shrink-0 text-tag text-muted-foreground">Stopped</span>;
-  }
-}
+const badge = {
+  running: "rounded-full bg-success/12 px-1.5 py-px text-success-foreground",
+  done: "flex items-center gap-1 rounded-full bg-muted px-1.5 py-px text-muted-foreground",
+  failed: "rounded-full bg-destructive/12 px-1.5 py-px text-destructive-foreground",
+  stopped: "text-muted-foreground",
+};
 
-function fmtDur(ms: number) {
-  const s = Math.max(0, Math.round(ms / 1000));
-  if (s < 60) return `${s}s`;
-  const m = Math.floor(s / 60);
-  if (m < 60) return `${m}m`;
-  return `${Math.floor(m / 60)}h ${m % 60}m`;
+function State({ job }: { job: ChatJob }) {
+  const { text, tone } = jobBadge(job);
+  return (
+    <span data-testid="job-state" className={cn("shrink-0 text-tag", badge[tone])}>
+      {tone === "done" && <CheckIcon className="size-3" />}
+      {text}
+    </span>
+  );
 }