Skip to content

Show a starting computer calmly, and tell a Bot to try again when a cold start runs long - #842

Closed
guidovizoso wants to merge 17 commits into
mainfrom
guido/calm-computer-start
Closed

guidovizoso wants to merge 17 commits into
mainfrom
guido/calm-computer-start

Conversation

@guidovizoso

Copy link
Copy Markdown
Collaborator

What this changes

A Bot's computer takes about a minute to start the first time (an image pull, a pod, a browser), and everything around that minute read as a fault. The Computer panel opened on "Can't show the screen right now… ask an admin", an action that waited on the start was marked "didn't work" in amber, and the model was handed something that looked like a broken computer and gave up on one seconds from ready. Looking at the panel was also enough to wake a sleeping computer.

The panel

  • Says what is happening: "Starting Sales's computer. The first start takes about a minute." or "Sales's computer is off. It starts when the Bot next uses it." A computer that is ready but has not sent a frame yet keeps "Waiting for the Bot's screen…".
  • Shows a problem only when the server says the computer can't be reached, or after 150 seconds without a picture. Before that, a failed screenshot is far more often a browser still launching.
  • A dropped live screen being retried shows a small "Reconnecting…" badge instead of the Retry panel, which is kept for when the retries run out.
  • A frame from before a computer went to sleep is dropped, so a sleeping computer isn't shown as live.

Looking no longer wakes it. GET /api/computers/:botId/screenshot asks the provider's status first, which touches nothing, and answers 200 with { ready: false, state } for a computer that is not ready. Only a ready one is screenshotted. The panel also polls the status route until the first frame.

A cold start that runs long is a wait, not a failure

  • New ComputerStillStartingError (a ComputerUnavailableError, so existing catches still apply), thrown only before anything reached the computer. Its message tells the model the call was not performed and to try again in a moment.
  • The headless tools answer it as a result (starting: true, notPerformed: true) instead of throwing; the routes mark the error body the same way; the browser draws the line as "…, not done yet".
  • Hosted (sandbox) computers: locate answers within 45 seconds instead of 120, under a load balancer's usual 60-second idle timeout. Behind an ALB the browser used to get a bare gateway error while the action could still land once the pod came up. The resume keeps going; the next call waits again.
  • Docker computers: the supervisor marks its own health-wait 503 as starting, and passes Docker's healthcheck verdict through its list, so a container that is running but still booting its browser reads as starting rather than ready.
  • The gateway carries a still-starting locate through to the attempt instead of forgetting it, so one click on a sleeping computer no longer waits through two cold starts.

Saying what it is about to do. The computer guidance gains one line: before its first computer action in a turn, the Bot says in one sentence what it is about to do and where, so the wait has a reason on screen.

Where it runs

  • New state that outlives a request? None on the server. The panel's grace timer and polled state live in the browser component. The sandbox provider's existing per-Bot restart map is unchanged.
  • What happens on the second replica? The same answer. The status route reads the provider's own record (the Sandbox resource, or the supervisor's container list), never in-process state, so a poll landing on any process gets the same state. A resume written by one process's locate is picked up by the next call wherever it lands.
  • Anything serialised? Nothing new. There is a small gap between the status read and the screenshot on the screenshot route: a computer suspended exactly inside it is woken by the screenshot. The cost is one computer started early, never a wrong frame.
  • Anything fanned out to a browser? No. The panel polls.
  • New listener, port, or schedule? No. The supervisor's existing /ensure and list answers gain a starting flag and a health field. Server and supervisor can be upgraded in either order: an older supervisor sends no verdict and reads as before, and an older server ignores the new fields.

Boundary and audit

  • Every acting call still goes through the gateway. The still-starting error is thrown inside the attempt, after the decision.
  • New refusals and new failures each write a row. An action turned away by a cold start is thrown inside the block that writes the failure row beside the decision, so the trail shows a permitted action that was not performed. The not-ready screenshot answer is not a failure and writes nothing, as a screenshot never did.
  • Nothing new is trusted from the client that the server can resolve itself. The computer's state is the provider's, read on the server.

Changelog

  • "A computer that is starting says so, instead of showing a problem" under Unreleased, including the supervisor note.

Merge notes

The search-routing PR removes the COMPUTER_GUIDANCE export in favour of computerGuidance({ search }), and splits the guidance into line lists; the line this PR adds belongs in its ACTING_LINES. The new test in shared/bot-prompt.test.ts here references COMPUTER_GUIDANCE, so whichever of the two merges second has to switch that test to computerGuidance, checking both { search: true } and { search: false }.

How it was checked

  • New tests:
    • app/tests/computer-calm-start.test.tsx: a starting computer says so with no problem shown; failed screenshots during a start are not a problem until the grace runs out; an unreachable computer is a problem at once; an off computer in the side panel says it starts when the Bot uses it; a dropped stream shows the badge, not the Retry panel; a computer that was off and then fails is not called off for ever; a computer that goes to sleep stops showing its last frame.
    • app/tests/computer-tool-refusals.test.ts and screen-connection.test.ts: a still-starting call reads as not done yet; a retried drop reports reconnecting and no problem until the retries run out.
    • server/tests/computer-routes.test.ts: the not-ready screenshot answer, including an unreachable computer's reason; an action whose cold start ran out is marked starting and not performed.
    • server/tests/computer-sandbox.test.ts, computer-supervisor.test.ts, headless-tools.test.ts, supervisor/tests/health.test.ts: the 45-second locate, the supervisor's timeout and 503 read as starting while its other faults stay faults, the healthcheck verdict, and the headless answer.
    • shared/bot-prompt.test.ts: the one-sentence line.
  • In a running app on a combined branch of this and five sibling PRs (local stack, 2026-10-26): "open news.ycombinator.com". The Bot said in one sentence what it was about to do, the panel stayed calm ("Waiting for the Bot's screen…") while the computer came up, and no error was shown.
  • The full server suite on the combined branch: 5,821 pass. The only failures were the assertion-shape guard, since fixed here (the allow-list follows the moved computer-supervisor and computer-sandbox lines), and one Mastra test that needs a package not installed locally. The app suite: 1,675 pass.

…ealth wait runs out

The supervisor answers a container that is up but not yet healthy with a 503 after 60s, which is
well inside the server's 120s ensure timeout, so the docker and desktop paths never reached the
gentle still-starting answer. The supervisor now marks that 503 as starting and the provider maps
it to ComputerStillStartingError. Also awaits a rejects assertion that was never checked.
…not ready

Docker says running as soon as the process exists, so status called a booting container ready, the
screenshot route went on to /ensure, and the poll was held for up to a minute on the health wait.
The supervisor now passes the healthcheck verdict through its list, and running with a starting
healthcheck reads as starting. An older supervisor sends no verdict and reads as before.
…timeout

locate waited up to 120s inside the person's request. Behind an ALB's default 60s idle timeout
the browser got a bare gateway error, and the action could still land once the pod came up after
the model had been told it failed. locate now gives up at 45s with the still-starting answer while
the pod keeps coming up; update's background restart keeps the full resume budget.
A screenshot failure after an 'absent' answer kept the old state, so the side panel went on
calling a failing computer off, and an off computer is never timed out, so the failure was never
shown. A failure now clears the polled state and the status read resumes. A not-ready answer also
drops the last frame, which showed a sleeping computer as live and kept its stream reconnecting.
@guidovizoso

Copy link
Copy Markdown
Collaborator Author

Landed in #847.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant