Skip to content

test(sandbox): exercise a sandbox session end to end - #403

Open
ItamarZand88 wants to merge 1 commit into
itamar/alien-75-sandbox-6-typescriptfrom
itamar/alien-75-sandbox-7-e2e
Open

test(sandbox): exercise a sandbox session end to end#403
ItamarZand88 wants to merge 1 commit into
itamar/alien-75-sandbox-6-typescriptfrom
itamar/alien-75-sandbox-7-e2e

Conversation

@ItamarZand88

@ItamarZand88 ItamarZand88 commented Aug 11, 2026

Copy link
Copy Markdown
Contributor

Summary

Exercises a sandbox end to end: the test app declares one, and the suite drives a real session through it rather than asserting on rendered templates.

What the test does when it runs:

  1. Deploys the comprehensive test app, which now declares a sandbox alongside its other resources.
  2. Creates a session through the binding and runs a command in it.
  3. Writes a file, reads it back, and checks the content — which is the only thing that proves the agent, the capability and the transport all line up.
  4. Terminates the session and confirms it is gone.

What I did

The assertions are on observed behaviour, not on artifacts. A test that checks a template contains a string would pass against a sandbox that never starts, so this one runs code inside the session and reads back what it wrote.

Files touched

  • crates/alien-test/ — the sandbox case in the deployment suite.
  • tests/e2e/test-apps/comprehensive-typescript/ — the declaration and the handler the test drives.

How I tested

The suite itself is the test: it creates a session, runs a command, round-trips a file and
terminates, asserting on what came back rather than on rendered artifacts.

I have not run it against a live deployment in this change — it needs cloud credentials and a
free slot. It should be run before merge, and the thing to watch is teardown: an accepted delete
is not a completed one, so confirm the session is actually gone rather than trusting the call.

@greptile-apps

greptile-apps Bot commented Aug 11, 2026

Copy link
Copy Markdown

Greptile Summary

The PR adds end-to-end sandbox coverage to the comprehensive Rust and TypeScript test applications.

  • Declares and links a Local-only sandbox with execution permissions.
  • Exercises command execution and file round-tripping through a real session.
  • Terminates the session and verifies that cleanup completed.
  • Registers the sandbox binding in the shared E2E runner.

Confidence Score: 5/5

The PR appears safe to merge.

No blocking failure remains; the previously reported false-success cleanup path now runs termination after the exercise and propagates cleanup failure.

Important Files Changed

Filename Overview
tests/e2e/test-apps/comprehensive-rust/src/handlers/sandbox.rs Adds a complete Rust sandbox exercise and now propagates cleanup or termination-confirmation failures.
tests/e2e/test-apps/comprehensive-typescript/src/handlers/sandbox.ts Adds the equivalent TypeScript sandbox exercise while ensuring cleanup runs even when functional checks fail.
crates/alien-test/tests/common/bindings.rs Adds the runner-side sandbox endpoint check and validates the successful binding response.
crates/alien-test/src/e2e.rs Registers sandbox as a supported binding for the Local deployment model.

Sequence Diagram

sequenceDiagram
    participant Runner as E2E runner
    participant App as Comprehensive test app
    participant Sandbox as Sandbox binding
    Runner->>App: POST /sandbox-test/alien-sandbox
    App->>Sandbox: create session
    App->>Sandbox: run command
    Sandbox-->>App: stdout and exit frame
    App->>Sandbox: write file
    App->>Sandbox: read file
    Sandbox-->>App: matching bytes
    App->>Sandbox: terminate session
    App->>Sandbox: get session
    Sandbox-->>App: absent or terminated
    App-->>Runner: success
Loading

Reviews (6): Last reviewed commit: "test(sandbox): exercise a sandbox sessio..." | Re-trigger Greptile

Comment thread tests/e2e/test-apps/comprehensive-rust/src/handlers/sandbox.rs Outdated
@ItamarZand88
ItamarZand88 force-pushed the itamar/alien-75-sandbox-6-typescript branch from 1b99225 to b89ff3c Compare August 11, 2026 21:22
@ItamarZand88
ItamarZand88 force-pushed the itamar/alien-75-sandbox-7-e2e branch 2 times, most recently from 2cbe0ec to 4a813ec Compare August 11, 2026 21:26
@ItamarZand88
ItamarZand88 force-pushed the itamar/alien-75-sandbox-6-typescript branch from b89ff3c to 52bc095 Compare August 11, 2026 21:26
@ItamarZand88
ItamarZand88 force-pushed the itamar/alien-75-sandbox-7-e2e branch from 4a813ec to 9e50cc3 Compare August 11, 2026 22:16
@ItamarZand88
ItamarZand88 force-pushed the itamar/alien-75-sandbox-6-typescript branch from 52bc095 to 28faa2f Compare August 11, 2026 22:16
greptile-apps[bot]
greptile-apps Bot previously approved these changes Aug 11, 2026
@ItamarZand88
ItamarZand88 force-pushed the itamar/alien-75-sandbox-7-e2e branch from 9e50cc3 to d835538 Compare August 11, 2026 22:24
@ItamarZand88
ItamarZand88 force-pushed the itamar/alien-75-sandbox-6-typescript branch from 28faa2f to 57c6f58 Compare August 11, 2026 22:24
@greptile-apps
greptile-apps Bot dismissed their stale review August 11, 2026 22:24

Dismissed because a newer commit was pushed; Greptile will re-review the current head.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant