Skip to content

Enforce role-isolated workflows and persist SWE traces - #8

Merged
areshand merged 5 commits into
mainfrom
codex/persist-swe-traces
Aug 16, 2026
Merged

Enforce role-isolated workflows and persist SWE traces#8
areshand merged 5 commits into
mainfrom
codex/persist-swe-traces

Conversation

@areshand

@areshand areshand commented Aug 15, 2026

Copy link
Copy Markdown
Owner

What changed

  • Run orchestrator, writer, and read-only reviewer agents under fixed Linux identities so source writes are enforced by the OS rather than prompt compliance.
  • Keep the orchestrator read-only while allowing it to inspect shared state, logs, and tmux-backed worker/reviewer progress.
  • Route writer launches through a fixed privileged bridge that revalidates the assignment, approved decision/plan, owned paths, and live workflow phase immediately before dropping privileges.
  • Make workflow completion a hard seal. The orchestrator UID and UID sandbox cannot disable lifecycle enforcement with an environment override, and no writer can launch after completion.
  • Require decision-authority and post-implementation review evidence, including hash-bound acceptance and structured finding/TODO closure.
  • Terminate writer/reviewer process groups when a pane is killed so descendants cannot continue mutating after cancellation.
  • Persist every SWE-bench row trace outside its task container, with bounded transfer, size/SHA-256 verification, and an atomic per-row archive/manifest.
  • Keep the EvalScope adapter limited to organizing the workspace diff and handing it to the official SWE-bench scorer; it performs no duplicate semantic verification.
  • Remove obsolete workflow guidance and document the Rust/Python/shell responsibility boundary.

Security and responsibility boundary

  • Rust CLI/supervisor: role identities, write policy, lifecycle state, assignment ownership, reviewer gates, process supervision, and trace-safe runtime mechanics.
  • Python native solver/adapter: container bootstrap, task metadata, solver orchestration, diff packaging, and trace export.
  • launch.sh: compatibility entrypoint only; it launches the Rust workflow and contains no orchestration policy.
  • Official SWE-bench scorer: the sole authority for evaluation correctness.

The supervisor applies predetermined rules only. It does not make implementation or acceptance decisions. The orchestrator can read shared files and tmux sessions, but Linux permissions prevent direct repository writes.

Validation

  • cargo fmt -- --check
  • cargo test --all-targets — 18 passed
  • cargo clippy --all-targets -- -D warnings
  • bash tests/lifecycle.sh — passed, including the UID-sandbox environment-override regression test
  • bash tests/run.sh — full Rust, shell, lifecycle, evaluation, provenance, and migration suite passed
  • git diff --check
  • Final native solver image built for linux/amd64; the official targeted rerun executed that baked Rust CLI successfully.
  • GitHub Actions Framework contract tests — Ubuntu, macOS, Python 3.8, and Python 3.13 jobs all passed.

SWE-bench Pro regression evidence

Pinned first-10 baseline:

[0, 1, 1, 1, 0, 1, 1, 0, 0, 0]

The complete first-10 rerun scored 5/10:

[0, 1, 1, 1, 1, 0, 1, 0, 0, 0]

Rows 1, 2, 3, and 6 stayed solved, and row 4 improved. Row 5 initially regressed. Its persisted trace showed that the accepted worker patch was correct, but the orchestrator disabled lifecycle enforcement after completion and launched a repair worker that reverted the new call contract to stale visible tests.

The supervisor seal was then added, all local tests were rerun, and official row 5 was rerun from a newly baked final-code image. The official scorer returned 1/1, restoring every previously solved baseline row. Traces are retained at the configured host trace roots for later analysis.

The pull request remains draft and is not merged.

@areshand
areshand force-pushed the codex/persist-swe-traces branch from 4f57ebd to 12000e8 Compare August 15, 2026 18:46
@areshand areshand changed the title Persist per-row SWE traces outside task containers Enforce role-isolated workflows and persist SWE traces Aug 16, 2026
@areshand
areshand marked this pull request as ready for review August 16, 2026 02:18
@areshand
areshand merged commit 477e818 into main Aug 16, 2026
4 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant