[chubes4]

Related issues

• Of the 120 PRs merged since Oct 1, 20 changed only Studio Code agent files (apps/cli/ai/**, apps/cli/commands/ai/**, scripts/eval/**). Each one still waited on desktop E2E and Performance Metrics, the two slowest jobs (about 17.5 min each on build 23481), even though neither suite runs agent code.

How AI was used in this PR

Claude (via the Pi coding agent in a Roadie session) measured job times from Buildkite logs, classified merged PRs by changed paths, checked which suites can reach agent code, and wrote the change. I reviewed the diff and the evidence.

Proposed Changes

should-skip-job.sh gets an AGENT_PATTERNS list and an e2e job type:

• Desktop E2E (run-e2e-tests.sh) now uses --job-type e2e: the validation rules, plus a skip when every changed file is an agent file.
• Performance Metrics adds the same agent patterns to its existing skip list.
• Lint, Unit Tests and CLI E2E keep --job-type validation and still run on agent-only PRs.

A skipped job exits 0 the same way the existing docs-only skips do, so the required statuses still report as passed.

Why these two suites never exercise agent code:

• The desktop E2E sessions launch with agenticFeaturesEnabled: false (apps/studio/e2e/e2e-helpers.ts), and no desktop test drives the chat or Studio Code.
• The metrics suites (tools/metrics/tests/{app-size,site-editor,site-startup}) measure app size, site editor load and site startup.
• The agent is loaded lazily from apps/cli/index.ts (cli/commands/ai) and from studio mcp. The one module under apps/cli/ai/ used outside the agent, browser-utils.ts, is reached only through URL imports (data-liberation-client, capture-preview). The desktop E2E suite doesn't test those, and Unit and CLI E2E still run on any change to it.

A PR that also touches anything else (apps/ui, packages/common, other apps/cli code, package.json) runs every job as before. One thing an agent-only PR no longer gets: the app-size line of the performance comment. compare-perf never fails a PR, and dependency changes still run metrics.

Expected effect on agent-only PRs: the gate drops from about 19 min (p50 of the 39 PR builds since build 23377) to about 10 min, set by CLI E2E on Windows.

Testing Instructions

This PR only touches .buildkite/**, which the existing rules already skip, so its own CI doesn't run the new path. To check it locally:

• Real diff: from a checkout of Studio Code: Show a core block's declared settings in tool results that show the block #5307 (daa71b227, agent-only), run BUILDKITE_PULL_REQUEST=5307 BUILDKITE_PULL_REQUEST_BASE_BRANCH=trunk .buildkite/commands/should-skip-job.sh --job-type <type>. Result: validation runs (exit 1), and e2e and metrics skip (exit 0).
• Stubbed pr_changed_files with fixed file lists:

Changed files	validation	e2e	metrics	build
agent only (apps/cli/ai/** + tests)	run	skip	skip	run
apps/cli/commands/ai/** + scripts/eval/**	run	skip	skip	run
agent + apps/cli/commands/site/create.ts	run	run	run	run
agent + apps/ui/**	run	run	run	run
agent + packages/common/ai/**	run	run	run	run
apps/ui/** only	run	run	run	run
docs only	skip	skip	skip	skip
apps/studio/e2e/** only	run	run	skip	run

• bash -n passes. shellcheck reports nothing on the changed lines; its two findings are on existing lines.

Pre-merge Checklist

• Lint, Unit Tests and CLI E2E still run on agent-only PRs.
• Mixed PRs run every job as before.