Skip to content

chore: raise the bar for CI-runner-time-only changes - #6213

Closed
prql-bot wants to merge 1 commit into
mainfrom
skills/ci-only-bar-32259566236
Closed

chore: raise the bar for CI-runner-time-only changes#6213
prql-bot wants to merge 1 commit into
mainfrom
skills/ci-only-bar-32259566236

Conversation

@prql-bot

Copy link
Copy Markdown
Collaborator

Adds a Bar for CI-only changes section to the running-tend skill, so future sweeps hold self-initiated CI work to a higher bar than they did on #6211. Requested by @max-sixty in this comment — "we should have a much higher bar for changes that only affect CI runner time".

The rule requires all three of: recurrence across distinct incident windows (not runs), a cost beyond runner minutes, and a fix proportionate to that — a line or two, not a new script. When it doesn't clear the bar, the sweep notes the run IDs on the thread and stops rather than opening an issue or PR. Maintainer-requested CI work and a genuinely red main are carved out.

Worth flagging that tend's bundled running-in-ci skill already carries this rule, in its Weighing a Fix section — "fix waste only when the fix is a simple knob (a cadence value, a deleted step, a one-line condition); otherwise note the cost where the maintainer will see it and move on." #6211 is a straight violation of it. The reason it didn't bind is framing: every example there is about tend's own session compute ("a no-op session, a duplicated survey, a run lost to a blip that a later tick retries"), so a sweep looking at the repo's CI runner time doesn't read itself as the addressee. So this section is written to point back at that rule and widen its scope rather than restate it, which keeps the overlay short. I've also filed the framing gap upstream in tend, since it would fire the same way in any consumer.

The proportionality clause is the one #6211 most needed: that PR started as a timeout addition and reached 88 added lines across 6 files, because each review round found a real gap and each fix was individually justified. So the bar has to be re-checked as the diff grows, not just at the point the work is proposed.

On "looks like this only happened once?"

Close, but not exactly — the shape recurred, which is why I want the bar written in terms of cost and proportionality rather than a recurrence count alone.

Scanning tests runs on main for jobs that ran past 60 minutes, the 360-minute cancelled shape on the musl build-prqlc* legs appears in four runs across ~3.5 months, in three distinct incident windows:

run started cancelled jobs at 360m
26818821087 2026-06-02 1
27778015378 2026-06-18 2
32160030161 2026-08-18 3
32223676248 2026-08-19 1

The last two are ~14 hours apart and are one mirror incident, not two. I confirmed the apt-stall cause only for the August window — the June runs match on job shape and duration, but their logs weren't checked, so the common cause there is inferred rather than verified.

Either way the conclusion is the same as the one in the comment: roughly monthly, self-correcting on a re-run, and costing runner minutes only. That doesn't earn a retry script across six workflow files.

Method: repos/PRQL/prql/actions/runs/<id>/jobs over the 60 most recent completed tests runs on main, filtered to jobs with a completed-minus-started duration over 60 minutes; widened to 400 runs (2026-04-30 → present) using run-level duration over 300 minutes, since a 6h job forces a 6h run.

@prql-bot

Copy link
Copy Markdown
Collaborator Author

Closing as superseded — max-sixty/tend#1015 makes this change in the bundled running-in-ci skill instead, which is the right home for it. We reached the same diagnosis independently (the Weighing a Fix examples are all tend's own compute, so a repo's CI runner time read as a category with no bar on it), and that PR covers both bars this one did — evidence counted in incident windows rather than runs, and a remedy capped at a one-line knob — plus binding the reviewer, which this overlay didn't. A per-repo overlay restating bundled guidance is exactly what the skill-PR workflow says not to open.

Not deleting the branch, in case any of the wording is worth salvaging.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant