Skip to content

[ci] Define a typed CI coverage policy - #3575

Open
joshlf wants to merge 1 commit into
G2no4jjfqasubxit6mn7xvdcyd3ko5ji3from
Gqofuqcjxqt64bw7di2snqyunualoi3ht
Open

[ci] Define a typed CI coverage policy#3575
joshlf wants to merge 1 commit into
G2no4jjfqasubxit6mn7xvdcyd3ko5ji3from
Gqofuqcjxqt64bw7di2snqyunualoi3ht

Conversation

@joshlf

@joshlf joshlf commented Aug 25, 2026

Copy link
Copy Markdown
Member

Describe ordinary build, Miri, and semver coverage in a versioned TOML
file which the repository-owned zc tool parses into typed data. Keep
workflow permissions, secrets, runners, action references, and
publication in the hand-written workflows so generated data cannot
grant privileges.

Reject unknown fields, unsafe paths, duplicate or dangling identifiers,
overlapping work, incomplete target coverage, stale waivers, and matrix
limits above GitHub's bounds. Require semver checks to use the semantic
stable-aggregate profile and the pinned-stable toolchain source; policy
names may change without weakening either constraint. Report independent
semantic errors together and preserve deterministic ordering throughout
the validated model.

Preflight selected Cartesian products with saturating arithmetic before
allocating their cells. Count the complete required Miri universe
without materializing it. Validated selected cells are a distinct subset
of that universe, so equal cardinality proves complete coverage. A
malformed policy with a large missing product receives one bounded
summary instead of one allocation and diagnostic per missing cell.

This commit only establishes and tests the policy boundary. Existing
workflows remain authoritative until later commits add inventory,
planning, baseline comparison, and shadow-mode integration.

Tests: CARGO_NET_OFFLINE=true ./ci/check_tools.sh
Tests: ./ci/check_fmt.sh

Authored by an agent, posting via joshlf's account


Latest Update: v10 — Compare vs v9

📚 Full Patch History

Links show the diff between the row version and the column version.

Version v9 v8 v7 v6 v5 v4 v3 v2 v1 Base
v10 v9 v8 v7 v6 v5 v4 v3 v2 v1 Base
v9 v8 v7 v6 v5 v4 v3 v2 v1 Base
v8 v7 v6 v5 v4 v3 v2 v1 Base
v7 v6 v5 v4 v3 v2 v1 Base
v6 v5 v4 v3 v2 v1 Base
v5 v4 v3 v2 v1 Base
v4 v3 v2 v1 Base
v3 v2 v1 Base
v2 v1 Base
v1 Base
⬇️ Download this PR

Branch

git fetch origin refs/heads/Gqofuqcjxqt64bw7di2snqyunualoi3ht && git checkout -b pr-Gqofuqcjxqt64bw7di2snqyunualoi3ht FETCH_HEAD

Checkout

git fetch origin refs/heads/Gqofuqcjxqt64bw7di2snqyunualoi3ht && git checkout FETCH_HEAD

Cherry Pick

git fetch origin refs/heads/Gqofuqcjxqt64bw7di2snqyunualoi3ht && git cherry-pick FETCH_HEAD

Pull

git pull origin refs/heads/Gqofuqcjxqt64bw7di2snqyunualoi3ht

Stacked PRs enabled by GHerrit.

@codecov-commenter

codecov-commenter commented Aug 25, 2026

Copy link
Copy Markdown

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 91.85%. Comparing base (540594a) to head (dabba7d).

Additional details and impacted files
@@                        Coverage Diff                         @@
##           G2no4jjfqasubxit6mn7xvdcyd3ko5ji3    #3575   +/-   ##
==================================================================
  Coverage                              91.85%   91.85%           
==================================================================
  Files                                     20       20           
  Lines                                   6093     6093           
==================================================================
  Hits                                    5597     5597           
  Misses                                   496      496           

☔ View full report in Codecov by Harness.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: 4aa8517069

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

Comment thread tools/zc/src/policy.rs
Comment thread tools/zc/src/policy.rs Outdated

joshlf commented Aug 25, 2026

Copy link
Copy Markdown
Member Author

Authored by an agent, posting via joshlf's account

@codex review

@chatgpt-codex-connector

Copy link
Copy Markdown

Codex Review: Didn't find any major issues. More of your lovely PRs please.

Reviewed commit: 0c5a889b73

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

joshlf commented Aug 25, 2026

Copy link
Copy Markdown
Member Author

Authored by an agent, posting via joshlf's account

@codex review

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: 057e851aee

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

Comment thread tools/zc/src/policy.rs
Comment thread tools/zc/src/policy.rs
@joshlf
joshlf force-pushed the G2no4jjfqasubxit6mn7xvdcyd3ko5ji3 branch from f3da816 to f25c5da Compare August 26, 2026 01:28
@joshlf
joshlf force-pushed the Gqofuqcjxqt64bw7di2snqyunualoi3ht branch from 057e851 to 0be1058 Compare August 26, 2026 01:28

joshlf commented Aug 26, 2026

Copy link
Copy Markdown
Member Author

Authored by an agent, posting via joshlf's account

@codex review

Please review the current head. Treat earlier reviews of older commits as stale.

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: 0be1058023

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

Comment thread tools/zc/src/policy.rs Outdated
Describe ordinary build, Miri, and semver coverage in a versioned TOML
file which the repository-owned zc tool parses into typed data. Keep
workflow permissions, secrets, runners, action references, and
publication in the hand-written workflows so generated data cannot
grant privileges.

Reject unknown fields, unsafe paths, duplicate or dangling identifiers,
overlapping work, incomplete target coverage, stale waivers, and matrix
limits above GitHub's bounds. Require semver checks to use the semantic
stable-aggregate profile and the pinned-stable toolchain source; policy
names may change without weakening either constraint. Report independent
semantic errors together and preserve deterministic ordering throughout
the validated model.

Preflight selected Cartesian products with saturating arithmetic before
allocating their cells. Count the complete required Miri universe
without materializing it. Validated selected cells are a distinct subset
of that universe, so equal cardinality proves complete coverage. A
malformed policy with a large missing product receives one bounded
summary instead of one allocation and diagnostic per missing cell.

This commit only establishes and tests the policy boundary. Existing
workflows remain authoritative until later commits add inventory,
planning, baseline comparison, and shadow-mode integration.

Tests: CARGO_NET_OFFLINE=true ./ci/check_tools.sh
Tests: ./ci/check_fmt.sh

*Authored by an agent, posting via joshlf's account*

gherrit-pr-id: Gqofuqcjxqt64bw7di2snqyunualoi3ht
@joshlf
joshlf force-pushed the Gqofuqcjxqt64bw7di2snqyunualoi3ht branch from 0be1058 to dabba7d Compare August 26, 2026 09:58
@joshlf
joshlf force-pushed the G2no4jjfqasubxit6mn7xvdcyd3ko5ji3 branch from f25c5da to 540594a Compare August 26, 2026 09:58
@joshlf

joshlf commented Aug 26, 2026

Copy link
Copy Markdown
Member Author

Authored by an agent, posting via joshlf's account

@codex review

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: dabba7debd

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

Comment thread tools/zc/src/policy.rs
Comment on lines +1561 to +1565
|| miri_logical_cell_count > materialization_limit
|| reduced_event_cell_count > materialization_limit
|| full_event_cell_count > materialization_limit
{
return;

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P3 Badge Continue non-materializing validation before returning

When either event exceeds max_plan_cells, this returns from all remaining validation, not just the unsafe Cartesian expansion. As a result, the same policy can also contain an unknown Miri toolchain, invalid semver references, or unused target sets without reporting those independent errors, forcing policy authors through multiple validation cycles despite the validator's aggregated-diagnostics contract. Keep the cell materialization and audits behind this guard, but still run the later checks that do not construct Cartesian products.

Useful? React with 👍 / 👎.

Comment thread tools/zc/src/policy.rs
Comment on lines +1165 to +1171
let mut members = match raw.selection {
RawTargetSetSelection::All => targets.keys().cloned().collect(),
RawTargetSetSelection::MiriEligible => targets
.iter()
.filter(|(_, target)| target.miri_eligible)
.map(|(id, _)| id.clone())
.collect(),

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P2 Badge Bound target-set expansion before cloning every target

When a malformed policy declares many targets and many distinct selection = "all" target sets, each set clones every target into its own BTreeSet here before any planning limit is checked. Thus an O(n)-sized policy can allocate O(n²) identifiers and exhaust CI memory even if its eventual scopes exceed max_plan_cells and are rejected by the later preflight. Apply a hard declaration/aggregate-membership bound before resolving these sets, or preserve broad selections symbolically until bounded cell expansion.

Useful? React with 👍 / 👎.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants