Skip to content

test(verification): separate semantic fixtures from deadline boundaries - #222

Open
metaphorics wants to merge 2 commits into
stabilize/typescript-chaos-e2efrom
stabilize/test262-fixture-deadlines
Open

metaphorics wants to merge 2 commits into
stabilize/typescript-chaos-e2efrom
stabilize/test262-fixture-deadlines

Conversation

@metaphorics

Copy link
Copy Markdown
Collaborator

Change

Keep real-engine Test262 completion/error fixtures independent of host scheduling jitter by giving only the test helper a finite 10-second budget. Production DEFAULT_ASYNC_DEADLINE remains 500 ms. Add deterministic success/error checks exactly at that boundary and one nanosecond after it.

Evidence

The earlier full-workspace run reproduced two semantic fixtures reporting Done(Late) instead of the expected completion/error result under parallel load. After the change, the fresh pinned-toolchain run of cargo test --workspace --lib --locked passed 4,397 tests, with zero failures and four existing ignored tests. This includes all 697 verification-library tests and the real interpreter fixtures.

The synchronized stack also passed all 58 real-process CLI E2E tests, including pinned TypeScript 7.0.2 / Node 24.18.0 differential workloads, eight deterministic arithmetic seeds, malformed-frame recovery, backpressure, and watchdog coverage.

Stack

Depends on #221. That PR contains the real TypeScript/chaos suite and the Windows oracle provisioning fix. This PR changes only Test262 test code; it does not relax a production timeout, an authority gate, or a receipt requirement. Root product completion and Windows release validation are not claimed.

@coderabbitai

coderabbitai Bot commented Sep 20, 2026

Copy link
Copy Markdown
Contributor

Review Change StackReview Change Stack

No actionable comments were generated in the recent review. 🎉

ℹ️ Recent review info
⚙️ Run configuration

Configuration used: Organization UI

Review profile: CHILL

Plan: Advanced

Run ID: ee8f0b2d-cb4b-49b9-91ff-8d0a5e1ec453

📥 Commits

Reviewing files that changed from the base of the PR and between 5a952f7 and 0e4129c.

📒 Files selected for processing (1)
  • crates/bamts-verification/src/oracles/test262.rs

Included review availability: Your plan provides up to 1 included review per hour; 0 remain after this review.

📜 Recent review details
⏰ Context from checks skipped due to timeout. (3)
  • GitHub Check: Analyze (rust)
  • GitHub Check: Analyze (actions)
  • GitHub Check: Analyze (javascript-typescript)
🔇 Additional comments (1)
crates/bamts-verification/src/oracles/test262.rs (1)

1279-1282: LGTM!

Also applies to: 1290-1290, 1646-1666


📝 Summary

Summary by CodeRabbit

  • Bug Fixes
    • Improved timing behavior for asynchronous verification scenarios.
    • Ensured events occurring exactly at the configured deadline are handled correctly, while events arriving afterward are recognized as late.
    • Updated semantic fixture checks to use an appropriate 10-second timeout.

Walkthrough

Test262 semantic fixtures now use a 10-second deadline instead of the 500ms async deadline. A new test verifies exact, error, and one-nanosecond-late completion behavior.

Changes

Deadline handling

Layer / File(s) Summary
Fixture deadline and boundary validation
crates/bamts-verification/src/oracles/test262.rs
run_variant sets RunRequest.deadline to the local 10-second SEMANTIC_FIXTURE_DEADLINE. The new test verifies that completion at exactly 500ms is accepted, while completion at 500ms plus 1ns is late. Error events at the exact deadline remain DoneFailure::Error.

Priority: ⬇️ Low

Change: Other

🚥 Pre-merge checks | ✅ 5
✅ Passed checks (5 passed)
Check name Status Explanation
Title check ✅ Passed The title uses Conventional Commits syntax with the test(verification) scope and accurately describes the deadline-boundary and semantic-fixture changes.
Description check ✅ Passed The description clearly explains the changes, their purpose, test evidence, scope, and dependency on PR #221. It is directly related to the changeset.
Docstring Coverage ✅ Passed Docstring coverage is 100.00% which is sufficient. The required threshold is 70.00%. Docstring coverage is scoped to functions touched by this diff. Analyzed 4 functions across 1 files.
Linked Issues check ✅ Passed Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check ✅ Passed Check skipped because no linked issues were found for this pull request.
✨ Finishing Touches 💡 1
🛠️ Fix failing CI checks 💡
  • Commit to this branch
  • Create a new PR
✨ Simplify code
  • Commit to this branch
  • Create a new PR

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

Comment @coderabbitai help to get the list of available commands.

@chatgpt-codex-connector

chatgpt-codex-connector Bot commented Sep 20, 2026

Copy link
Copy Markdown

Codex Review Summary

This comment shows the latest Codex review activity on this pull request.

Review Status Commit Review trigger
📝 Code Review Completed 2026-09-20T12:28:20.623725Z 0e4129c PR opened
ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review" or "@codex security review".

Codex reacts with 👀 while any review is running, comments if it has suggestions, and reacts with 👍 once all reviews finish with no findings.

@amazon-q-developer amazon-q-developer Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The changes correctly separate semantic test fixture deadlines from production timeout boundaries. The new SEMANTIC_FIXTURE_DEADLINE constant (10 seconds) gives test fixtures adequate budget under parallel CI load while preserving the production DEFAULT_ASYNC_DEADLINE at 500ms. The added test production_done_deadline_boundary_remains_exact() validates that production deadline behavior remains unchanged, verifying acceptance at exactly 500ms and rejection at 500ms + 1 nanosecond. No issues found that would block merge.


You can now have the agent implement changes and create commits directly on your pull request's source branch. Simply comment with /q followed by your request in natural language to ask the agent to make changes.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant