Repository navigation
test: extract handwritten regressions from generated tests - #1190
Merged
Merged
Conversation
Codex Review SummaryThis comment shows the latest Codex review activity on this pull request.
ℹ️ About Codex in GitHubYour team has set up Codex to review pull requests in this repo. Reviews are triggered when you
Codex reacts with 👀 while any review is running, comments if it has suggestions, and reacts with 👍 once all reviews finish with no findings. |
Contributor
Contributor
Castiron custom codeEvaluated main: ✅ No new custom-code files detected. 105 mixed files remain; 3 existing customizations changed; 8 customizations removed. Compared
102 existing customizations unchanged
62 more in the full report. A changed generated baseline means this report cannot reliably identify which handwritten lines changed. Inspect the custom-code diffDownload the exact patch produced by this run (requires repository access): gh run download 38835625165 --repo openai/openai-java \
--name castiron-custom-code-38835625165-1 --dir /tmp/castiron-custom-code-38835625165-1
git apply --stat /tmp/castiron-custom-code-38835625165-1/custom-code.patch
cat /tmp/castiron-custom-code-38835625165-1/custom-code.patchOr reproduce it from an SDK checkout containing the vendored reporter: git fetch --no-tags origin c36bc50f4cb86bbf1fcbdc6a7f85f1a451319f1e 0130a2db81b8d0d65ff9d8e76ca5cb8f9619f616
python3 scripts/castiron/custom_code_report.py report \
--base c36bc50f4cb86bbf1fcbdc6a7f85f1a451319f1e \
--head 0130a2db81b8d0d65ff9d8e76ca5cb8f9619f616 --fetch --require-head-hash --public \
--out /tmp/castiron-custom-code-0130a2db81b8
cat /tmp/castiron-custom-code-0130a2db81b8/custom-code.patchThis is the current full custom patch for mixed files, not an attribution of only the handwritten lines changed by this PR. |
markstuart-oai
approved these changes
Oct 16, 2026
markstuart-oai
left a comment
markstuart-oai
left a comment
Contributor
There was a problem hiding this comment.
Reviewed all 22 changed files at 0130a2db81b8d0d65ff9d8e76ca5cb8f9619f616. No actionable findings. The 34 extracted tests retain their assertions and parameter values; the new classes stay in the same packages and Gradle test source sets. The webhook cases keep their fixed-clock fixtures and verify signatures locally, so removing their mock-server dependency is appropriate. The change reduces handwritten edits to generated files without changing SDK behavior or the budget policy.
Validation: the exact-head check list has 14 successes and one skip, including passing build, tests, Jackson and API/runtime compatibility checks. The separate OkTest summary still reports running. No local candidate execution; the targeted 367-test result is author-reported.
This was referenced Oct 16, 2026
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Handwritten regressions in generated test classes consume customization budget and complicate regeneration. Move 34 test methods into 11 dedicated handwritten test files, preserving their coverage and leaving necessary changes to generated assertions and fixtures in place. The extracted webhook signature tests run without the mock server.
This reduces the generated-file custom patch from 2,986 to 2,116 lines (870 lines, 29%) without changing runtime code, generation ownership, exclusions, or the budget. Eight generated test files now match their baseline exactly.
Validation: 367 targeted tests passed with no failures or skips; Kotlin formatting and
git diff --checkpassed. Reviewed the extraction for preserved test bodies, fixtures, and discovery.