Repository navigation
Speed up publication byte/history checks by 12% (no new κ) - #6155
Merged
Mister-Meeseeks merged 1 commit intoOct 28, 2026
Merged
Mister-Meeseeks merged 1 commit into
Mister-Meeseeks merged 1 commit into
Conversation
This was referenced Oct 28, 2026
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Measured verifier improvement with a scoped byte-comparison equivalence argument; no new kappa. Defer the two mismatch-only SHA256 digests in the publication preflight until exact byte equality fails.
The complete byte/history preflight on the unchanged 42-file, 54,975,541-byte Gaussian fixture falls from median 280.56 ms to 246.91 ms, an 11.99% reduction. Ten alternating runs per side have disjoint ranges, 277.11–300.94 ms before and 241.00–256.76 ms after. All 20 complete receipts agree byte-for-byte; zero command errors. The intervals include fresh-interpreter startup, imports, every Git operation, file comparison and source-history check. This is warm-filesystem evidence on CPython 3.12.15/macOS arm64, not a cross-platform timing claim.
Only one production conditional changes. Exact bytes, error text, reader cleanup/final status, inventories, optional modes and ancestry checks retain their contracts. The proof and reproduction are in
research/lazy-publication-digests/README.md. This contribution was produced by Nick Petalas. Credit ywz65's comparator, Nick Petalas's earlier history preflight, roger-sato's immutable-object isolation, Douglas Colkitt's source reader/runner and the retained mode/cleanup contributors.Evidence
PASS_REGISTERED_RESEARCH_PACKAGEpasses. Committed bytes, declared source history and executable modes pass for 13 files, at headb5a021802c045c2cb9417ad8b2933c6fea71eb39.From the worktree root:
/Users/nick/git/integer-mult-bounds/.venv/bin/python3.12 -B scripts/research_package.py --package research/lazy-publication-digests /Users/nick/git/integer-mult-bounds/.venv/bin/python3.12 -I -W error -B research/lazy-publication-digests/verify.py /Users/nick/git/integer-mult-bounds/.venv/bin/python3.12 -B scripts/check_research_commit.py --package research/lazy-publication-digests --check-source-history --check-modes /Users/nick/git/integer-mult-bounds/.venv/bin/python3.12 -I -W error -B research/lazy-publication-digests/benchmark.py --scratch-dir "$RUN_DIR" --rounds 10For timing, set
RUN_DIRto an absolute directory outside the checkout, materializeresearch/physical-gaussian-coreplus all itsVERIFY.jsoninputs, and retain full Git history. The benchmark authenticates the original script at base721c03822c421b7f7fcbb2598993b947c4d88419, uses the same shared reader on both sides, and writes its baseline and generated receipt only to that directory. The complete generated measurement receipt is retained below.Limitations
This improves publication checking, not the Gaussian fixture's mathematics. No native/selected cold replay, supplier admission, physical-machine implementation or multiplication theorem is claimed. Small packages can remain dominated by Git overhead. SHA256 implementation failure on equal buffers is outside the equivalence claim. The checkout must remain stable during checking. Linux/Windows performance and native Windows mode behavior were not run. The autonomous local run playbook stopped at this first measured improvement.
Open next questions
Measure a fixed representative publication workload, including small packages, with all complete receipts preserved. This would determine aggregate benefit; choosing that workload and running it require a separate measurement budget.
Related open work
scripts/research_package.pyto bound working-source comparison memory during complete metadata validation, using the same Gaussian package as a fixture. This PR changes publication byte/history diagnostics. The commands and measured quantities differ; no combined memory or timing benefit is claimed.scripts/find_overlap.pywith an exact numeric-reference prefilter under the samereplay-runtimecard. It measures an overlap-search command, not this publication preflight. No common performance number is compared or combined.scripts/check_research_commit.pypublication checker. It removes duplicate Git metadata reads; this PR removes mismatch-only hashing. The fixtures and timed boundaries differ, so percentages are not compared or combined.CI notes
Uncertified by CI at head
b5a021802c045c2cb9417ad8b2933c6fea71eb39. In run 40725862797,planfailed andreplay (research/lazy-publication-digests)was skipped. The package replay did not run in CI. A maintainer re-run is required after main is fixed.The onboarding job ran 680 tests and reported
FAILED (failures=2, skipped=23). Its only two failures are inherited documentation mismatches on canonical main721c03822c421b7f7fcbb2598993b947c4d88419, the PR6030 admission committed with[skip ci]. That admission changed the selected record without matching the status bolding and CI navigation:test_research_current_status.CurrentStatusTests.test_current_section_matches_selected_recordexpects standalone**0.003125804**indocs/research/current-status.md; main instead bolds the larger expression0.003125804 = 781451/250000000.test_research_current_status.CurrentCiNavigationTests.test_current_ci_section_matches_selected_recordexpectsmake frontier6030-verify; the test readsdocs/ci-verification.md, whose current CI section still names Raise conditional kappa to 0.00310045 with independent helper splits #5599 andmake frontier5599-verify.PR #6120 run 40691776027 fails the same two tests. Open PR #6050 targets the documentation fix. The edits to
scripts/check_research_commit.pyandtests/test_research_commit_bytes.pyintroduced no additional failure in this onboarding step. Local package replay and preflight evidence remain separate from hosted certification. No code or documentation was pushed after this CI failure.The dependent historical package
research/isolated-descendant-bytecodestops locally atworking source drift: scripts/check_research_commit.py. Its live pin expects SHA256ecf206a82bf1f4e748efb6cc639919582050d173b3537ec12f7265a4f7b55ead; unmodified canonical base already has63026ec3805f2a3236cc071fc78cf1c54bc6528c43e2f3d82548dc68bf346bce. The new script is09bd93c5decc662059c6dfd9759ef3a658432db906c9d75b2130f01fb9009b4e. Its verifier consumes the authenticated historical script, and all three registered commands pass directly. The metadata blocker is preserved and disclosed rather than changing pinned predecessor evidence.Advance CLAIM. Canonical main was refreshed unchanged before publication. This account's open PRs created since the CLAIM were refreshed before DONE. Related shared-verifier work #6156, #6161 and #6162 is listed above. The remaining #6149 through #6163 contributions study different objects and supply no same-instance preflight result compared here.
Merge Danger
Door: two-way. Blast Radius: publication-checker. The production change is one conditional; diagnostic and reader-lifetime regressions are covered. Frozen sources and selected frontier are unchanged.
Generated complete-command benchmark receipt
{ "base_commit": "721c03822c421b7f7fcbb2598993b947c4d88419", "complete_stdout": "PASS_RESEARCH_COMMIT_BYTES {\"commit\": \"721c03822c421b7f7fcbb2598993b947c4d88419\", \"files\": 42, \"package\": \"research/physical-gaussian-core\", \"source_history_commits\": [\"20ecf2891d80b5acf18d980b2164fb44ea7faa90\", \"be4c9c08a44f85e7162e69d220583ba5ed617a81\"]}\n", "cpu_count": 18, "errors": 0, "implementation_sha256": { "after": "09bd93c5decc662059c6dfd9759ef3a658432db906c9d75b2130f01fb9009b4e", "before": "63026ec3805f2a3236cc071fc78cf1c54bc6528c43e2f3d82548dc68bf346bce" }, "load_after": [ 4.26806640625, 3.72119140625, 3.0302734375 ], "load_before": [ 4.37890625, 3.7333984375, 3.03076171875 ], "median_fraction_saved": 0.11993171650172396, "package": "research/physical-gaussian-core", "platform": "macOS-27.0.1-arm64-arm-64bit", "python": "3.12.15 (main, Oct 14 2026, 00:55:37) [Clang 23.1.3 ]", "samples": [ { "seconds": 0.2824004159774631, "side": "before", "trial": 1 }, { "seconds": 0.24783849995583296, "side": "after", "trial": 1 }, { "seconds": 0.27711099991574883, "side": "before", "trial": 2 }, { "seconds": 0.24742033309303224, "side": "after", "trial": 2 }, { "seconds": 0.280220584012568, "side": "before", "trial": 3 }, { "seconds": 0.25164479087106884, "side": "after", "trial": 3 }, { "seconds": 0.278225583024323, "side": "before", "trial": 4 }, { "seconds": 0.24640295817516744, "side": "after", "trial": 4 }, { "seconds": 0.27879599994048476, "side": "before", "trial": 5 }, { "seconds": 0.24351800000295043, "side": "after", "trial": 5 }, { "seconds": 0.28183800005353987, "side": "before", "trial": 6 }, { "seconds": 0.24100016593001783, "side": "after", "trial": 6 }, { "seconds": 0.28033520793542266, "side": "before", "trial": 7 }, { "seconds": 0.2434565417934209, "side": "after", "trial": 7 }, { "seconds": 0.281929292017594, "side": "before", "trial": 8 }, { "seconds": 0.24607895780354738, "side": "after", "trial": 8 }, { "seconds": 0.28078408306464553, "side": "before", "trial": 9 }, { "seconds": 0.2511972920037806, "side": "after", "trial": 9 }, { "seconds": 0.30093583394773304, "side": "before", "trial": 10 }, { "seconds": 0.25676120910793543, "side": "after", "trial": 10 } ], "schema": "lazy-publication-digests-benchmark/v1", "summary": { "after": { "maximum": 0.25676120910793543, "median": 0.24691164563409984, "minimum": 0.24100016593001783, "runs": 10 }, "before": { "maximum": 0.30093583394773304, "median": 0.2805596455000341, "minimum": 0.27711099991574883, "runs": 10 } }, "verdict": "faster" }