diff --git a/research/complex-rank7-end-pair/NOTICE.md b/research/complex-rank7-end-pair/NOTICE.md new file mode 100644 index 0000000000..ace4da0b33 --- /dev/null +++ b/research/complex-rank7-end-pair/NOTICE.md @@ -0,0 +1,40 @@ +# Attribution and scope + +New contribution by Alexander Schwartzman (`AlexS-scale`), prepared with +substantial OpenAI Codex assistance, including delegated research and review. +The new code and research artifacts use the repository's Apache-2.0 license. +No independent human peer review or full formal proof is claimed. + +This contribution changes two helper ends in the retained PR869 U8 construction +and supplies a new physical owner assignment. It builds on Jeremy Kothe's +(`jeremykothe2`) PR869 triangle-hybrid deliveries, credited to NewBArch, and +PR799 X4 assembly with its credited JointXF read exchanges, +Chafik Boukhalfa's (`chafreaky`) PR741 frame coalescences and PR685 ownership +work, icekylinx's Gauge44 construction, and the retained PR635 cold-end and +residual-wrapper machinery. BlockInvert's entrance promotions, G44Pack's +packing work, and the other predecessor layers retain their original credit. +The new end recipe uses the existing cold-end principle; it does not claim +that principle as new. + +The source packages are fetched at exact commits, with their complete source +manifests, notices, and licenses preserved. PR777 by Tekalign (`tekuuu`) +provides the unchanged generic global runner, spec freezing, and independent +owner/moment checker. `code/prepare_candidate.py` adapts its data preparation +to fresh native widths and supplied owners. The mathematical admission +predicates come from the pinned PR799 package and its retained predecessors. + +The bit supplier is the cited PR589 record, including its PR555/eumemic +ancestry and DreamingOfClouds' transport. Douglas Colkitt's repository, +integration, and retained analytic framework are also prerequisites. This +notice supplements the original notices; it does not replace them. + +The comparison snapshot is Jonathan Capalbo's (`jon314159`) PR942 selected +result, following Chafik Boukhalfa's PR890 and the PR858/869/870 lineage. +Its fixed-width all-replica bound does not cover the new width inventory. + +The investigation and submission branch start at +`66849e1475c930fd7732b1a0067ffee7e1e95f60`. The final development and comparison +snapshot is `407d52ff15b2217b6e22bce44728b4c3907b6515`, which selects PR942. +`SOURCES.json` records both bases and all consumed source commits. Only the +new research package is submitted from the earlier branch base. No source +from PR784 is used. diff --git a/research/complex-rank7-end-pair/PROOF.md b/research/complex-rank7-end-pair/PROOF.md new file mode 100644 index 0000000000..9c4a299944 --- /dev/null +++ b/research/complex-rank7-end-pair/PROOF.md @@ -0,0 +1,91 @@ +# The end pair and its complete cost + +All indices below refer to PR869's pinned U8 raw word. The recipe records both +the old and new value at every changed position and checks the complete input +and output content hashes. + +For both helper registers 588 and 589, replace the old dimension-eight end +with the subspace + +``` +U = span_GF(2) {256, 128, 96, 17, 9, 5, 3}. +``` + +The integers encode vectors in GF(2)^9. The seven generators are independent, +so U has 128 points. In the exact scalar prefix, inverse row 588 is +`e588 - e412`, and inverse row 589 is `e589 - e414`. Each of the two helper +dirt columns has only its diagonal entry. For each helper, the required +end-of-B frames from its inverse row span precisely U. Thus U contains the +complete inverse-row and dirt-column cones required by the retained residual +wrapper contract. + +The new end requires eight B-frame changes, at zero-based positions +1018, 1019, 1050, 1051, 1130, 1131, 1162, and 1163. The two early-cleanup +frames at entries 148 and 149 also become U. No scalar gate changes. The +independent point audit checks containment for every adjacent frame, every +source and target boundary, the center interface, all paid jumps, and the +unchanged scalar program. The unchanged native auditors independently derive +the scalar prefix and check both end cones and both complete orientations. + +There are eight paid changes from rank 3 to rank 2, ten from rank 1 to rank 2, +and four from rank 2 to rank 1. Therefore the local profile change is exactly + +``` +ΔH_local = {1:-6, 2:+14, 3:-8}. +``` + +Both helpers have empty entrances. Each end reduction releases one coordinate, +so the helper width becomes 3776 per call. The width inventory is +`{4:80, 5:32, 7:56, 8:120, 9:216}`. Keeping the predecessor owner assignment +would increase the complete cost. The new assignment is an essential part of +the construction. + +The frozen flow lists width patterns and call counts for the 144 existing +physical windows, each with 112 ports. For a pattern used N times, each width +slot receives N items and each call supplies at most N items. Split the width +counts between those slots and add dummy slots to obtain an N-regular +bipartite multigraph. Repeated perfect matchings give N colors. Each color is +one physical bin with the stated width pattern and at most one item from any +call. Dummy slots do not create helper roles. + +`build_owners.py` performs this integer construction deterministically. It +then checks the literal assignment separately: all 36,288 call/helper pairs +occur exactly once; families are distinct within each call; every role uses +a valid window; and the coordinate intervals are disjoint and fit the stated +capacity. The retained global auditor independently reconstructs the physical +windows and checks the final assignment against the native helper widths. +The complete unused-space profile is exactly `{48:2400}`. + +Across all 72 calls, the global profile change from the U8 construction is + +``` +ΔH_global = {1:-433, 2:+1008, 3:-577, 44:-1, 48:+4}. +``` + +The rank mass does not change: +`-433 + 2*1008 - 3*577 - 44 + 4*48 = 0`. +The resulting mass is 520704, retaining the deficit 2560 and the same physical +geometry. The certified global prefix bound is 257 bits, within the retained +512-bit allowance. The physical row coefficient is 16467, below 20161. +Admission checks both retained rational interval engines and the complete +precision and tape invoice. + +The native complex recurrence is accepted at +`b = 6659172436825204659 / 4000000000000000000000` +and rejected at `b + 10^-24`. The complete bit profile retains its bad-residue +surcharge. Both moment engines and all 47 outer inequalities certify + +``` +κ = 166202617947539 / 100000000000000000. +``` + +This exceeds the selected PR942 value by exactly +`65631761443 / 1000000000000000000`. PR942's all-replica bound fixes the +previous U8 profile and width inventory. The new helper ends change both, +so that bound does not exclude this construction. + +The scalar program and its finite envelopes retain their source predicates. +The new frames, wrapper ranks, local profile, owner map, global profile, +moments, and outer witness are freshly admitted. This establishes a finite +conditional improvement under the predecessor interfaces, with the scope +stated in `README.md`. diff --git a/research/complex-rank7-end-pair/README.md b/research/complex-rank7-end-pair/README.md new file mode 100644 index 0000000000..9ff6a51c3e --- /dev/null +++ b/research/complex-rank7-end-pair/README.md @@ -0,0 +1,91 @@ +# U8 with two rank-seven helper ends: conditional κ = 0.00166202617947539 + +This construction gives **κ = 0.00166202617947539**, above the pinned selected +PR942 result, `0.001661960547713947`. The exact increase is +`65631761443 / 1000000000000000000`, approximately **0.00394906%** of that κ. +This compares asymptotic exponent savings, not measured runtime. + +The construction combines PR869's U8 word with two smaller helper ends and +an explicit owner assignment. Helpers 588 and 589 finish in dimension seven +instead of eight. Eight B frames and two early-cleanup frames change with +those ends. The owner assignment combines the released space into rank-48 +residuals. All raw scalar gates and their order retain their U8 values. The +ports, center interface, physical geometry, and 72-call schedule retain their +PR799 values. + +| Quantity | U8 before this change | This construction | +| --- | ---: | ---: | +| Local rank-one children | 2548 | 2542 | +| Local rank-two children | 338 | 352 | +| Local rank-three children | 446 | 438 | +| Local rank-four / five / eight children | 162 / 72 / 8 | 162 / 72 / 8 | +| Helper width per call | 3778 | 3776 | +| Global idle profile | `{1:1, 3:1, 44:1, 48:2396}` | `{48:2400}` | + +The complete global profile is +`{1:183024, 2:25344, 3:31536, 4:11664, 5:5184, 8:576, 48:2400}`. +Its rank mass is 520704, with deficit 2560. The certified complex endpoint is +`0.001664793109206301164750`; the retained PR589 bit endpoint is +`0.001733427039589844234749`. The complex supplier remains the limiting one. +The conservative calculation separately gives `κ = 0.001662026176592891`. + +The 72-call construction has a certified global prefix bound of 257 bits and +retains the 512-bit allowance. Its physical row coefficient is 16467, below +the retained limit 20161. The comparison result uses 333 calls and a 1024-bit +allowance. Its all-replica bound fixes the previous local profile and helper +widths; this construction changes both. + +From the repository root, use Python 3.11 or newer and Git: + +```sh +python3 -B research/complex-rank7-end-pair/reproduce.py --work /tmp/rank7-end-pair +``` + +Use a new output directory. The command fetches the four exact source commits +in `SOURCES.json`. It rebuilds the PR741 ancestry, PR799 word, and PR869 U8 +word; applies the twelve changed annotations; rebuilds the owner assignment; +and runs complete native and global admission. No optimizer or Python package +installation is required. If a local repository contains the pinned commits, +add `--git-repository /absolute/path/to/repository` to use its objects without +fetching. Source files are extracted into the output directory in either case. + +Expected final output: + +```text +PASS_COLD_RANK7_END_PAIR_COMPLETE_ADMISSION 0.00166202617947539 +``` + +The required checks include all 38 U8 construction controls, every 728-input +column in both orientations, all 2,950 phase edges, exact residual wrappers, +25 native controls, and the 89 applicable inherited cold-end controls. +Dedicated controls cover the new dimension-seven ends and their frame changes. +The independent point audit uses explicit subsets of the 512-point frame +space. Owner checks cover all 36,288 roles, complete availability windows, +distinct families per call, capacity, and disjoint compact coordinate prefixes. +Discovery is followed by frozen strict admission, 69 admission controls, +12 bit-supplier controls, seven independent owner controls, and both exact +moment engines with all 47 outer inequalities. `data/EXPECTED.json` pins +the resulting word, programs, phase recipes, owner map, profile, and κ. + +`evidence/VALIDATION.json` records the final cold run's environment, coverage, +and input hashes. `evidence/ASSEMBLY.json.gz` preserves the unchanged +assembler's complete output. Its historical descriptions and comparison +labels refer to its predecessors. The profile, moment endpoints, κ, and +outer inequalities are the computed results for this candidate. + +`PROOF.md` explains the end frames, packing certificate, and complete cost +change. `SEARCH.json` records the finite discovery and subsequent composition +with U8. Search optimality is not needed to verify the construction. +`data/SELECTED942.json.gz` preserves the selected-result snapshot used only +for the numerical comparison; its source commit and hash are in `SOURCES.json`. + +This is a finite conditional witness for +`T(n) = O(n (log n)^(1-kappa))`. The inherited all-size compiler, +common-label and mutable-source realization, whole-call hosting, precision, +tape, exact-recovery, analytic-transfer, and bit H1–H5 assumptions remain. +The command reprices the pinned PR589 bit supplier; it does not repeat its +bit-circuit construction. Finite replay does not prove the full multiplication +theorem. The selected certificate is outside this contribution's scope. + +The contribution is independent of PR784. See `NOTICE.md` for predecessor +credit, licensing, and the disclosure of AI assistance. diff --git a/research/complex-rank7-end-pair/SEARCH.json b/research/complex-rank7-end-pair/SEARCH.json new file mode 100644 index 0000000000..32d2ce2fac --- /dev/null +++ b/research/complex-rank7-end-pair/SEARCH.json @@ -0,0 +1,106 @@ +{ + "scope": "Discovery of one finite construction; no universal or global optimality claim.", + "research_seed": 143265410407226848, + "winning_frame_search": { + "source_pr": 799, + "source_commit": "80e6463157cfc80b85af09e7e2f14748fe2dac60", + "domain": "20 groups of existing dimension-eight helper ends with dimension-seven lower spaces under fixed scalar code and interfaces. Exact subspace refitting of B and early-cleanup frames.", + "completed_groups": 20, + "selected_group": 17, + "selected_helpers": [ + 588, + 589 + ], + "solver_seed": 73, + "stopping_rule": "Select a legal word with positive complete packing headroom, obtain a literal owner map, and require unchanged full native and global admission.", + "minimalization": "Retain only the eight B frames, two tail frames, and two ends necessary for the verified profile.", + "screen_only": "Local cost and aggregate packing screens are not admitted kappa witnesses." + }, + "packing_search": { + "date": "2026-10-13", + "status": "FROZEN_FEASIBLE_INTEGER_FLOW_NOT_A_GENERAL_OPTIMALITY_CLAIM", + "source_pr": 799, + "source_commit": "80e6463157cfc80b85af09e7e2f14748fe2dac60", + "owner_lineage": "chafreaky PR685, reused by PR741 and adapted by G44Pack for PR799", + "command": "pattern_flow.py --plan source-799/research/gauge44-frames-x/data/PLAN.json --owners source-799/research/gauge44-frames-x/data/OWNERS.json.gz --word exact-search/end-deepen-17-0.word.json.gz --out order-search/packing/end17-free4 --free-legacy --distance 4 --seconds 60 --workers 2", + "random_seed": 73, + "distance": 4, + "free_legacy": true, + "workers": 2, + "time_limit_seconds": 60, + "objective": "feasibility; all nonempty physical bins are full", + "solver": "OR-Tools CP-SAT", + "solver_version": "9.15.6755", + "python_version": "3.12.12", + "solver_receipt": { + "status": "OPTIMAL", + "pattern_variables": 503, + "flow_variables": 25699, + "seconds": 58.33655075000206, + "input_word_sha256": "9f63de243f867e2c52212b18d2253059a3bc965db4a981ecf18b73d83969006c", + "fixed_partial_bins": [] + }, + "search_code_sha256": "88e905223d3691c00bef5e6873d6709273ab2bdf7b2142797448ebab6a390730", + "search_word_content_sha256": "9f63de243f867e2c52212b18d2253059a3bc965db4a981ecf18b73d83969006c", + "rebinding": "The first minimal end-pair word removed unrelated frame rechoices while preserving the search profile and helper widths. Later U8 composition changes the local paid profile but retains the entire helper-width vector, frozen flow and literal owners.", + "original_flow_content_sha256": "7ff3e4c2bbe9e0beb2592888de7db962fd34971a025be841c8f1711b5139f8f1", + "owner_content_sha256": "02bde66a90b8ae9b0c4d8d563ca2218ad1de851fa6dec3d1abb5d3a30ca3dd2a", + "idle_profile": { + "48": 2400 + }, + "rank_mass": 520704, + "deficit": 2560, + "search_moment_delta_at_baseline_b": "-0.119035908243901811918210669949724217035275333404321692154", + "required_admission": "unchanged native local and strict global replay plus independent exact repricing", + "end_pair_only_minimal_word_content_sha256": "796a40505ed9dc38d64c5b7856a0908e70bd9f22cd869edf01ebe66ac02ab15a", + "search_global_profile": { + "1": 183024, + "2": 25344, + "3": 30384, + "4": 13968, + "5": 4032, + "8": 576, + "48": 2400 + } + }, + "earlier_searches": [ + { + "domain": "Both pinned words, fixed scalar order and fixed entrance/end interfaces; full induced GF(2)^9 subspace intervals in 140 components.", + "result": "No lower profile found. Integer-rounded moment objectives do not prove a universal real-objective optimum." + }, + { + "domain": "Legal local commuting reorders and exact adjacent shear rewrites with correction gates.", + "result": "No improved candidate passed the cost screen; these searches are not impossibility results." + }, + { + "domain": "Six E4-to-E5 entrance promotions, independently verified natively, plus uniform and small mixed owner inventories.", + "result": "Local savings did not overcome global packing cost. The new end dimensions change the width inventory and escape that restriction." + } + ], + "final_raw_word_sha256": "096ed4abee4bb02c0d6f03bb8191510e57f6b1fcd34e8644f4fd336254e46411", + "final_owner_sha256": "02bde66a90b8ae9b0c4d8d563ca2218ad1de851fa6dec3d1abb5d3a30ca3dd2a", + "admission_status": "PASS_STRICTLY_IMPROVED_KAPPA_UNCHANGED_ADMISSION", + "kappa": "166202617947539/100000000000000000", + "composition": { + "date": "2026-10-13", + "source_pr": 869, + "source_commit": "73e86f69bf9a391e3c334999f7bbe3825dc42759", + "method": "Apply the same end subspace and eight B frames to the pinned U8 word. Its eight new early tails move the two affected tail entries from 140/141 to 148/149. All 504 helper widths otherwise match the prior end-pair candidate.", + "new_search_required": false, + "independent_point_audit": "Exact twelve-annotation difference and local delta {1:-6,2:+14,3:-8}; U8 scalar gates unchanged.", + "owner_reuse": "The full per-helper width vector is identical, so the frozen integer flow and literal owner payload remain valid. All roles, windows, charts, and controls are rechecked.", + "new_global_profile": { + "1": 183024, + "2": 25344, + "3": 31536, + "4": 11664, + "5": 5184, + "8": 576, + "48": 2400 + }, + "selected_comparison_pr": 942, + "selected_comparison_kappa": "1661960547713947/1000000000000000000", + "scope": "PR942 all-replica bound fixes the previous U8 profile and helper widths; this construction changes both." + }, + "original_end_pair_only_kappa": "1661712108827881/1000000000000000000" +} diff --git a/research/complex-rank7-end-pair/SOURCES.json b/research/complex-rank7-end-pair/SOURCES.json new file mode 100644 index 0000000000..44936224d9 --- /dev/null +++ b/research/complex-rank7-end-pair/SOURCES.json @@ -0,0 +1,46 @@ +{ + "schema": "complex-rank7-end-pair-sources/1", + "base_commit": "407d52ff15b2217b6e22bce44728b4c3907b6515", + "sources": [ + { + "pr": 741, + "repository": "https://github.com/CrocSwap/integer-mult-bounds.git", + "commit": "e516b1463606840a15f80aac1995d64ce0092e89", + "path": "research/complex-sw4-frame8", + "manifest_sha256": "b9b39c206fcd2613382b0dcbda23a371e8781d1bedaf8e2f8e7df1e79dc1fa89" + }, + { + "pr": 799, + "repository": "https://github.com/CrocSwap/integer-mult-bounds.git", + "commit": "80e6463157cfc80b85af09e7e2f14748fe2dac60", + "path": "research/gauge44-frames-x", + "manifest_sha256": "e1b7998c8f7cb24f8674d250d0510f2b5b351b2ed0b35f17a17e6bacf3d9824b" + }, + { + "pr": 777, + "repository": "https://github.com/CrocSwap/integer-mult-bounds.git", + "commit": "eca5c317d8176401a1f71a5ea1e81384c2eab1e0", + "path": "research/pr699-frame-retiming", + "manifest_sha256": "e0102586186663b2a667768565f61db184638b370f68f0ea58dd673e523173ec" + }, + { + "pr": 869, + "repository": "https://github.com/CrocSwap/integer-mult-bounds.git", + "commit": "73e86f69bf9a391e3c334999f7bbe3825dc42759", + "path": "research/gauge44-barch", + "manifest_sha256": "ffcafdcb4693211fdb95d1b7b3f5bfc54f203aa0cd2cd9dbdcdad32f42104e17" + } + ], + "research_start_commit": "66849e1475c930fd7732b1a0067ffee7e1e95f60", + "submission_branch_base_commit": "66849e1475c930fd7732b1a0067ffee7e1e95f60", + "comparison_snapshot": { + "pr": 942, + "artifact": "data/SELECTED942.json.gz", + "repository": "https://github.com/CrocSwap/integer-mult-bounds.git", + "commit": "407d52ff15b2217b6e22bce44728b4c3907b6515", + "path": "certificates/selected-result.json", + "source_content_sha256": "1d35283cb0ce3f8db1f285683933517f9e87475d295e2b6316399ad9928c81c3", + "audited_source_commit": "1960cfa919a0945d0ca4cb14ea96237000037903", + "scope": "Immutable selected-result snapshot for numerical comparison; no T37 construction or enlarged prefix allowance is used." + } +} diff --git a/research/complex-rank7-end-pair/audit_points.py b/research/complex-rank7-end-pair/audit_points.py new file mode 100644 index 0000000000..32459874b2 --- /dev/null +++ b/research/complex-rank7-end-pair/audit_points.py @@ -0,0 +1,223 @@ +"""Independent frame accounting by explicit subsets of the 512-point space. + +No builder, solver, native auditor, or Gaussian elimination is imported. +Scalar identities, row/column cones, phases, and global admission are separate. +Prepared with OpenAI Codex assistance. Apache-2.0. +""" +import argparse +from collections import Counter +import copy +from functools import lru_cache +import gzip +import hashlib +import json +from pathlib import Path + + +BASE_SHA = '090fd574747fbf9628e70cb9beb792732914cfa52e1df147579fe4dbccc68c5a' +WORD_SHA = '096ed4abee4bb02c0d6f03bb8191510e57f6b1fcd34e8644f4fd336254e46411' +RECIPE = Path(__file__).resolve().parent / 'data/END-RECIPE.json' + + +class Reject(ValueError): + pass + + +def require(condition, message): + if not condition: + raise Reject(message) + + +def canonical(value): + return json.dumps(value, sort_keys=True, separators=(',', ':')).encode() + + +def read(path): + raw = gzip.decompress(path.read_bytes()) + return json.loads(raw), hashlib.sha256(raw).hexdigest() + + +@lru_cache(None) +def points(vectors): + space = {0} + for vector in vectors: + require(type(vector) is int and 0 <= vector < 512, 'invalid GF2 vector') + space |= {value ^ vector for value in space} + require(len(space) == 1 << len(vectors), 'dependent frame basis') + return frozenset(space) + + +def dimension(space): + return len(space).bit_length() - 1 + + +def registers(event): + require(event[0] in ('add', 'scale'), 'unknown gate') + result = event[1:3] if event[0] == 'add' else event[1:2] + require(all(type(q) is int and 0 <= q < 728 for q in result), 'register range') + return result + + +def recount(word): + require(tuple(word['report'][k] for k in ('v', 'h', 'R')) == (112, 9, 504), 'dimensions') + require(len(word['ports']) == len(set(word['ports'])) == 112, 'ports') + full, zero = frozenset(range(512)), points(()) + start = [points((p,)) for p in word['ports']] + [zero] * 616 + for q, frame in word['start_extra'].items(): + require(224 <= int(q) < 728, 'start register') + start[int(q)] = points(tuple(frame)) + current, paid = start.copy(), Counter() + + def climb(q, target): + require(current[q] <= target, 'nonmonotone frame') + degree = dimension(target) - dimension(current[q]) + if degree: + paid[degree] += 1 + current[q] = target + + for section in ('A', 'B'): + if section == 'B': + levels = {int(q): points(tuple(frame)) for q, frame in word['target_levels'].items()} + require(all(current[112 + q] == levels.get(q, zero) for q in range(112)), 'target interface') + require(len(word['centers']) == len(word['center_frames']) == 8, 'center count') + for q, frame in zip(word['centers'], word['center_frames']): + frame = points(tuple(frame)) + require(current[q] == frame, 'center interface') + served = {levels.get(event[1] - 112, zero) for event in word['cut'] if event[2] == q} + for target in sorted(served, key=len, reverse=True): + require(target <= frame, 'center descent') + degree = dimension(frame) - dimension(target) + if degree: + paid[degree] += 1 + frame = target + for event in word[section]: + frame = points(tuple(event[-1])) + for q in registers(event): + climb(q, frame) + seen = set() + for index, frame in word['early_cleanup']: + require(type(index) is int and 0 <= index < len(word['cleanup']), 'tail index') + require(index not in seen, 'duplicate tail') + seen.add(index) + for q in registers(word['cleanup'][index]): + climb(q, points(tuple(frame))) + end = ([full] * 112 + + [frozenset(x for x in range(512) if (x & p).bit_count() % 2 == 0) for p in word['ports']] + + [full] * 504) + for q, frame in word['helper_final'].items(): + require(224 <= int(q) < 728, 'end register') + end[int(q)] = points(tuple(frame)) + for q, frame in enumerate(end): + climb(q, frame) + for index, event in enumerate(word['cleanup']): + if index not in seen: + require(all(end[q] == full for q in registers(event)), 'full cleanup reads a low end') + profile = {str(k): count for k, count in sorted(paid.items())} + require(canonical(profile) == canonical(word['report']['local_profile']), 'paid profile') + mass = sum(rank * count for rank, count in paid.items()) + require(type(word['report']['local_rank_mass']) is int and word['report']['local_rank_mass'] == mass, 'rank mass') + widths = [dimension(end[q]) - dimension(start[q]) for q in range(224, 728)] + require(all(0 < width <= 9 for width in widths), 'helper width range') + inventory = {str(k): count for k, count in sorted(Counter(widths).items())} + require(canonical(inventory) == canonical(word['report']['width_inventory']), 'width inventory') + return paid, widths + + +def check_edits(baseline, candidate, recipe): + require(recipe['input_sha256'] == BASE_SHA and recipe['expected']['output_sha256'] == WORD_SHA, 'recipe pins') + restored = copy.deepcopy(candidate) + require([q for q, _, _ in recipe['ends']] == [588, 589], 'end sites') + require([i for i, _, _ in recipe['B_frames']] == + [1018, 1019, 1050, 1051, 1130, 1131, 1162, 1163], 'B-frame sites') + require([i for i, _, _, _ in recipe['early_cleanup']] == [148, 149], 'tail sites') + changes = [] + for q, old, new in recipe['ends']: + require(baseline['helper_final'][str(q)] == old and candidate['helper_final'][str(q)] == new, 'literal end edit') + restored['helper_final'][str(q)] = old + changes.append((old, new)) + for index, old, new in recipe['B_frames']: + require(baseline['B'][index][-1] == old and candidate['B'][index][-1] == new, 'literal B edit') + restored['B'][index][-1] = old + changes.append((old, new)) + for index, cleanup_index, old, new in recipe['early_cleanup']: + require(baseline['early_cleanup'][index] == [cleanup_index, old] and + candidate['early_cleanup'][index] == [cleanup_index, new], 'literal tail edit') + restored['early_cleanup'][index] = [cleanup_index, old] + changes.append((old, new)) + for old, new in changes: + a, b = points(tuple(old)), points(tuple(new)) + require(b < a and len(a) == 2 * len(b), 'codimension-one frame reduction') + for key in ('local_profile', 'local_rank_mass', 'width_inventory'): + require(canonical(candidate['report'][key]) == canonical(recipe['expected'][key]), 'reported recipe result') + restored['report'][key] = baseline['report'][key] + require(canonical(restored) == canonical(baseline), 'changes outside the twelve annotations and accounting') + + +def controls(candidate, recipe): + cases = [] + mutant = copy.deepcopy(candidate) + mutant['report']['local_profile']['1'] += 2 + mutant['report']['local_profile']['2'] -= 1 + cases.append(('mass-preserving false profile', mutant, 'paid profile')) + mutant = copy.deepcopy(candidate) + mutant['report']['local_profile']['1'] = float(mutant['report']['local_profile']['1']) + cases.append(('floating profile count', mutant, 'paid profile')) + mutant = copy.deepcopy(candidate) + mutant['report']['width_inventory']['7'] -= 1 + mutant['report']['width_inventory']['8'] += 1 + cases.append(('false helper widths', mutant, 'width inventory')) + mutant = copy.deepcopy(candidate) + mutant['report']['local_rank_mass'] += 1 + cases.append(('false rank mass', mutant, 'rank mass')) + mutant = copy.deepcopy(candidate) + index, old, _ = recipe['B_frames'][0] + mutant['B'][index][-1] = old + cases.append(('omitted required B lowering', mutant, 'nonmonotone frame')) + result = [] + for name, mutant, expected in cases: + try: + recount(mutant) + except Reject as error: + require(str(error) == expected, 'unexpected point-control rejection: ' + name) + result.append(dict(name=name, rejected=expected)) + else: + raise Reject('accepted point control: ' + name) + return result + + +def main(): + parser = argparse.ArgumentParser(description=__doc__) + parser.add_argument('--baseline', type=Path, required=True) + parser.add_argument('--word', type=Path, required=True) + parser.add_argument('--recipe', type=Path, default=RECIPE) + parser.add_argument('--output', type=Path, required=True) + args = parser.parse_args() + baseline, before_sha = read(args.baseline) + candidate, after_sha = read(args.word) + require((before_sha, after_sha) == (BASE_SHA, WORD_SHA), 'input content pins') + recipe = json.loads(args.recipe.read_text(encoding='utf-8')) + check_edits(baseline, candidate, recipe) + before, old_widths = recount(baseline) + after, widths = recount(candidate) + delta = {r: after[r] - before[r] for r in before.keys() | after.keys() if before[r] != after[r]} + require(delta == {1: -6, 2: 14, 3: -8}, 'profile delta') + changes = [dict(helper=q + 224, before=a, after=b) for q, (a, b) in enumerate(zip(old_widths, widths)) if a != b] + require(changes == [dict(helper=588, before=8, after=7), dict(helper=589, before=8, after=7)], 'individual helper widths') + report = dict(status='PASS_EXPLICIT_512_POINT_END_PAIR_ACCOUNTING', + input_sha256=before_sha, output_sha256=after_sha, + changed_annotations=dict(helper_final=2, B=8, early_cleanup=2), + local_profile=dict(sorted(after.items())), delta=delta, + rank_mass_before=sum(r * n for r, n in before.items()), + rank_mass_after=sum(r * n for r, n in after.items()), + width_inventory=dict(sorted(Counter(widths).items())), + width_sum_before=sum(old_widths), width_sum_after=sum(widths), + changed_helper_widths=changes, rejected_controls=controls(candidate, recipe), + scope='Literal source, target, center, gate, tail, and end frames; paid profile; individual helper widths; ' + 'exact twelve-annotation recipe. Scalar identities, cones, phases, owners, and kappa are separate.') + args.output.parent.mkdir(parents=True, exist_ok=True) + args.output.write_text(json.dumps(report, indent=2) + '\n', encoding='utf-8') + print(report['status'], '12 annotations; 5 rejected controls', flush=True) + + +if __name__ == '__main__': + main() diff --git a/research/complex-rank7-end-pair/build_owners.py b/research/complex-rank7-end-pair/build_owners.py new file mode 100644 index 0000000000..27c856f06d --- /dev/null +++ b/research/complex-rank7-end-pair/build_owners.py @@ -0,0 +1,387 @@ +#!/usr/bin/env python3 +"""Reproduce and check the frozen rank-seven whole-owner map. + +The whole-owner interface and source schedule are inherited from the +#685/#741/#799 lineage. No optimizer runs during this reproduction. +""" +import argparse +from collections import Counter, defaultdict, deque +import copy +import gzip +import hashlib +import json +from pathlib import Path + + +def raw(path): + data = Path(path).read_bytes() + return gzip.decompress(data) if str(path).endswith('.gz') else data + + +def read(path): + return json.loads(raw(path)) + + +def schedule_geometry(parent, order, ports): + capacities = {} + child_ports = defaultdict(set) + for row in parent['idle_children']: + child, port, rank = row['child'], row['port'], row['rank'] + assert row['replicas'] == 8 and 0 <= port < ports + assert port not in child_ports[child] + child_ports[child].add(port) + assert capacities.get(child, rank) == rank + capacities[child] = rank + assert all(ps == set(range(ports)) for ps in child_ports.values()) + blocks, source_ids, globals_ = {}, [], [] + stage = None + for index, event in enumerate(parent['execution_schedule']): + if event['kind'] in ('bank_padding_cover_sweep', 'translate_every_live_family'): + globals_.append(index) + continue + if event['kind'] == 'local_cover_sweep': + stage = event['stage'] + assert stage not in blocks + blocks[stage] = [] + assert stage is not None + blocks[stage].append((index, event)) + source_ids.append(index) + assert globals_ == [30, 31] and source_ids == list(range(30)) + stage_pairs = {e['stage']: e['pair'] for e in parent['recipe'] if e['kind'] == 'local'} + projections = defaultdict(list) + last, windows, pairs, sides, calls = {}, {}, {}, {}, {} + position = 0 + for time, (stage, replica) in enumerate(order): + for index, event in blocks[stage]: + projections[replica].append(index) + if event['kind'] == 'local_cover_sweep': + calls[time] = position + last[replica, stage_pairs[stage]] = position + elif event['kind'] == 'bridge': + last[replica, event['src']] = position + last[replica, event['dst']] = position + elif event['kind'] == 'idle_cover_sweep': + for side, child in enumerate(event['child_indices']): + if child not in capacities: + continue + assert (replica, child) not in windows + pair = event['pair'] + assert pairs.get(child, pair) == pair + assert sides.get(child, side) == side + pairs[child], sides[child] = pair, side + windows[replica, child] = (last.get((replica, pair), -1), position) + position += 1 + assert set(projections) == set(range(8)) + assert all(indices == source_ids for indices in projections.values()) + assert set(windows) == {(replica, child) for replica in range(8) for child in capacities} + result = { + f'{replica},{child}': [capacities[child], [t for t in range(72) if left < calls[t] < right]] + for (replica, child), (left, right) in windows.items() + } + return result, pairs, sides + + +def audit(flow, owners, plan, word, parent, word_hash): + ports, helpers = word['report']['v'], word['report']['R'] + assert (ports, helpers) == (112, 504) + order = [tuple(call) for call in plan['order']] + assert len(order) == 72 and set(order) == {(stage, replica) for stage in range(9) for replica in range(8)} + geometry = {key: plan[key] for key in ('order', 'windows', 'pairs', 'sides')} + assert digest(geometry) == flow['geometry_sha256'] + if parent is None: + windows = plan['windows'] + pairs = {int(child): pair for child, pair in plan['pairs'].items()} + sides = {int(child): side for child, side in plan['sides'].items()} + else: + windows, pairs, sides = schedule_geometry(parent, order, ports) + assert plan['windows'] == windows + assert plan['pairs'] == {str(child): pair for child, pair in pairs.items()} + assert plan['sides'] == {str(child): side for child, side in sides.items()} + widths = [ + len(word['helper_final'].get(str(2 * ports + h), range(9))) + - len(word['start_extra'].get(str(2 * ports + h), [])) + for h in range(helpers) + ] + assert flow['input_word_sha256'] == word_hash + assert flow['helper_widths'] == widths and all(1 <= w <= 9 for w in widths) + active_data = { + (call['stage'], call['replica']): {family for family, route in call['owners'][:2 * ports]} + for call in parent['calls'] + } if parent is not None else None + seen, families, bins = set(), defaultdict(set), defaultdict(list) + assert len(owners) == 72 * helpers + fields = {'child', 'copy', 'family', 'helper', 'offset', 'owner_replica', 'port', 'replica', 'stage', 'time', 'width'} + for row in owners: + assert set(row) == fields and all(type(value) is int for value in row.values()) + stage, replica, helper = row['stage'], row['replica'], row['helper'] + role, time = (stage, replica, helper), row['time'] + assert 0 <= helper < helpers and 0 <= time < 72 and order[time] == (stage, replica) + assert role not in seen + seen.add(role) + owner, child, port = row['owner_replica'], row['child'], row['port'] + assert 0 <= owner < 8 and 0 <= port < ports + cap, times = windows[f'{owner},{child}'] + assert time in times and row['copy'] == owner * ports + port + family = owner * 8 * ports + (2 * pairs[child] + sides[child]) * ports + port + assert row['family'] == family and family not in families[time] + if active_data is not None: + assert family not in active_data[stage, replica] + families[time].add(family) + width, offset = row['width'], row['offset'] + assert width == widths[helper] and 0 <= offset and offset + width <= cap + bins[owner, child, port].append((offset, time, width)) + assert seen == {(stage, replica, h) for stage, replica in order for h in range(helpers)} + slots, idle = {}, Counter() + for key, (cap, times) in windows.items(): + owner, child = map(int, key.split(',')) + for port in range(ports): + key3, offset = (owner, child, port), 0 + slot = [] + for actual_offset, time, width in sorted(bins[key3]): + assert actual_offset == offset + offset += width + slot.append((time, width)) + assert offset <= cap + slots[key3] = sorted(slot) + if cap > offset: + idle[cap - offset] += 1 + assert flow['fixed_partial_bins'] == [] + assert len(flow['windows']) == len(windows) + assert {window['key'] for window in flow['windows']} == set(windows) + pattern_count = 0 + for window in flow['windows']: + owner, child = map(int, window['key'].split(',')) + cap, times = windows[window['key']] + used = 0 + for pattern in window['patterns']: + count = pattern['count'] + assert type(count) is int and 0 < count <= ports - used + expected_widths = Counter(pattern['widths']) + assert all(type(w) is int and 1 <= w <= 9 for w in pattern['widths']) + assert sum(pattern['widths']) <= cap + declared = {} + for time, width, amount in pattern['flows']: + assert all(type(value) is int for value in (time, width, amount)) + assert time in times and width in expected_widths and 0 < amount <= count + assert (time, width) not in declared + declared[time, width] = amount + actual = Counter() + for port in range(used, used + count): + slot = slots[owner, child, port] + assert Counter(width for time, width in slot) == expected_widths + assert len({time for time, width in slot}) == len(slot) + actual.update(slot) + assert actual == Counter(declared) + assert all(sum(n for (t, w), n in declared.items() if t == time) <= count for time in times) + used += count + pattern_count += 1 + assert used == ports + profile = Counter({int(rank): 72 * count for rank, count in word['report']['local_profile'].items()}) + profile.update(idle) + mass = sum(rank * count for rank, count in profile.items()) + assert mass == 520704 and 73 * 7168 - mass == 2560 + assert idle == Counter({48: 2400}) + assert dict(profile) == {int(rank): count for rank, count in flow['global_profile'].items()} + return { + 'status': 'PASS_LITERAL_PATTERN_FLOW_AND_ALL_36288_OWNERS', + 'source_windows_reconstructed': parent is not None, + 'roles': len(owners), 'physical_bins': len(slots), 'window_count': len(windows), + 'patterns': pattern_count, 'rank_mass': mass, 'deficit': 2560, + 'helper_width_inventory': dict(sorted(Counter(widths).items())), + 'idle_profile': dict(sorted(idle.items())), 'global_profile': dict(sorted(profile.items())), + } + + +def controls(flow, owners, plan, word, parent, word_hash): + def wrong_flow(candidate): + pattern = next(p for w in candidate[0]['windows'] for p in w['patterns'] if p['flows']) + pattern['flows'][0][2] += 1 + + def wrong_count(candidate): + next(p for w in candidate[0]['windows'] for p in w['patterns'])['count'] += 1 + + def outside_window(candidate): + row = candidate[1][0] + times = candidate[2]['windows'][f"{row['owner_replica']},{row['child']}"][1] + time = next(t for t in range(72) if t not in times) + row['time'] = time + row['stage'], row['replica'] = candidate[2]['order'][time] + + mutations = { + 'call outside owner window': outside_window, + 'duplicate helper role': lambda c: c[1].__setitem__(-1, copy.deepcopy(c[1][0])), + 'wrong native residual width': lambda c: c[1][0].__setitem__('width', c[1][0]['width'] + 1), + 'noncompact prefix': lambda c: c[1][0].__setitem__('offset', c[1][0]['offset'] + 1), + 'wrong physical family': lambda c: c[1][0].__setitem__('family', c[1][0]['family'] + 1), + 'wrong source window capacity': lambda c: next(iter(c[2]['windows'].values())).__setitem__(0, 49), + 'wrong flow multiplicity': wrong_flow, + 'wrong pattern multiplicity': wrong_count, + } + rejected = [] + for name, mutate in mutations.items(): + candidate = [copy.deepcopy(flow), copy.deepcopy(owners), copy.deepcopy(plan)] + mutate(candidate) + try: + audit(*candidate, word, parent, word_hash) + except (AssertionError, KeyError): + rejected.append(name) + else: + raise AssertionError(f'accepted negative control: {name}') + return rejected + + + +def slots_from_flow(pattern): + count, widths = pattern['count'], pattern['widths'] + sockets = len(widths) + if not sockets: + assert not pattern['flows'] + return [[] for _ in range(count)] + flows = {(time, width): n for time, width, n in pattern['flows']} + assert len(flows) == len(pattern['flows']) + times = sorted({time for time, width in flows}) + assert len(times) >= sockets + assert all(sum(n for (t, w), n in flows.items() if t == time) <= count for time in times) + size = len(times) + time_index = {time: index for index, time in enumerate(times)} + edges = [[0] * size for _ in range(size)] + for width in sorted(set(widths)): + available = deque((time, n) for (time, w), n in sorted(flows.items()) if w == width) + for socket, socket_width in enumerate(widths): + if socket_width != width: + continue + left = count + while left: + time, n = available.popleft() + amount = min(left, n) + edges[socket][time_index[time]] += amount + left -= amount + if n > amount: + available.appendleft((time, n - amount)) + assert not available + + # Equal degrees allow one perfect matching for each physical bin. + deficits = deque() + for time in range(size): + missing = count - sum(edges[socket][time] for socket in range(sockets)) + if missing: + deficits.append((time, missing)) + for socket in range(sockets, size): + left = count + while left: + time, n = deficits.popleft() + amount = min(left, n) + edges[socket][time] += amount + left -= amount + if n > amount: + deficits.appendleft((time, n - amount)) + assert not deficits + assert all(sum(row) == count for row in edges) + assert all(sum(edges[i][j] for i in range(size)) == count for j in range(size)) + slots = [] + for _ in range(count): + matched = [-1] * size + + def augment(socket, seen): + for time, multiplicity in enumerate(edges[socket]): + if not multiplicity or time in seen: + continue + seen.add(time) + if matched[time] < 0 or augment(matched[time], seen): + matched[time] = socket + return True + return False + + complete = all(augment(socket, set()) for socket in range(size)) + assert complete, 'regular bipartite graph has no perfect matching' + slot = [] + for time, socket in enumerate(matched): + assert socket >= 0 and edges[socket][time] > 0 + edges[socket][time] -= 1 + if socket < sockets: + slot.append([times[time], widths[socket]]) + slot.sort() + assert len(slot) == sockets and len({time for time, width in slot}) == sockets + slots.append(slot) + assert not any(any(row) for row in edges) + actual = Counter((time, width) for slot in slots for time, width in slot) + assert actual == Counter(flows) + return slots + +def canonical(value): + return json.dumps(value, sort_keys=True, separators=(',', ':')).encode() + + +def digest(value): + return hashlib.sha256(canonical(value)).hexdigest() + + +def expand(flow, plan): + widths = flow['helper_widths'] + pools = defaultdict(deque) + for time in range(72): + for helper, width in enumerate(widths): + pools[time, width].append(helper) + assert flow['fixed_partial_bins'] == [] + owners = [] + + def emit(owner, child, port, slot): + family = owner * 896 + (2 * plan['pairs'][str(child)] + plan['sides'][str(child)]) * 112 + port + offset = 0 + for time, width in sorted(slot): + helper = pools[time, width].popleft() + stage, replica = plan['order'][time] + owners.append(dict(child=child, copy=owner * 112 + port, family=family, + helper=helper, offset=offset, owner_replica=owner, port=port, + replica=replica, stage=stage, time=time, width=width)) + offset += width + + for window in flow['windows']: + owner, child = map(int, window['key'].split(',')) + ports = deque(range(112)) + for pattern in window['patterns']: + for slot in slots_from_flow(pattern): + emit(owner, child, ports.popleft(), slot) + assert not ports + assert not any(pools.values()) + return sorted(owners, key=lambda row: (row['time'], row['helper'])) + + +def main(): + parser = argparse.ArgumentParser(description=__doc__) + parser.add_argument('--word', type=Path, required=True) + parser.add_argument('--plan', type=Path, required=True) + parser.add_argument('--out', type=Path, required=True) + parser.add_argument('--source-template', type=Path) + args = parser.parse_args() + flow_path = Path(__file__).resolve().parent / 'data' / 'OWNER-FLOW.json.gz' + flow, word, plan = map(read, (flow_path, args.word, args.plan)) + parent = read(args.source_template) if args.source_template else None + word_hash = hashlib.sha256(raw(args.word)).hexdigest() + assert word_hash == '096ed4abee4bb02c0d6f03bb8191510e57f6b1fcd34e8644f4fd336254e46411' + assert flow['schema'] == 'frozen-rank7-owner-flow/1' + owners = expand(flow, plan) + report = audit(flow, owners, plan, word, parent, word_hash) + payload = canonical(owners) + owner_hash = hashlib.sha256(payload).hexdigest() + assert owner_hash == '02bde66a90b8ae9b0c4d8d563ca2218ad1de851fa6dec3d1abb5d3a30ca3dd2a' + assert owner_hash == flow['owner_content_sha256'] + report['rejected_controls'] = controls(flow, owners, plan, word, parent, word_hash) + report['input_content_sha256'] = { + 'flow': hashlib.sha256(raw(flow_path)).hexdigest(), + 'word': word_hash, 'plan': hashlib.sha256(raw(args.plan)).hexdigest(), + } + if args.source_template: + report['input_content_sha256']['source_template'] = hashlib.sha256(raw(args.source_template)).hexdigest() + report['owner_content_sha256'] = owner_hash + report['geometry_sha256'] = flow['geometry_sha256'] + args.out.mkdir(parents=True, exist_ok=True) + with (args.out / 'OWNERS.json.gz').open('wb') as stream: + with gzip.GzipFile(filename='', mode='wb', fileobj=stream, mtime=0) as archive: + archive.write(payload) + (args.out / 'OWNER-CERTIFICATE.json').write_text(json.dumps(report, indent=2) + '\n') + print(json.dumps(report, indent=2)) + + +if __name__ == '__main__': + main() diff --git a/research/complex-rank7-end-pair/build_word.py b/research/complex-rank7-end-pair/build_word.py new file mode 100644 index 0000000000..915610f40d --- /dev/null +++ b/research/complex-rank7-end-pair/build_word.py @@ -0,0 +1,68 @@ +"""Apply the pinned twelve-frame recipe to the retained PR869 U8 word. + +Prepared with OpenAI Codex assistance. Apache-2.0. +""" +import argparse +import gzip +import hashlib +import json +from pathlib import Path + + +BASE_SHA = '090fd574747fbf9628e70cb9beb792732914cfa52e1df147579fe4dbccc68c5a' +WORD_SHA = '096ed4abee4bb02c0d6f03bb8191510e57f6b1fcd34e8644f4fd336254e46411' +RECIPE = Path(__file__).resolve().parent / 'data/END-RECIPE.json' + + +def require(condition, message): + if not condition: + raise ValueError(message) + + +def build(source, recipe): + raw = gzip.decompress(source.read_bytes()) + require(hashlib.sha256(raw).hexdigest() == recipe['input_sha256'] == BASE_SHA, + 'baseline content pin') + require(recipe['schema'] == 1, 'recipe schema') + word = json.loads(raw) + require([q for q, _, _ in recipe['ends']] == [588, 589], 'end sites') + require([i for i, _, _ in recipe['B_frames']] == + [1018, 1019, 1050, 1051, 1130, 1131, 1162, 1163], 'B-frame sites') + require([i for i, _, _, _ in recipe['early_cleanup']] == [148, 149], 'tail sites') + for q, old, new in recipe['ends']: + require(word['helper_final'][str(q)] == old, 'old helper end') + word['helper_final'][str(q)] = new + for index, old, new in recipe['B_frames']: + require(word['B'][index][-1] == old, 'old B frame') + word['B'][index][-1] = new + for index, cleanup_index, old, new in recipe['early_cleanup']: + require(word['early_cleanup'][index] == [cleanup_index, old], 'old tail frame') + word['early_cleanup'][index][1] = new + for key in ('local_profile', 'local_rank_mass', 'width_inventory'): + word['report'][key] = recipe['expected'][key] + result = json.dumps(word, sort_keys=True, separators=(',', ':')).encode() + require(hashlib.sha256(result).hexdigest() == recipe['expected']['output_sha256'] == WORD_SHA, + 'constructed word content pin') + return result + + +def main(): + parser = argparse.ArgumentParser(description=__doc__) + parser.add_argument('--input', type=Path, required=True) + parser.add_argument('--recipe', type=Path, default=RECIPE) + parser.add_argument('--output', type=Path, required=True) + args = parser.parse_args() + recipe = json.loads(args.recipe.read_text(encoding='utf-8')) + raw = build(args.input, recipe) + compressed = gzip.compress(raw, compresslevel=9, mtime=0) + # Python versions differ in the gzip OS header byte. + compressed = compressed[:9] + b'\xff' + compressed[10:] + args.output.parent.mkdir(parents=True, exist_ok=True) + args.output.write_bytes(compressed) + print(json.dumps(dict(status='PASS_PINNED_TWELVE_FRAME_CONSTRUCTION', + content_sha256=WORD_SHA, compressed_sha256=hashlib.sha256(compressed).hexdigest(), + changed_annotations=12)), flush=True) + + +if __name__ == '__main__': + main() diff --git a/research/complex-rank7-end-pair/code/admit_candidate.py b/research/complex-rank7-end-pair/code/admit_candidate.py new file mode 100644 index 0000000000..18c47e0e54 --- /dev/null +++ b/research/complex-rank7-end-pair/code/admit_candidate.py @@ -0,0 +1,188 @@ +"""Admit a fresh native candidate with unchanged predecessor predicates. + +The prior full ancestor replay is reused only after its receipt is verified. +No source package is copied or edited; only candidate data are prepared. +""" +import argparse +from fractions import Fraction +import gzip +import hashlib +import json +import os +from pathlib import Path +import subprocess +import sys +import time +from prepare_candidate import add_native_arguments, selected_dispatch, selected_native_directories + + +HERE = Path(__file__).resolve().parent + + +def read(path): + raw = path.read_bytes() + return json.loads(gzip.decompress(raw) if path.suffix == '.gz' else raw) + + +def sha(path): + return hashlib.sha256(path.read_bytes()).hexdigest() + + +def dump(path, value): + path.write_text(json.dumps(value, indent=2) + '\n') + + +def need(value, message): + if not value: + raise ValueError(message) + + +def verify_manifest(package): + actual = {p.relative_to(package).as_posix(): sha(p) for p in package.rglob('*') + if p.is_file() and p != package / 'MANIFEST.json' and '__pycache__' not in p.parts} + need(actual == read(package / 'MANIFEST.json'), 'Changed source package: ' + str(package)) + + +def verify_receipt(directory): + receipt = read(directory / 'RECEIPT.json') + for name, expected in receipt['artifacts'].items(): + need(sha(directory / name) == expected, 'Changed replay artifact: ' + str(directory / name)) + return receipt + + +def main(): + ap = argparse.ArgumentParser(description=__doc__) + for name in ('ancestor-package', 'ancestor-replay', 'dependency', 'checker-package', 'out'): + ap.add_argument('--' + name, type=Path, required=True) + add_native_arguments(ap) + ap.add_argument('--owners', type=Path, help='Specified gzipped owner rows for the fresh native widths') + ap.add_argument('--comparison-record', type=Path, required=True, + help='Pinned selected-result record used for the strict improvement comparison') + args = ap.parse_args() + ancestor, replay, dep, checker, out = [getattr(args, name).resolve() for name in + ('ancestor_package', 'ancestor_replay', 'dependency', 'checker_package', 'out')] + natives = selected_native_directories(args) + dispatch = selected_dispatch(args, natives) + need(__debug__, 'Assertions must remain enabled') + comparison_path = args.comparison_record.resolve() + comparison_sha = sha(comparison_path) + comparison = read(comparison_path) + need(Fraction(comparison['kappa']) == Fraction(comparison['decimal']), 'Comparison record value') + need(not out.exists(), 'Use a new output directory') + need(not any(out.is_relative_to(p) for p in (ancestor, replay, dep, checker, *natives.values())), + 'Output must be separate from input artifacts and source packages') + for package in (ancestor, dep, checker): + verify_manifest(package) + ancestor_receipt = verify_receipt(replay) + need(ancestor_receipt['manifest_sha256'] == sha(ancestor / 'MANIFEST.json'), 'Ancestor source binding') + need(read(replay / 'RESULT.json')['status'] == + 'PASS_FRESH_SW4_EIGHT_FRAMES_COMPLETE_NATIVE_OWNER_AND_EXACT_GLOBAL_REPLAY', 'Full ancestor replay required') + need(read(replay / 'ANCESTRAL-LINK.json')['status'] == + 'PASS_FRESH_ANCESTRAL_INPUTS_MATCH_PINNED_BASELINE', 'Fresh ancestor semantic linkage required') + need((ancestor / 'data/UPSTREAM-PINS.json').read_bytes() == (dep / 'data/UPSTREAM-PINS.json').read_bytes(), + 'Global ancestry differs from the replayed predecessor') + for native in natives.values(): + verify_receipt(native) + need(read(native / 'RESULT.json')['status'] == 'PASS_COMPLETE_NATIVE_REPLAY_UNCHANGED_CHECKERS', + 'Fresh native pipeline required') + for key, package in (('ancestor_manifest', ancestor), ('dependency_manifest', dep)): + need(read(native / 'RESULT.json')['input_sha256'][key] == sha(package / 'MANIFEST.json'), 'Native source binding') + out.mkdir(parents=True) + root = ancestor / 'vendor/main' + env = dict(os.environ, PYTHONDONTWRITEBYTECODE='1', PYTHONUTF8='1', PYTHONINTMAXSTRDIGITS='0') + for key in ('PYTHONOPTIMIZE', 'WITNESS_SPEC', 'WITNESS_DISCOVER', 'WITNESS_RECORD'): + env.pop(key, None) + stages = [] + started = time.monotonic() + dump(out / 'ANCESTRY-REUSE.json', {'status': 'PASS_HASH_VERIFIED_FRESH_ANCESTRY_REUSE', + 'ancestor_result_sha256': sha(replay / 'RESULT.json'), + 'ancestor_receipt_sha256': sha(replay / 'RECEIPT.json'), + 'ancestral_link_sha256': sha(replay / 'ANCESTRAL-LINK.json'), + 'upstream_pins_sha256': sha(dep / 'data/UPSTREAM-PINS.json'), + 'native_receipt_sha256': {pid: sha(n / 'RECEIPT.json') for pid, n in natives.items()}, + 'source_manifests': {str(p): sha(p / 'MANIFEST.json') for p in (ancestor, dep, checker)}, + 'scope': 'Unchanged ancestral circuits and literal geometry were freshly replayed once. The changed native word and full global admission are checked separately.'}) + + def run(name, script, *arguments): + print('RUN', name, flush=True) + cmd = [sys.executable, '-B', str(script), *map(str, arguments)] + log = out / (name + '.log') + before = time.monotonic() + with log.open('w') as stream: + result = subprocess.run(cmd, stdout=stream, stderr=subprocess.STDOUT, env=env) + stages.append({'name': name, 'command': cmd, 'script_sha256': sha(script), + 'exit_code': result.returncode, 'seconds': time.monotonic() - before, + 'log_sha256': sha(log)}) + dump(out / 'STAGES.json', stages) + if result.returncode: + raise RuntimeError(name + ' failed:\n' + log.read_text()[-8000:]) + print('PASS', name, flush=True) + + witness = out / 'witness' + owner_args = ['--owners', args.owners.resolve()] if args.owners else [] + if args.native: + native_args = ['--native', args.native.resolve()] + else: + frozen_dispatch = out / 'DISPATCH.json' + dump(frozen_dispatch, [[s, r, dispatch[s, r]] for s in range(9) for r in range(8)]) + native_args = [part for pid, path in sorted(natives.items()) + for part in ('--native-provider', pid + '=' + str(path))] + native_args += ['--dispatch', frozen_dispatch] + run('prepare-data', HERE / 'prepare_candidate.py', '--dependency', dep, *native_args, + '--out', witness, *owner_args) + plan, owners, spec = [witness / 'data' / name for name in ('PLAN.json', 'OWNERS.json.gz', 'SPEC.json')] + provider_args = [part for pid, path in sorted(natives.items()) + for part in ('--provider', pid + '=' + str(path / 'local'))] + global_args = ['--main', root, '--dependency', dep, '--plan', plan, '--owners', owners, + '--spec', spec, *provider_args] + run('discovery', checker / 'verify_global.py', *global_args, '--work', out / 'discovery', '--discover') + discovery_spec_sha = sha(spec) + run('freeze-spec', checker / 'freeze_spec.py', '--spec', spec, '--discovery', out / 'discovery') + frozen_spec_sha = sha(spec) + need(discovery_spec_sha != frozen_spec_sha, 'Discovery values must be frozen before strict replay') + run('strict-global', checker / 'verify_global.py', *global_args, '--work', out / 'global-replay') + run('independent-owner-moment', checker / 'audit_owners.py', '--dependency', dep, '--plan', plan, + '--owners', owners, '--spec', spec, *provider_args, + '--output', out / 'INDEPENDENT-OWNERS.json') + run('independent-two-engine-outer47', dep / 'independent/price_profile.py', '--main', root, + '--profile', plan, '--output', out / 'INDEPENDENT-PRICE.json') + need(sha(spec) == frozen_spec_sha, 'Frozen spec changed during strict verification') + assembly = read(out / 'global-replay/ASSEMBLY.json') + expected = read(spec)['expected'] + owner_audit = read(out / 'INDEPENDENT-OWNERS.json') + price = read(out / 'INDEPENDENT-PRICE.json') + kappa = expected['kappa'] + need(kappa == assembly['kappa'] == owner_audit['kappa'] == price['prices']['native']['outer_with_bit']['kappa'], + 'Independent kappa witnesses disagree') + baseline_kappa = comparison['kappa'] + need(Fraction(kappa) > Fraction(baseline_kappa), 'A strictly improved kappa is required') + need(len(owner_audit['negative_controls']) == 7 and owner_audit['roles'] == 36288, 'Independent owner census') + for mode in ('native', 'conservative'): + need(price['prices'][mode]['outer_with_bit']['strict_constraints_passed'] == 47, 'All outer constraints') + admission = read(out / 'global-replay/ADMISSION-CONTROLS.json') + discovery_controls = read(out / 'discovery/ADMISSION-CONTROLS.json') + census = discovery_controls['rejected_controls'] + discovery_controls['discovery_accepted_controls'] + need(len(census) == len(set(census)) == expected['assembly_controls_rejected'] + and set(admission['rejected_controls']) == set(census) + and len(admission['rejected_controls']) == len(census) and not admission['discovery_accepted_controls'], + 'Strict admission mutation census') + bit = read(out / 'global-replay/BIT-SUPPLIER-CONTROLS.json') + need(len(bit['rejected_controls']) == 12 and bit['derived_deficit'] == 10272, 'Bit controls') + result = {'status': 'PASS_STRICTLY_IMPROVED_KAPPA_UNCHANGED_ADMISSION', 'kappa': kappa, + 'kappa_decimal': assembly['kappa_decimal'], 'baseline_kappa': baseline_kappa, + 'comparison_record_sha256': comparison_sha, + 'gain': str(Fraction(kappa) - Fraction(baseline_kappa)), 'complex_lower': assembly['complex_lower'], + 'bit_lower': assembly['bit_lower'], 'global_profile': assembly['complex_profile'], + 'seconds': time.monotonic() - started, 'stages': stages, + 'scope': 'Finite conditional native exact-complex witness. Hash-verified reuse of a fresh ancestral replay; full changed-native and global admission. Cited PR589 bit circuit is repriced with its surcharge and controls. All inherited all-size and analytic-transfer assumptions remain.'} + need(sha(comparison_path) == comparison_sha, 'Comparison record changed during admission') + dump(out / 'RESULT.json', result) + retained = [p for p in out.rglob('*') if p.is_file() and p.name != 'RECEIPT.json'] + dump(out / 'RECEIPT.json', {'status': result['status'], + 'native_receipt_sha256': {pid: sha(n / 'RECEIPT.json') for pid, n in natives.items()}, + 'artifacts': {p.relative_to(out).as_posix(): sha(p) for p in retained}}) + print(result['status'], result['kappa_decimal'], flush=True) + + +if __name__ == '__main__': + main() diff --git a/research/complex-rank7-end-pair/code/prepare_candidate.py b/research/complex-rank7-end-pair/code/prepare_candidate.py new file mode 100644 index 0000000000..152ca5f5f8 --- /dev/null +++ b/research/complex-rank7-end-pair/code/prepare_candidate.py @@ -0,0 +1,239 @@ +"""Bind fresh native artifacts and specified owners to the fixed #799 geometry. + +Data preparation only; unchanged dependency checkers independently admit every +result. Adapted from PR777 prepare_plan.py, with OpenAI Codex assistance. +""" +import argparse +from collections import Counter, defaultdict +import copy +import gzip +import hashlib +import importlib.util +import json +from pathlib import Path +import re +import shutil + + +def read(path): + raw = path.read_bytes() + return json.loads(gzip.decompress(raw) if path.suffix == '.gz' else raw) + + +def sha(path): + return hashlib.sha256(path.read_bytes()).hexdigest() + + +def dump(path, value): + path.write_text(json.dumps(value, indent=2) + '\n') + + +def need(value, message): + if not value: + raise ValueError(message) + + +def verify_manifest(package): + actual = {p.relative_to(package).as_posix(): sha(p) for p in package.rglob('*') + if p.is_file() and p != package / 'MANIFEST.json' and '__pycache__' not in p.parts} + need(actual == read(package / 'MANIFEST.json'), 'Dependency source manifest') + + +def add_native_arguments(parser): + selection = parser.add_mutually_exclusive_group(required=True) + selection.add_argument('--native', type=Path, help='One complete native replay, named gauge44') + selection.add_argument('--native-provider', action='append', metavar='PID=DIR', + help='Named complete native replay; repeat for mixed providers') + parser.add_argument('--dispatch', type=Path, help='JSON array of 72 [stage, replica, provider] rows') + + +def selected_native_directories(args): + if args.native is not None: + need(args.dispatch is None, 'Use named providers for explicit dispatch') + return {'gauge44': args.native.resolve()} + result = {} + for item in args.native_provider: + pid, separator, directory = item.partition('=') + need(separator and re.fullmatch(r'[a-z][a-z0-9_]*', pid) and pid not in result, + 'Unique simple provider names required') + result[pid] = Path(directory).resolve() + need(args.dispatch is not None, 'Explicit dispatch required for named providers') + return result + + +def selected_dispatch(args, providers): + rows = read(args.dispatch) if args.dispatch else [[s, r, 'gauge44'] for s in range(9) for r in range(8)] + need(type(rows) is list and len(rows) == 72, 'Exactly 72 dispatch rows required') + result = {} + for row in rows: + need(type(row) is list and len(row) == 3, 'Dispatch row shape') + stage, replica, pid = row + need(type(stage) is int and type(replica) is int and type(pid) is str + and 0 <= stage < 9 and 0 <= replica < 8 and pid in providers, 'Dispatch row domain') + need((stage, replica) not in result, 'Duplicate dispatch invocation') + result[stage, replica] = pid + need(set(result) == {(s, r) for s in range(9) for r in range(8)}, 'Complete dispatch invocation census') + need(set(result.values()) == set(providers), 'Every named provider must be used') + return result + + +def main(): + ap = argparse.ArgumentParser(description=__doc__) + for name in ('dependency', 'out'): + ap.add_argument('--' + name, type=Path, required=True) + add_native_arguments(ap) + ap.add_argument('--owners', type=Path, help='Explicit gzipped owner rows; defaults to the dependency map') + args = ap.parse_args() + dep, out = [getattr(args, name).resolve() for name in ('dependency', 'out')] + natives = selected_native_directories(args) + dispatch = selected_dispatch(args, natives) + owners_path = (args.owners or dep / 'data/OWNERS.json.gz').resolve() + need(owners_path.suffix == '.gz', 'Supply a gzipped JSON owner map') + owners_blob = owners_path.read_bytes() + owners_sha256 = hashlib.sha256(owners_blob).hexdigest() + need(not out.exists(), 'New output directory required') + need(not out.is_relative_to(dep) and not any(out.is_relative_to(n) for n in natives.values()), + 'Output must be separate') + verify_manifest(dep) + plan = read(dep / 'data/PLAN.json') + original = copy.deepcopy(plan) + need(set(original['provider_registry']) == {'gauge44'}, 'Pinned single-provider source geometry') + module_spec = importlib.util.spec_from_file_location('identity', dep / 'code/generated_identity.py') + identity = importlib.util.module_from_spec(module_spec) + module_spec.loader.exec_module(identity) + widths_by_provider, paid_by_provider = {}, {} + plan['provider_registry'] = {} + for pid, native in sorted(natives.items()): + result = read(native / 'RESULT.json') + need(result['status'] == 'PASS_COMPLETE_NATIVE_REPLAY_UNCHANGED_CHECKERS', 'Complete native replay required') + need(result['input_sha256']['dependency_manifest'] == sha(dep / 'MANIFEST.json'), 'Native dependency binding') + for relative, expected in read(native / 'RECEIPT.json')['artifacts'].items(): + need(sha(native / relative) == expected, 'Native output changed: ' + relative) + local = native / 'local' + canonical = read(local / 'CANONICAL-WORD.json.gz') + programs = read(local / 'LOCAL-PROGRAMS.json.gz') + checks = read(local / 'LOCAL-CHECKS.json') + need((canonical['v'], canonical['h'], canonical['R']) == (112, 9, 504), 'Unchanged native geometry') + desc = copy.deepcopy(original['provider_registry']['gauge44']) + widths = [len(canonical['frames'][canonical['final'][h]]) - len(canonical['frames'][canonical['start'][h]]) + for h in range(224, 728)] + need(canonical['ports'] == desc['ports'] and canonical['centers'] == desc['centers'], + 'Unchanged ports and centers') + need(len(widths) == 504 and all(type(w) is int and 0 < w <= 9 for w in widths), + 'Valid fresh native residual widths') + paid = Counter(event[-1] for event in programs['forward'] if event[0] in ('climb', 'center')) + need(paid == Counter(event[-1] for event in programs['backward'] if event[0] in ('climb', 'center')), + 'Both orientation profiles agree') + need(dict(paid) == {int(r): n for r, n in result['profile'].items()}, 'Independent native profile') + desc['artifact_semantic_sha256'] = identity.artifact_identities(local) + desc['artifact_sha256'] = {name: sha(local / name) for name in desc['artifact_sha256']} + desc['actual_paid_profile'] = dict(sorted(paid.items())) + desc['widths'] = widths + desc['orientation_program_sha256'] = {orientation: checks[orientation]['sha256'] for orientation in checks} + desc['directory'] = 'providers/' + pid + plan['provider_registry'][pid] = desc + widths_by_provider[pid], paid_by_provider[pid] = widths, paid + for call in plan['provider_calls']: + call['provider'] = dispatch[call['stage'], call['replica']] + call['local_program_sha256'] = plan['provider_registry'][call['provider']]['orientation_program_sha256'][call['orientation']] + need(len(plan['per_call_widths']) == 72, 'Complete per-call width inventory') + need({(c['stage'], c['replica']) for c in plan['per_call_widths']} == + {(s, r) for s in range(9) for r in range(8)}, 'Complete per-call width identities') + for call in plan['per_call_widths']: + call['widths'] = widths_by_provider[dispatch[call['stage'], call['replica']]].copy() + rows = json.loads(gzip.decompress(owners_blob)) + used = defaultdict(set) + helpers = defaultdict(set) + families = defaultdict(set) + calls = {call['time']: call for call in plan['provider_calls']} + need(set(calls) == set(range(72)), 'Complete invocation chronology') + stage_pair = {0: 0, 1: 0, 2: 0, 3: 1, 4: 1, 5: 2, 6: 2, 7: 3, 8: 3} + for row in rows: + need(all(type(row[k]) is int for k in ('time', 'helper', 'stage', 'replica', 'width', + 'owner_replica', 'child', 'port', 'offset', 'family', 'copy')), + 'Integer owner role fields') + time, helper = row['time'], row['helper'] + need(type(time) is int and time in calls and type(helper) is int and 0 <= helper < 504, + 'Owner role domain') + call = calls[time] + need((row['stage'], row['replica']) == (call['stage'], call['replica']) == tuple(plan['order'][time]), + 'Owner invocation identity') + need(row['width'] == widths_by_provider[call['provider']][helper], + 'Owner width equals the fresh native footprint') + replica, child, port, offset = [row[k] for k in ('owner_replica', 'child', 'port', 'offset')] + need(all(type(v) is int for v in (replica, child, port, offset)) + and 0 <= replica < 8 and 0 <= port < 112, 'Physical owner domain') + window_key = f'{replica},{child}' + need(window_key in plan['windows'], 'Known physical owner window') + capacity, times = plan['windows'][window_key] + need(time in times and 0 <= offset and offset + row['width'] <= capacity, + 'Owner footprint fits its invocation window') + pair, side = plan['pairs'][str(child)], plan['sides'][str(child)] + family = replica * 896 + (2 * pair + side) * 112 + port + need(row['family'] == family and row['copy'] == replica * 112 + port, 'Physical family identity') + need((replica, pair) != (row['replica'], stage_pair[row['stage']]), 'Owner does not alias active data') + need(helper not in helpers[time] and family not in families[time], 'Unique helper and family per invocation') + helpers[time].add(helper) + families[time].add(family) + key = replica, child, port + cells = set(range(row['offset'], row['offset'] + row['width'])) + need(not used[key] & cells, 'Nonoverlapping owned coordinates') + used[key].update(cells) + need(len(rows) == 36288 and all(helpers[t] == set(range(504)) for t in range(72)), + 'Every native helper has exactly one owner per invocation') + idle = Counter() + for key, (capacity, window) in plan['windows'].items(): + replica, child = map(int, key.split(',')) + for port in range(112): + occupied = used[replica, child, port] + need(occupied == set(range(len(occupied))) and len(occupied) <= capacity, 'Compacted owner prefix') + if len(occupied) < capacity: + idle[capacity - len(occupied)] += 1 + profile = Counter() + for call in calls.values(): + profile.update(paid_by_provider[call['provider']]) + profile.update(idle) + need(sum(rank * count for rank, count in profile.items()) == 520704, 'Global rank mass') + plan['profile'] = dict(sorted(profile.items())) + plan['assignment_sha256'] = owners_sha256 + need(all(plan[key] == original[key] for key in original + if key not in ('profile', 'provider_calls', 'provider_registry', 'per_call_widths', 'assignment_sha256')), + 'Inherited plan structure changed') + out.mkdir(parents=True) + (out / 'data').mkdir() + dump(out / 'data/PLAN.json', plan) + (out / 'data/OWNERS.json.gz').write_bytes(owners_blob) + old_spec = read(dep / 'data/SPEC.json') + spec = {key: copy.deepcopy(value) for key, value in old_spec.items() + if key not in ('data', 'providers', 'expected', 'name')} + spec.update(name='joint-frame-candidate', + data={'plan_sha256': sha(out / 'data/PLAN.json'), + 'owners_sha256': sha(out / 'data/OWNERS.json.gz')}, + providers={pid: {} for pid in sorted(natives)}, + dispatch=[[s, r, dispatch[s, r]] for s in range(9) for r in range(8)], + expected={'gain_baseline_kappa': old_spec['expected']['gain_baseline_kappa']}) + pivot_path = spec['pivot_swap']['recipe'] + (out / pivot_path).parent.mkdir(parents=True, exist_ok=True) + shutil.copyfile(dep / pivot_path, out / pivot_path) + need(sha(out / pivot_path) == spec['pivot_swap']['sha256'], 'Retained pivot recipe binding') + dump(out / 'data/SPEC.json', spec) + dump(out / 'REBINDING.json', {'status': 'PREPARED_FOR_UNCHANGED_GLOBAL_ADMISSION', + 'dependency_manifest_sha256': sha(dep / 'MANIFEST.json'), + 'dependency_plan_sha256': sha(dep / 'data/PLAN.json'), + 'native_receipt_sha256': {pid: sha(n / 'RECEIPT.json') for pid, n in natives.items()}, + 'plan_sha256': sha(out / 'data/PLAN.json'), 'owners_sha256': sha(out / 'data/OWNERS.json.gz'), + 'source_owners_sha256': sha(dep / 'data/OWNERS.json.gz'), + 'native_widths': widths_by_provider, + 'width_changes': {pid: [[h, old, new] for h, (old, new) in + enumerate(zip(original['provider_registry']['gauge44']['widths'], widths)) if old != new] + for pid, widths in widths_by_provider.items()}, + 'local_profiles': {pid: dict(paid) for pid, paid in paid_by_provider.items()}, + 'provider_counts': dict(Counter(dispatch.values())), 'idle_profile': dict(idle), 'global_profile': dict(profile), + 'changed_plan_fields': ['profile', 'provider_calls.provider', 'provider_calls.local_program_sha256', 'provider_registry', + 'per_call_widths.widths', 'assignment_sha256'], + 'scope': 'Data rebinding only. Complete independent native replay already passed. Discovery, strict global admission, independent owners/moments and outer checks are required separately.'}) + print('PREPARED_CANDIDATE', json.dumps(dict(profile))) + + +if __name__ == '__main__': + main() diff --git a/research/complex-rank7-end-pair/code/replay_native.py b/research/complex-rank7-end-pair/code/replay_native.py new file mode 100644 index 0000000000..fd7a9b6530 --- /dev/null +++ b/research/complex-rank7-end-pair/code/replay_native.py @@ -0,0 +1,184 @@ +"""Run unchanged native checks on an explicit raw complex supplier word.""" +import argparse +import gzip +import hashlib +import json +import os +from pathlib import Path +import subprocess +import sys +import time + + +def read(path): + raw = path.read_bytes() + return json.loads(gzip.decompress(raw) if path.suffix == '.gz' else raw) + + +def sha(path): + return hashlib.sha256(path.read_bytes()).hexdigest() + + +def payload_sha(path): + return hashlib.sha256(gzip.decompress(path.read_bytes())).hexdigest() + + +def dump(path, data): + path.write_text(json.dumps(data, indent=2) + '\n') + + +def need(value, message): + if not value: + raise ValueError(message) + + +def cold_control_census(word): + meta = word['refactored_cleanup'] + registers = lambda gate: gate[1:3] if gate[0] == 'add' else gate[1:2] + entering = list(map(int, meta.get('entering_low_frame_ended_helpers', []))) + premix = word['A'][:meta['premix_gates']] + cold = list(map(int, meta['cold_low_frame_ended_helpers'])) + hyperplanes = [helper for helper in cold if len(word['helper_final'][str(helper)]) == 8] + absorptions = meta.get('forward_absorption_A_indices', []) + ended = set(map(int, word['helper_final'])) + tail = [(index, word['cleanup'][index], frame) for index, frame in word['early_cleanup']] + accumulators = set() + for position, (_, gate, frame) in enumerate(tail): + if (gate[0] == 'add' and gate[1] not in ended and gate[2] not in ended and len(frame) < 8 + and any(later[0] == 'add' and later[2] == gate[1] and later[1] in hyperplanes + for _, later, _ in tail[position + 1:])): + accumulators.add(gate[1]) + return {'common': 14, 'entering_ends': 3 * len(entering), + 'entering_premix': 3 * sum(any(helper in registers(gate) for gate in premix) for helper in entering), + 'premix_target_writes': 5 if meta.get('premix_target_writes', 0) else 0, + 'forward_absorptions': 7 if absorptions else 0, + 'shared_anchor_and_release': 5 if hyperplanes and absorptions else 0, + 'dimension_eight_ends': 3 * len(hyperplanes), 'factored_accumulators': 2 * len(accumulators)} + + +def main(): + ap = argparse.ArgumentParser(description=__doc__) + for name in ('ancestor-package', 'dependency', 'raw', 'out'): + ap.add_argument('--' + name, type=Path, required=True) + ap.add_argument('--expected-local-pins', type=Path) + ap.add_argument('--finalize-completed', action='store_true', + help='Validate existing successful stage logs and finalize their receipt without rerunning checks') + args = ap.parse_args() + need(__debug__, 'Assertions must remain enabled') + ancestor, dep, raw, out = [getattr(args, name).resolve() + for name in ('ancestor_package', 'dependency', 'raw', 'out')] + need(out.is_dir() if args.finalize_completed else not out.exists(), 'Use the required output directory state') + need(not any(out.is_relative_to(p) for p in (ancestor, dep)), 'Output must be outside source packages') + completed_stages = read(out / 'STAGES.json') if args.finalize_completed else [] + if not args.finalize_completed: + out.mkdir(parents=True) + else: + need(not (out / 'RESULT.json').exists() and not (out / 'RECEIPT.json').exists(), + 'Only incomplete receipt finalization may be resumed') + root = ancestor / 'vendor/main' + source = root / 'research/complex-k492-owner-repair/vendor/multicold' + pr635 = ancestor / 'vendor/pr635/research/complex-571-cold14-st2' + base = source / 'vendor/pr558' + parent = base / 'vendor/pr552' + routing = parent / 'complex-local/inputs/phase-baseline/CANONICAL-WORD.json.gz' + env = dict(os.environ, PYTHONDONTWRITEBYTECODE='1', PYTHONUTF8='1', PYTHONINTMAXSTRDIGITS='0', + MULTICOLD_SOURCE=str(source), GAIN_PR564_SOURCE=str(source)) + for key in ('PYTHONOPTIMIZE', 'WITNESS_SPEC', 'WITNESS_DISCOVER', 'WITNESS_RECORD', + 'XPARAM', 'EXTRAH', 'PAIRING_RANKS', 'DIAG', 'HYBBUDGET', 'PR635_CODE', 'SW4_PIN'): + env.pop(key, None) + stages = [] + inputs = {'raw_word_content': payload_sha(raw), 'raw_word_file': sha(raw), + 'ancestor_manifest': sha(ancestor / 'MANIFEST.json'), + 'dependency_manifest': sha(dep / 'MANIFEST.json')} + started = time.monotonic() + + def run(name, script, *arguments, portable=True, extra_env=None): + print('RUN', name, flush=True) + cmd = [sys.executable, '-B'] + ([str(source / 'code/run_portable.py')] if portable else []) + cmd += [str(script), *map(str, arguments)] + log = out / (name + '.log') + if args.finalize_completed: + need(len(stages) < len(completed_stages), 'Missing completed checker stage') + record = completed_stages[len(stages)] + need(record['name'] == name and record['command'] == cmd and record['exit_code'] == 0 + and record['script_sha256'] == sha(script) and record['log_sha256'] == sha(log), + 'Completed checker stage binding: ' + name) + stages.append(record) + print('VERIFIED_COMPLETED', name, flush=True) + return + before = time.monotonic() + with log.open('w') as stream: + result = subprocess.run(cmd, stdout=stream, stderr=subprocess.STDOUT, + env=dict(env, **(extra_env or {}))) + stages.append({'name': name, 'command': cmd, 'script_sha256': sha(script), + 'exit_code': result.returncode, 'seconds': time.monotonic() - before, + 'log_sha256': sha(log)}) + dump(out / 'STAGES.json', stages) + if result.returncode: + raise RuntimeError(name + ' failed:\n' + log.read_text()[-8000:]) + print('PASS', name, flush=True) + + run('ancestor-manifest', ancestor / 'verify_manifest.py', portable=False) + run('dependency-manifest', dep / 'verify_manifest.py', portable=False) + local = out / 'local' + run('independent-raw-audit', dep / 'independent/audit_local.py', '--word', raw, + '--main', root, '--output', out / 'INDEPENDENT-AUDIT.json', portable=False) + run('native', pr635 / 'code/hpe_audit_and_transport.py', '--raw', raw, + '--routing', routing, '--out', local) + run('ledgers', pr635 / 'code/hpe_emit_local.py', local) + run('local-replay', pr635 / 'code/hpe_audit_local.py', local) + run('wrappers', pr635 / 'code/hpe_adapters_local.py', local / 'CANONICAL-WORD.json.gz', + local / 'RESIDUAL-ADAPTERS.json', local / 'WRAPPER-CHECKS.json') + run('cold-end-controls', dep / 'construction/layers_p6n/layer_end_controls.py', raw, + routing, out / 'controls', extra_env={'PR635_CODE': str(pr635 / 'code')}) + run('phases', base / 'code/phases_full.py', local, + parent / 'complex-global/vendor531/vendor/clifford') + need(not args.finalize_completed or len(stages) == len(completed_stages), 'Complete stage census') + audit = read(out / 'INDEPENDENT-AUDIT.json') + need(audit['status'] == 'PASS_INDEPENDENT_EXACT_FORWARD_AND_DUAL_ENDPOINT_FRAMES_ENTRANCES_CONES' + and audit['checked_columns'] == 728 and audit['content_sha256'] == inputs['raw_word_content'], + 'Independent raw audit failed or bound to another input') + native = read(local / 'INDEPENDENT-LOCAL-REPLAY.json') + need(native['status'] == 'PASS_INDEPENDENT_LOCAL_SCALAR_FRAME_AND_WRAPPER_REPLAY' + and len(native['negative_controls_rejected']) == 25, 'Incomplete native controls') + for orientation in ('forward', 'backward'): + need(native['local'][orientation]['all_exact_columns'] == 728 + and native['local'][orientation]['private_work_erased'], 'Native column restoration') + need(native['paid_profiles'][orientation] == audit['local_profile'], 'Native paid profile mismatch') + wrapper = read(local / 'WRAPPER-CHECKS.json') + need(wrapper['status'] == 'PASS_RESIDUAL_WRAPPERS_ARE_EXACT', 'Residual wrapper failure') + control = read(out / 'controls/COLD-END-CONTROLS.json') + cold_census = cold_control_census(read(raw)) + need(control['status'] == 'PASS_COLD_END_RAW_WORD_CONTROLS' + and len(control['rejected']) == len(set(control['rejected'])) == sum(cold_census.values()), + 'Incomplete cold-end controls') + phase = read(local / 'CLIFFORD-REVIEW.json') + need(phase['status'] == 'PASS_NEW_WORD_EXACT_LOCAL_PHASE_RECIPES' + and phase['all_child_equalities_include_global_phase'] is True, 'Incomplete phase equality') + need(phase['canonical_word_sha256'] == sha(local / 'CANONICAL-WORD.json.gz') + and phase['recipe_sha256'] == sha(local / 'CLIFFORD-RECIPES.json'), 'Stale phase binding') + checks = read(local / 'LOCAL-CHECKS.json') + pins = {'canonical_sha256': payload_sha(local / 'CANONICAL-WORD.json.gz'), + 'program_sha256': {o: checks[o]['sha256'] for o in ('forward', 'backward')}, + 'clifford_recipes_sha256': sha(local / 'CLIFFORD-RECIPES.json')} + if args.expected_local_pins: + expected = read(args.expected_local_pins) + need(all(expected[k] == value for k, value in pins.items()), 'Nominated native pins differ') + inputs['expected_local_pins'] = sha(args.expected_local_pins) + need(payload_sha(raw) == inputs['raw_word_content'] and sha(raw) == inputs['raw_word_file'], + 'Raw input changed during native replay') + result = {'status': 'PASS_COMPLETE_NATIVE_REPLAY_UNCHANGED_CHECKERS', 'input_sha256': inputs, + 'profile': audit['local_profile'], 'phase_edges': phase['directed_edges'], + 'pins': pins, 'seconds': sum(stage['seconds'] for stage in stages), 'stages': stages, + 'cold_control_census': cold_census, 'cold_controls_rejected': len(control['rejected']), + 'finalized_from_completed_stages': args.finalize_completed, + 'scope': 'Fresh raw/native forward and dual columns, scalar/frame/cones, wrappers, 25 native controls, every applicable cold-end control, and every exact phase edge. Global admission and construction ancestry remain separate.'} + dump(out / 'RESULT.json', result) + artifacts = [p for p in out.rglob('*') if p.is_file() and p.name != 'RECEIPT.json'] + dump(out / 'RECEIPT.json', {'status': result['status'], + 'artifacts': {p.relative_to(out).as_posix(): sha(p) for p in artifacts}}) + print(result['status'], json.dumps(result['profile']), flush=True) + + +if __name__ == '__main__': + main() diff --git a/research/complex-rank7-end-pair/data/END-RECIPE.json b/research/complex-rank7-end-pair/data/END-RECIPE.json new file mode 100644 index 0000000000..c301e8d383 --- /dev/null +++ b/research/complex-rank7-end-pair/data/END-RECIPE.json @@ -0,0 +1,28 @@ +{ + "schema": 1, + "input_sha256": "090fd574747fbf9628e70cb9beb792732914cfa52e1df147579fe4dbccc68c5a", + "ends": [ + [588, [256, 128, 96, 16, 8, 4, 2, 1], [256, 128, 96, 17, 9, 5, 3]], + [589, [256, 128, 96, 16, 8, 4, 2, 1], [256, 128, 96, 17, 9, 5, 3]] + ], + "B_frames": [ + [1018, [260, 130, 96, 20, 10, 1], [261, 131, 96, 20, 10]], + [1019, [258, 132, 96, 20, 10, 1], [259, 133, 96, 20, 10]], + [1050, [264, 137, 96, 24, 4, 3], [268, 137, 96, 24, 3]], + [1051, [265, 136, 96, 24, 4, 3], [265, 140, 96, 24, 3]], + [1130, [260, 130, 96, 20, 10, 1], [261, 131, 96, 20, 10]], + [1131, [258, 132, 96, 20, 10, 1], [259, 133, 96, 20, 10]], + [1162, [264, 137, 96, 24, 4, 3], [268, 137, 96, 24, 3]], + [1163, [265, 136, 96, 24, 4, 3], [265, 140, 96, 24, 3]] + ], + "early_cleanup": [ + [148, 148, [256, 128, 96, 16, 8, 4, 2, 1], [256, 128, 96, 17, 9, 5, 3]], + [149, 149, [256, 128, 96, 16, 8, 4, 2, 1], [256, 128, 96, 17, 9, 5, 3]] + ], + "expected": { + "output_sha256": "096ed4abee4bb02c0d6f03bb8191510e57f6b1fcd34e8644f4fd336254e46411", + "local_profile": {"1": 2542, "2": 352, "3": 438, "4": 162, "5": 72, "8": 8}, + "local_rank_mass": 5632, + "width_inventory": {"4": 80, "5": 32, "7": 56, "8": 120, "9": 216} + } +} diff --git a/research/complex-rank7-end-pair/data/EXPECTED.json b/research/complex-rank7-end-pair/data/EXPECTED.json new file mode 100644 index 0000000000..083c78b951 --- /dev/null +++ b/research/complex-rank7-end-pair/data/EXPECTED.json @@ -0,0 +1,43 @@ +{ + "kappa": "166202617947539/100000000000000000", + "kappa_decimal": "0.00166202617947539", + "baseline_kappa": "1661960547713947/1000000000000000000", + "gain": "65631761443/1000000000000000000", + "complex_lower": "6659172436825204659/4000000000000000000000", + "bit_lower": "1733427039589844234749/1000000000000000000000000", + "global_profile": { + "1": 183024, + "2": 25344, + "3": 31536, + "4": 11664, + "5": 5184, + "8": 576, + "48": 2400 + }, + "baseline_word_sha256": "1e1d99ba7aab877d6b573c41f7030ee73de9a52fee51d0a822ddaae71251999f", + "word_sha256": "096ed4abee4bb02c0d6f03bb8191510e57f6b1fcd34e8644f4fd336254e46411", + "owners_sha256": "02bde66a90b8ae9b0c4d8d563ca2218ad1de851fa6dec3d1abb5d3a30ca3dd2a", + "native_pins": { + "canonical_sha256": "9c071e0491181435c2fcb6dc84343a6a878e38be5ece1545199af06b62310d3f", + "program_sha256": { + "forward": "1acecc4bc9daaad1f68bfd65d87b6c14a12319695d23d2faf4e13c9f5efc9b37", + "backward": "53adb146aafd679ac18a3b7498b134c4e99f15a9b89a86f6a4df829a9e6a8168" + }, + "clifford_recipes_sha256": "dc9323bd5bde9d238688685fb15b25f15e8badeea328eddba7419fc2d5404bf7" + }, + "local_profile": { + "1": 2542, + "2": 352, + "3": 438, + "4": 162, + "5": 72, + "8": 8 + }, + "phase_edges": 2950, + "native_controls": 25, + "inherited_cold_controls": 89, + "strict_admission_controls": 69, + "bit_controls": 12, + "independent_owner_controls": 7, + "u8_word_sha256": "090fd574747fbf9628e70cb9beb792732914cfa52e1df147579fe4dbccc68c5a" +} diff --git a/research/complex-rank7-end-pair/data/OWNER-FLOW.json.gz b/research/complex-rank7-end-pair/data/OWNER-FLOW.json.gz new file mode 100644 index 0000000000..59cf38b88e Binary files /dev/null and b/research/complex-rank7-end-pair/data/OWNER-FLOW.json.gz differ diff --git a/research/complex-rank7-end-pair/data/SELECTED942.json.gz b/research/complex-rank7-end-pair/data/SELECTED942.json.gz new file mode 100644 index 0000000000..547cae7e92 Binary files /dev/null and b/research/complex-rank7-end-pair/data/SELECTED942.json.gz differ diff --git a/research/complex-rank7-end-pair/end_controls.py b/research/complex-rank7-end-pair/end_controls.py new file mode 100644 index 0000000000..9dacd8b285 --- /dev/null +++ b/research/complex-rank7-end-pair/end_controls.py @@ -0,0 +1,150 @@ +"""Reason-specific raw-auditor controls for both new dimension-seven ends. + +Prepared with OpenAI Codex assistance. Apache-2.0. +""" +import argparse +import copy +import gzip +import hashlib +import json +from pathlib import Path +import subprocess +import sys +import tempfile + + +WORD_SHA = '096ed4abee4bb02c0d6f03bb8191510e57f6b1fcd34e8644f4fd336254e46411' +RECIPE = Path(__file__).resolve().parent / 'data/END-RECIPE.json' + + +def require(condition, message): + if not condition: + raise RuntimeError(message) + + +def cases(word, recipe): + result = [] + endpoint = dict(native='arbitrary-dirt endpoint', independent='all-column endpoint failure') + nonmonotone = dict(native="'nonmonotone'", independent='frame not monotone') + profile = dict(native='paid local profile differs from the reported local_profile', + independent='paid profile mismatch') + + def add(name, mutation, reasons): + mutant = copy.deepcopy(word) + mutation(mutant) + result.append((name, mutant, reasons)) + + def drop_gate(mutant, index): + mutant['cleanup'].pop(index) + mutant['early_cleanup'] = [[j - (j > index), frame] + for j, frame in mutant['early_cleanup'] if j != index] + + tails = {word['cleanup'][cleanup_index][1]: (index, cleanup_index, old, new) + for index, cleanup_index, old, new in recipe['early_cleanup']} + require(set(tails) == {588, 589}, 'changed tail helpers') + for q, old, new in recipe['ends']: + index, cleanup_index, old_tail, new_tail = tails[q] + require(word['helper_final'][str(q)] == new and word['early_cleanup'][index] == + [cleanup_index, new_tail], 'changed end and tail') + add(f'end-{q}-other-dim7', lambda w: w['helper_final'].__setitem__(str(q), new[:-1] + [1]), + dict(native='hpe cone pre-check (row / column)', independent='frame not monotone')) + add(f'end-{q}-drop-cleanup-gate', lambda w: drop_gate(w, cleanup_index), endpoint) + add(f'end-{q}-wrong-cleanup-coefficient', + lambda w: w['cleanup'][cleanup_index].__setitem__(3, 2 * w['cleanup'][cleanup_index][3]), endpoint) + add(f'end-{q}-cleanup-at-full', lambda w: w['early_cleanup'].pop(index), + dict(native='cleanup on non-FULL end', independent='full cleanup reads a low end')) + add(f'end-{q}-revert-tail-frame', lambda w: w['early_cleanup'][index].__setitem__(1, old_tail), nonmonotone) + add(f'end-{q}-removed-end', lambda w: w['helper_final'].pop(str(q)), profile) + for index, old, new in recipe['B_frames']: + require(word['B'][index][-1] == new, 'changed B frame') + reasons = nonmonotone if index in (1018, 1019, 1050, 1051) else profile + add(f'B-{index}-omitted-lowering', lambda w: w['B'][index].__setitem__(-1, old), reasons) + + def false_profile(mutant): + mutant['report']['local_profile']['1'] += 2 + mutant['report']['local_profile']['2'] -= 1 + + def false_widths(mutant): + mutant['report']['width_inventory']['7'] -= 1 + mutant['report']['width_inventory']['8'] += 1 + + add('false-profile-same-rank-mass', false_profile, profile) + # The independent raw auditor checks the reported width inventory. + add('false-width-inventory', false_widths, dict(independent='width inventory mismatch')) + require(len(result) == 22, 'targeted control count') + return result + + +def run_check(args, name, checker, word, expected=None): + payload = json.dumps(word, sort_keys=True, separators=(',', ':')).encode() + with tempfile.TemporaryDirectory(prefix=f'{name}-{checker}-', dir=args.output) as temporary: + root = Path(temporary) + path = root / 'WORD.json.gz' + path.write_bytes(gzip.compress(payload, compresslevel=9, mtime=0)) + command = [sys.executable, '-I', '-B'] + if checker == 'native': + command += [str(args.portable_runner), str(args.native_auditor), '--raw', str(path), + '--routing', str(args.routing), '--out', str(root / 'native')] + receipt = root / 'native/SCALAR-FRAME-AUDIT.json' + error_type = 'AssertionError:' + hash_key = 'input_content_sha256' + else: + receipt = root / 'independent.json' + command += [str(args.independent_auditor), '--word', str(path), '--output', str(receipt)] + if args.main is not None: + command += ['--main', str(args.main)] + error_type = 'RuntimeError:' + hash_key = 'content_sha256' + completed = subprocess.run(command, capture_output=True, text=True, timeout=120) + log = completed.stdout + completed.stderr + (args.output / f'{name}.{checker}.log').write_text(log, encoding='utf-8') + if expected is None: + require(completed.returncode == 0, f'{checker} rejected the positive control: {log[-1000:]}') + result = json.loads(receipt.read_text(encoding='utf-8')) + require(result[hash_key] == WORD_SHA, 'positive control content pin') + return dict(checker=checker, accepted=True, status=result['status']) + last_line = log.strip().splitlines()[-1] if log.strip() else '' + require(completed.returncode == 1 and last_line.startswith(error_type) and expected in last_line, + f'{name}: {checker} expected {expected!r}, got exit {completed.returncode}: {last_line}') + return dict(checker=checker, rejected=last_line) + + +def main(): + parser = argparse.ArgumentParser(description=__doc__) + parser.add_argument('--word', type=Path, required=True) + parser.add_argument('--recipe', type=Path, default=RECIPE) + parser.add_argument('--independent-auditor', type=Path, required=True) + parser.add_argument('--native-auditor', type=Path, required=True) + parser.add_argument('--portable-runner', type=Path, required=True) + parser.add_argument('--routing', type=Path, required=True) + parser.add_argument('--main', type=Path) + parser.add_argument('--output', type=Path, required=True) + args = parser.parse_args() + raw = gzip.decompress(args.word.read_bytes()) + require(hashlib.sha256(raw).hexdigest() == WORD_SHA, 'candidate content pin') + recipe = json.loads(args.recipe.read_text(encoding='utf-8')) + require(recipe['expected']['output_sha256'] == WORD_SHA, 'recipe output pin') + word = json.loads(raw) + args.output.mkdir(parents=True, exist_ok=True) + positive = [run_check(args, 'positive', checker, word) for checker in ('native', 'independent')] + rejected = [] + for name, mutant, reasons in cases(word, recipe): + checks = [run_check(args, name, checker, mutant, expected) for checker, expected in reasons.items()] + rejected.append(dict(name=name, checks=checks)) + print('rejected', name, flush=True) + report = dict(status='PASS_DIMENSION_SEVEN_END_PAIR_RAW_CONTROLS', content_sha256=WORD_SHA, + checked_helpers=[588, 589], positive_controls=positive, + mutations=22, raw_rejections=sum(len(case['checks']) for case in rejected), + rejected_controls=rejected, + checker_sha256={name: hashlib.sha256(path.read_bytes()).hexdigest() + for name, path in (('independent', args.independent_auditor), + ('native', args.native_auditor), + ('portable_runner', args.portable_runner))}, + scope='Supplemental controls for both dimension-seven ends, both tail gates and frames, all eight B ' + 'lowerings, paid profiles, and reported widths. Inherited controls are counted separately.') + (args.output / 'END-CONTROLS.json').write_text(json.dumps(report, indent=2) + '\n', encoding='utf-8') + print(report['status'], '22 mutants; 43 raw rejections', flush=True) + + +if __name__ == '__main__': + main() diff --git a/research/complex-rank7-end-pair/evidence/ASSEMBLY.json.gz b/research/complex-rank7-end-pair/evidence/ASSEMBLY.json.gz new file mode 100644 index 0000000000..9ec207f2db Binary files /dev/null and b/research/complex-rank7-end-pair/evidence/ASSEMBLY.json.gz differ diff --git a/research/complex-rank7-end-pair/evidence/END-CONTROLS.json b/research/complex-rank7-end-pair/evidence/END-CONTROLS.json new file mode 100644 index 0000000000..598f62e660 --- /dev/null +++ b/research/complex-rank7-end-pair/evidence/END-CONTROLS.json @@ -0,0 +1,312 @@ +{ + "status": "PASS_DIMENSION_SEVEN_END_PAIR_RAW_CONTROLS", + "content_sha256": "096ed4abee4bb02c0d6f03bb8191510e57f6b1fcd34e8644f4fd336254e46411", + "checked_helpers": [ + 588, + 589 + ], + "positive_controls": [ + { + "checker": "native", + "accepted": true, + "status": "PASS_CHANGED_WORD_ALL_COLUMNS_AND_NATIVE_FRAMES" + }, + { + "checker": "independent", + "accepted": true, + "status": "PASS_INDEPENDENT_EXACT_FORWARD_AND_DUAL_ENDPOINT_FRAMES_ENTRANCES_CONES" + } + ], + "mutations": 22, + "raw_rejections": 43, + "rejected_controls": [ + { + "name": "end-588-other-dim7", + "checks": [ + { + "checker": "native", + "rejected": "AssertionError: ('hpe cone pre-check (row / column)', [('row', 588, 412), ('row', 588, 588)])" + }, + { + "checker": "independent", + "rejected": "RuntimeError: frame not monotone (588, (256, 128, 96, 17, 9, 5, 3), (256, 128, 96, 16, 8, 4, 1))" + } + ] + }, + { + "name": "end-588-drop-cleanup-gate", + "checks": [ + { + "checker": "native", + "rejected": "AssertionError: ('arbitrary-dirt endpoint', False)" + }, + { + "checker": "independent", + "rejected": "RuntimeError: all-column endpoint failure (False, [588])" + } + ] + }, + { + "name": "end-588-wrong-cleanup-coefficient", + "checks": [ + { + "checker": "native", + "rejected": "AssertionError: ('arbitrary-dirt endpoint', False)" + }, + { + "checker": "independent", + "rejected": "RuntimeError: all-column endpoint failure (False, [588])" + } + ] + }, + { + "name": "end-588-cleanup-at-full", + "checks": [ + { + "checker": "native", + "rejected": "AssertionError: ('cleanup on non-FULL end', ['add', 588, 412, -1, 1, 1001])" + }, + { + "checker": "independent", + "rejected": "RuntimeError: full cleanup reads a low end" + } + ] + }, + { + "name": "end-588-revert-tail-frame", + "checks": [ + { + "checker": "native", + "rejected": "AssertionError: ('nonmonotone', 588, 1001, 1005)" + }, + { + "checker": "independent", + "rejected": "RuntimeError: frame not monotone (588, (256, 128, 96, 16, 8, 4, 2, 1), (256, 128, 96, 17, 9, 5, 3))" + } + ] + }, + { + "name": "end-588-removed-end", + "checks": [ + { + "checker": "native", + "rejected": "AssertionError: paid local profile differs from the reported local_profile" + }, + { + "checker": "independent", + "rejected": "RuntimeError: paid profile mismatch" + } + ] + }, + { + "name": "end-589-other-dim7", + "checks": [ + { + "checker": "native", + "rejected": "AssertionError: ('hpe cone pre-check (row / column)', [('row', 589, 414), ('row', 589, 589)])" + }, + { + "checker": "independent", + "rejected": "RuntimeError: frame not monotone (589, (256, 128, 96, 17, 9, 5, 3), (256, 128, 96, 16, 8, 4, 1))" + } + ] + }, + { + "name": "end-589-drop-cleanup-gate", + "checks": [ + { + "checker": "native", + "rejected": "AssertionError: ('arbitrary-dirt endpoint', False)" + }, + { + "checker": "independent", + "rejected": "RuntimeError: all-column endpoint failure (False, [589])" + } + ] + }, + { + "name": "end-589-wrong-cleanup-coefficient", + "checks": [ + { + "checker": "native", + "rejected": "AssertionError: ('arbitrary-dirt endpoint', False)" + }, + { + "checker": "independent", + "rejected": "RuntimeError: all-column endpoint failure (False, [589])" + } + ] + }, + { + "name": "end-589-cleanup-at-full", + "checks": [ + { + "checker": "native", + "rejected": "AssertionError: ('cleanup on non-FULL end', ['add', 589, 414, -1, 1, 1001])" + }, + { + "checker": "independent", + "rejected": "RuntimeError: full cleanup reads a low end" + } + ] + }, + { + "name": "end-589-revert-tail-frame", + "checks": [ + { + "checker": "native", + "rejected": "AssertionError: ('nonmonotone', 589, 1001, 1005)" + }, + { + "checker": "independent", + "rejected": "RuntimeError: frame not monotone (589, (256, 128, 96, 16, 8, 4, 2, 1), (256, 128, 96, 17, 9, 5, 3))" + } + ] + }, + { + "name": "end-589-removed-end", + "checks": [ + { + "checker": "native", + "rejected": "AssertionError: paid local profile differs from the reported local_profile" + }, + { + "checker": "independent", + "rejected": "RuntimeError: paid profile mismatch" + } + ] + }, + { + "name": "B-1018-omitted-lowering", + "checks": [ + { + "checker": "native", + "rejected": "AssertionError: ('nonmonotone', 392, 658, 718)" + }, + { + "checker": "independent", + "rejected": "RuntimeError: frame not monotone (392, (260, 130, 96, 20, 10, 1), (261, 131, 96, 20, 10))" + } + ] + }, + { + "name": "B-1019-omitted-lowering", + "checks": [ + { + "checker": "native", + "rejected": "AssertionError: ('nonmonotone', 97, 659, 718)" + }, + { + "checker": "independent", + "rejected": "RuntimeError: frame not monotone (102, (258, 132, 96, 20, 10, 1), (259, 133, 96, 20, 10))" + } + ] + }, + { + "name": "B-1050-omitted-lowering", + "checks": [ + { + "checker": "native", + "rejected": "AssertionError: ('nonmonotone', 328, 690, 718)" + }, + { + "checker": "independent", + "rejected": "RuntimeError: frame not monotone (328, (264, 137, 96, 24, 4, 3), (268, 137, 96, 24, 3))" + } + ] + }, + { + "name": "B-1051-omitted-lowering", + "checks": [ + { + "checker": "native", + "rejected": "AssertionError: ('nonmonotone', 109, 691, 718)" + }, + { + "checker": "independent", + "rejected": "RuntimeError: frame not monotone (110, (265, 136, 96, 24, 4, 3), (265, 140, 96, 24, 3))" + } + ] + }, + { + "name": "B-1130-omitted-lowering", + "checks": [ + { + "checker": "native", + "rejected": "AssertionError: paid local profile differs from the reported local_profile" + }, + { + "checker": "independent", + "rejected": "RuntimeError: paid profile mismatch" + } + ] + }, + { + "name": "B-1131-omitted-lowering", + "checks": [ + { + "checker": "native", + "rejected": "AssertionError: paid local profile differs from the reported local_profile" + }, + { + "checker": "independent", + "rejected": "RuntimeError: paid profile mismatch" + } + ] + }, + { + "name": "B-1162-omitted-lowering", + "checks": [ + { + "checker": "native", + "rejected": "AssertionError: paid local profile differs from the reported local_profile" + }, + { + "checker": "independent", + "rejected": "RuntimeError: paid profile mismatch" + } + ] + }, + { + "name": "B-1163-omitted-lowering", + "checks": [ + { + "checker": "native", + "rejected": "AssertionError: paid local profile differs from the reported local_profile" + }, + { + "checker": "independent", + "rejected": "RuntimeError: paid profile mismatch" + } + ] + }, + { + "name": "false-profile-same-rank-mass", + "checks": [ + { + "checker": "native", + "rejected": "AssertionError: paid local profile differs from the reported local_profile" + }, + { + "checker": "independent", + "rejected": "RuntimeError: paid profile mismatch" + } + ] + }, + { + "name": "false-width-inventory", + "checks": [ + { + "checker": "independent", + "rejected": "RuntimeError: width inventory mismatch" + } + ] + } + ], + "checker_sha256": { + "independent": "03afcfc9da428b3f6edac81c1fee7f277ab81b64998efae78380b63498dec6cd", + "native": "c9a05ff291016b870b4fc925b317abd9c3a604c73ffc8920dfdbbeb475e20aa3", + "portable_runner": "322d29d43dbee8130cde15d4bf05396698b5e3dfe3735813899db91cc0ba7955" + }, + "scope": "Supplemental controls for both dimension-seven ends, both tail gates and frames, all eight B lowerings, paid profiles, and reported widths. Inherited controls are counted separately." +} diff --git a/research/complex-rank7-end-pair/evidence/POINT-AUDIT.json b/research/complex-rank7-end-pair/evidence/POINT-AUDIT.json new file mode 100644 index 0000000000..3fc44dda87 --- /dev/null +++ b/research/complex-rank7-end-pair/evidence/POINT-AUDIT.json @@ -0,0 +1,69 @@ +{ + "status": "PASS_EXPLICIT_512_POINT_END_PAIR_ACCOUNTING", + "input_sha256": "090fd574747fbf9628e70cb9beb792732914cfa52e1df147579fe4dbccc68c5a", + "output_sha256": "096ed4abee4bb02c0d6f03bb8191510e57f6b1fcd34e8644f4fd336254e46411", + "changed_annotations": { + "helper_final": 2, + "B": 8, + "early_cleanup": 2 + }, + "local_profile": { + "1": 2542, + "2": 352, + "3": 438, + "4": 162, + "5": 72, + "8": 8 + }, + "delta": { + "1": -6, + "2": 14, + "3": -8 + }, + "rank_mass_before": 5634, + "rank_mass_after": 5632, + "width_inventory": { + "4": 80, + "5": 32, + "7": 56, + "8": 120, + "9": 216 + }, + "width_sum_before": 3778, + "width_sum_after": 3776, + "changed_helper_widths": [ + { + "helper": 588, + "before": 8, + "after": 7 + }, + { + "helper": 589, + "before": 8, + "after": 7 + } + ], + "rejected_controls": [ + { + "name": "mass-preserving false profile", + "rejected": "paid profile" + }, + { + "name": "floating profile count", + "rejected": "paid profile" + }, + { + "name": "false helper widths", + "rejected": "width inventory" + }, + { + "name": "false rank mass", + "rejected": "rank mass" + }, + { + "name": "omitted required B lowering", + "rejected": "nonmonotone frame" + } + ], + "scope": "Literal source, target, center, gate, tail, and end frames; paid profile; individual helper widths; exact twelve-annotation recipe. Scalar identities, cones, phases, owners, and kappa are separate." +} diff --git a/research/complex-rank7-end-pair/evidence/VALIDATION.json b/research/complex-rank7-end-pair/evidence/VALIDATION.json new file mode 100644 index 0000000000..79db7d9074 --- /dev/null +++ b/research/complex-rank7-end-pair/evidence/VALIDATION.json @@ -0,0 +1,140 @@ +{ + "schema": "complex-rank7-end-pair-validation/1", + "status": "PASS_COLD_RANK7_END_PAIR_COMPLETE_ADMISSION", + "completed_at_utc": "2026-10-13T16:28:25.775821+00:00", + "reproduce": "python3 -B research/complex-rank7-end-pair/reproduce.py --work /tmp/rank7-end-pair", + "source_mode": "Default network fetch of four pinned commits into a new bare repository; no preexisting source objects or ancestor outputs reused.", + "environment": { + "python": "3.14.8", + "implementation": "CPython", + "system": "Darwin", + "macos": "15.7.3", + "machine": "arm64", + "git": "git version 2.39.5 (Apple Git-154)", + "zlib_runtime": "1.2.12" + }, + "seconds": 914.6039056249938, + "kappa": "166202617947539/100000000000000000", + "kappa_decimal": "0.00166202617947539", + "baseline_kappa": "1661960547713947/1000000000000000000", + "gain": "65631761443/1000000000000000000", + "complex_lower": "6659172436825204659/4000000000000000000000", + "bit_lower": "1733427039589844234749/1000000000000000000000000", + "conservative_kappa": "1662026176592891/1000000000000000000", + "global_profile": { + "1": 183024, + "2": 25344, + "3": 31536, + "4": 11664, + "5": 5184, + "8": 576, + "48": 2400 + }, + "rank_mass": 520704, + "deficit": 2560, + "certified_global_prefix_bits": 257, + "charged_prefix_bits": 512, + "physical_row_coefficient": 16467, + "coverage": { + "top_level_stages": 30, + "fresh_ancestor_stages": 22, + "x4_recipe_controls": 7, + "u8_construction_controls": 38, + "input_columns_each_orientation": 728, + "phase_edges": 2950, + "native_controls_rejected": 25, + "inherited_cold_controls_rejected": 89, + "targeted_end_mutations": 22, + "targeted_raw_rejections": 43, + "independent_point_controls_rejected": 5, + "owner_construction_controls_rejected": 8, + "literal_owner_roles": 36288, + "physical_windows": 144, + "strict_admission_controls_rejected": 69, + "bit_controls_rejected": 12, + "independent_owner_controls_rejected": 7, + "exact_moment_engines": 2, + "outer_inequalities_each_mode": { + "native": 47, + "conservative": 47 + } + }, + "stage_seconds": { + "initialize-source-cache": 0.06248908300040057, + "fetch-741": 71.473891792004, + "check-commit-741": 0.055783916999644134, + "archive-741": 0.09857637499953853, + "fetch-799": 32.114858417000505, + "check-commit-799": 0.05836458299745573, + "archive-799": 0.08208270899922354, + "fetch-777": 93.80813454200688, + "check-commit-777": 0.055930457994691096, + "archive-777": 0.06415995799761731, + "fetch-869": 47.21029845799785, + "check-commit-869": 0.05565891700098291, + "archive-869": 0.07226466700376477, + "complete-ancestor-replay": 429.68171058300504, + "rebuild-x4-word": 0.16151545799948508, + "x4-control-C0_identity": 0.14025937500264263, + "x4-control-C1_741_only": 0.14916787499532802, + "x4-control-M1_one_pair_448": 0.13701779199618613, + "x4-control-N1_drop_preread": 0.13800841699412558, + "x4-control-N2_wrong_sign_preread": 0.13180070799717214, + "x4-control-N3_plain_join_moved_reads": 0.13286591700307326, + "x4-control-N4_raise_dim7_at_855": 0.11271662500075763, + "build-u8-word": 0.10168629200052237, + "u8-construction-controls": 12.302578250004444, + "build-end-pair": 0.09477925000101095, + "independent-point-audit": 0.3898296670013224, + "end-pair-controls": 28.037287084000127, + "rebuild-owners": 1.1194444579959963, + "complete-native-replay": 166.08186187499814, + "complete-global-admission": 30.236285625003802 + }, + "checked_code_and_data_sha256": { + "SOURCES.json": "e39ba83324367939dde94805a6e0795024acab061f904d8b6788f142059984fa", + "audit_points.py": "970a3f48400f3eb98d6b611abec0f3d6e4c04f0041d591ec23daf36a83af2b54", + "build_owners.py": "af11428407c492fc9f67cbf9af76ff1f51377fb9536233c43c52519dbf903662", + "build_word.py": "01b9f4a0f2b9c841f9d24feec7777687b1c8b6ec3ae821351789a467ce60f17b", + "code/admit_candidate.py": "c87412c62a9db1c65f0f6cfeeeb41ae1e26e3cedad7ecdda3cbba3bbefbe0e2f", + "code/prepare_candidate.py": "a481fc323cccc0b9df5a5560f551669cbdefb1c4f0fa108cf51dd2305e10de64", + "code/replay_native.py": "73ce7d788ef2ace19193c86e9039a2d2f2b8f24d46dc0bc6b2dd2b6aebc0b2f9", + "data/END-RECIPE.json": "a50398b4543c71129be36fdbfee607b1423373d9b13cb5948b6a9395597b7dc2", + "data/EXPECTED.json": "a17d142483622b27402ec899859477b2d54d82fec2fac032416622f53339922c", + "data/OWNER-FLOW.json.gz": "71ad7c00394c8a4947fa5172cbe9a07ae819113e311224e23c4ee6b4220e7d65", + "data/SELECTED942.json.gz": "3979034247f26d31dc1397313750e99130709eae990dfa75c7d9d982034bd819", + "end_controls.py": "1b1ec3e93d626ab90039eeda8c484236e4025d3c98fdd8f5a2af7f8f00424e0b", + "reproduce.py": "bc364a91c488e50eb4f0c06b9e88ae586fb77a54bebdc421aa696c7b50b38524" + }, + "checked_code_and_data_unchanged_during_run": true, + "raw_word_content_sha256": "096ed4abee4bb02c0d6f03bb8191510e57f6b1fcd34e8644f4fd336254e46411", + "owner_content_sha256": "02bde66a90b8ae9b0c4d8d563ca2218ad1de851fa6dec3d1abb5d3a30ca3dd2a", + "native_pins": { + "canonical_sha256": "9c071e0491181435c2fcb6dc84343a6a878e38be5ece1545199af06b62310d3f", + "program_sha256": { + "forward": "1acecc4bc9daaad1f68bfd65d87b6c14a12319695d23d2faf4e13c9f5efc9b37", + "backward": "53adb146aafd679ac18a3b7498b134c4e99f15a9b89a86f6a4df829a9e6a8168" + }, + "clifford_recipes_sha256": "dc9323bd5bde9d238688685fb15b25f15e8badeea328eddba7419fc2d5404bf7" + }, + "archived_assembly_content_sha256": "838028f31857720706631bdeeaa5b36c466332252d6ff15b6e73323151a2e545", + "run_artifact_sha256": { + "RESULT.json": "cc2033bcc503f4dd04dbe9f6281be55d516f560c707f900d162c2b6fa6e2ebf6", + "RECEIPT.json": "68ffa599da52a573cfdb908348538f99c1a78ead5b1c7d948c38b4f656dea95e", + "STAGES.json": "842c90204d272e21d5a82bfa5e7d706b877af04e14f010b446fc3593e107d427", + "baseline741/RESULT.json": "2fab11837adfe77377aa3f31c349cf6ba655076168633c63e458c6bb54395c38", + "baseline741/RECEIPT.json": "c5576a8a2e2d16e3fe8c2a12335d8f944026c7456d32703ca22becbe0524fd75", + "native/RESULT.json": "3d92537fab73a6bd815a8e5bbfb0c3e018f5f1723ea26c9a72f2cf5fea173637", + "native/RECEIPT.json": "3572bdfd52040e9b46c010523259a13af19bb921285d2d43016b097768885b5a", + "admission/RESULT.json": "dab10645d76e4ce1987d0d752fb86053204e4703c2a18592c97cfb00eebe4e02", + "admission/RECEIPT.json": "8f66e22ae23a7b00bbc09513ef79e0514f350dd6fcbdf96cbb7a07d5b2d36b0d" + }, + "hash_scope": "Input code and data hashes are immutable pins. Run receipt and assembly provenance hashes identify this execution and can change with output paths or timing. Compare regenerated gzip payloads and mathematical fields, not gzip containers.", + "development_and_comparison_base_commit": "407d52ff15b2217b6e22bce44728b4c3907b6515", + "submission_branch_base_commit": "66849e1475c930fd7732b1a0067ffee7e1e95f60", + "scope": "Fresh pinned ancestor, X4 and U8 constructions, minimal end recipe, independent point audit, targeted controls, exact owner reconstruction, complete native and strict global admission. The cited PR589 bit circuit is repriced. Inherited uniform and analytic assumptions remain.", + "checks_not_run": [ + "Repository-wide make verify", + "Linux CI" + ] +} diff --git a/research/complex-rank7-end-pair/reproduce.py b/research/complex-rank7-end-pair/reproduce.py new file mode 100644 index 0000000000..bf0d70ff3c --- /dev/null +++ b/research/complex-rank7-end-pair/reproduce.py @@ -0,0 +1,224 @@ +"""Rebuild and admit the rank-seven end pair using pinned predecessor checkers.""" +import argparse +import gzip +import hashlib +import json +import os +from pathlib import Path, PurePosixPath +import shutil +import subprocess +import sys +import tarfile +import time + + +PACKAGE = Path(__file__).resolve().parent + + +def read(path): + data = path.read_bytes() + return json.loads(gzip.decompress(data) if path.suffix == '.gz' else data) + + +def sha(path): + return hashlib.sha256(path.read_bytes()).hexdigest() + + +def payload_sha(path): + return hashlib.sha256(gzip.decompress(path.read_bytes())).hexdigest() + + +def need(condition, message): + if not condition: + raise ValueError(message) + + +def dump(path, value): + path.write_text(json.dumps(value, indent=2) + '\n') + + +def extract_package(archive, destination, prefix): + with tarfile.open(archive) as source: + for member in source.getmembers(): + relative = PurePosixPath(member.name) + need(not relative.is_absolute() and '..' not in relative.parts, 'Unsafe archive member') + if relative == PurePosixPath(prefix) or PurePosixPath(prefix) in relative.parents: + target = destination.joinpath(*relative.parts) + if member.isdir(): + target.mkdir(parents=True, exist_ok=True) + else: + need(member.isfile(), 'Dependency archive must contain regular files') + target.parent.mkdir(parents=True, exist_ok=True) + with source.extractfile(member) as incoming, target.open('wb') as outgoing: + shutil.copyfileobj(incoming, outgoing) + else: + need(member.isdir() and relative in PurePosixPath(prefix).parents, + 'Unexpected file outside the pinned package') + + +def main(): + parser = argparse.ArgumentParser(description=__doc__) + parser.add_argument('--work', type=Path, required=True, help='New output directory outside this package') + parser.add_argument('--git-repository', type=Path, + help='Read pinned commits from an existing local repository instead of fetching') + args = parser.parse_args() + need(__debug__ and sys.version_info >= (3, 11), 'Python 3.11+ with assertions enabled is required') + work = args.work.resolve() + need(not work.exists() and not work.is_relative_to(PACKAGE), 'Use a new external output directory') + work.mkdir(parents=True) + pins, expected = read(PACKAGE / 'SOURCES.json'), read(PACKAGE / 'data/EXPECTED.json') + comparison = PACKAGE / pins['comparison_snapshot']['artifact'] + need(payload_sha(comparison) == pins['comparison_snapshot']['source_content_sha256'], + 'Pinned selected-result comparison snapshot') + need(read(comparison)['kappa'] == expected['baseline_kappa'], 'Selected comparison value') + environment = dict(os.environ, PYTHONDONTWRITEBYTECODE='1', PYTHONUTF8='1', PYTHONINTMAXSTRDIGITS='0') + for name in ('PYTHONOPTIMIZE', 'WITNESS_SPEC', 'WITNESS_DISCOVER', 'WITNESS_RECORD', + 'MULTICOLD_SOURCE', 'GAIN_PR564_SOURCE', 'SW4_PIN', 'PR635_CODE', + 'XPARAM', 'EXTRAH', 'PAIRING_RANKS', 'DIAG', 'HYBBUDGET'): + environment.pop(name, None) + stages = [] + started = time.monotonic() + + def run(name, command, binary_output=None): + print('RUN', name, flush=True) + begin = time.monotonic() + log = work / (name + '.log') + with log.open('wb') as stream: + if binary_output is None: + result = subprocess.run(list(map(str, command)), stdout=stream, stderr=subprocess.STDOUT, + env=environment) + else: + with binary_output.open('wb') as output: + result = subprocess.run(list(map(str, command)), stdout=output, stderr=stream, + env=environment) + stages.append(dict(name=name, exit_code=result.returncode, seconds=time.monotonic()-begin, + log_sha256=sha(log))) + dump(work / 'STAGES.json', stages) + if result.returncode: + raise RuntimeError(name + ' failed:\n' + log.read_text(errors='replace')[-6000:]) + print('PASS', name, flush=True) + return log.read_text(errors='replace') + + def python(name, script, *arguments): + return run(name, [sys.executable, '-B', script, *arguments]) + + repository = args.git_repository.resolve() if args.git_repository else work / 'source.git' + if args.git_repository is None: + run('initialize-source-cache', ['git', 'init', '--bare', repository]) + sources = {} + for row in pins['sources']: + number, commit, relative = row['pr'], row['commit'], row['path'] + if args.git_repository is None: + run(f'fetch-{number}', ['git', '-C', repository, 'fetch', '--no-tags', '--depth=1', + row['repository'], commit]) + run(f'check-commit-{number}', ['git', '-C', repository, 'cat-file', '-e', commit + '^{commit}']) + archive = work / f'source-{number}.tar' + run(f'archive-{number}', ['git', '-C', repository, 'archive', '--format=tar', commit, relative], + binary_output=archive) + destination = work / f'source-{number}' + extract_package(archive, destination, relative) + package = destination / relative + need(sha(package / 'MANIFEST.json') == row['manifest_sha256'], 'Pinned source manifest') + actual = {path.relative_to(package).as_posix(): sha(path) for path in package.rglob('*') + if path.is_file() and path != package / 'MANIFEST.json' and '__pycache__' not in path.parts} + manifest = read(package / 'MANIFEST.json') + need(actual == manifest, f'All source files match PR{number}') + sources[number] = package + ancestor, dependency, checker = [sources[number] for number in (741, 799, 777)] + root = ancestor / 'vendor/main' + native_source = root / 'research/complex-k492-owner-repair/vendor/multicold' + routing = native_source / 'vendor/pr558/vendor/pr552/complex-local/inputs/phase-baseline/CANONICAL-WORD.json.gz' + pr635 = ancestor / 'vendor/pr635/research/complex-571-cold14-st2' + baseline = work / 'baseline741' + python('complete-ancestor-replay', ancestor / 'verify.py', '--work', baseline) + + construction = work / 'baseline799-construction' + construction.mkdir() + base_word = construction / 'FULL-WORD.json.gz' + recipe = dependency / 'construction/jx/RECIPE-X4.json' + builder = dependency / 'construction/jx/build_jx.py' + python('rebuild-x4-word', builder, '--input', baseline / 'sw4/FULL-WORD.json.gz', + '--recipe', recipe, '--out', base_word) + x4 = read(recipe) + need(payload_sha(base_word) == x4['expected']['output_content_sha256'] == expected['baseline_word_sha256'], + 'Rebuilt X4 source word') + control_results = {} + for name, control in sorted(x4['controls'].items()): + log = python('x4-control-' + name, builder, '--input', baseline / 'sw4/FULL-WORD.json.gz', + '--recipe', recipe, '--control', name) + if 'rejection_expected' in control: + need('CONTROL PINNED' in log, 'Expected X4 construction rejection') + control_results[name] = {'rejected': control['rejection_expected']} + else: + need('OK ' + control['expected_sha'] in log, 'Expected X4 construction output') + control_results[name] = {'content_sha256': control['expected_sha']} + need(len(control_results) == 7, 'Complete predecessor construction control census') + dump(construction / 'CONTROLS.json', control_results) + + u8_source = sources[869] + u8_construction = work / 'baseline869-construction' + u8_construction.mkdir() + u8_word = u8_construction / 'FULL-WORD.json.gz' + u8_recipe = u8_source / 'construction/barch/RECIPE-U8.json' + python('build-u8-word', u8_source / 'construction/barch/build_tri.py', + base_word, u8_recipe, u8_word) + need(payload_sha(u8_word) == read(u8_recipe)['expect_sha256'] == expected['u8_word_sha256'], + 'Rebuilt U8 source word') + need(sha(u8_source / 'independent/audit_local.py') == sha(dependency / 'independent/audit_local.py'), + 'Unchanged independent U8 auditor') + python('u8-construction-controls', u8_source / 'construction/barch/run_controls.py', + base_word, u8_recipe, u8_source / 'independent/audit_local.py', u8_construction / 'controls') + u8_controls = read(u8_construction / 'controls/controls.json') + need(len(u8_controls) == 38 and all(row['ok'] is True for row in u8_controls.values()), + 'Complete U8 construction control census') + + word = work / 'FULL-WORD.json.gz' + python('build-end-pair', PACKAGE / 'build_word.py', '--input', u8_word, '--output', word) + need(payload_sha(word) == expected['word_sha256'], 'Minimal candidate word') + python('independent-point-audit', PACKAGE / 'audit_points.py', '--baseline', u8_word, + '--word', word, '--output', work / 'POINT-AUDIT.json') + python('end-pair-controls', PACKAGE / 'end_controls.py', '--word', word, + '--independent-auditor', dependency / 'independent/audit_local.py', + '--native-auditor', pr635 / 'code/hpe_audit_and_transport.py', + '--portable-runner', native_source / 'code/run_portable.py', + '--routing', routing, '--main', root, '--output', work / 'end-controls') + owners = work / 'owners' + python('rebuild-owners', PACKAGE / 'build_owners.py', '--word', word, + '--plan', dependency / 'data/PLAN.json', '--out', owners) + need(payload_sha(owners / 'OWNERS.json.gz') == expected['owners_sha256'], 'Exact literal owner assignment') + native = work / 'native' + python('complete-native-replay', PACKAGE / 'code/replay_native.py', '--ancestor-package', ancestor, + '--dependency', dependency, '--raw', word, '--out', native) + native_result = read(native / 'RESULT.json') + need(native_result['pins'] == expected['native_pins'], 'Every proof-relevant native output') + admission = work / 'admission' + python('complete-global-admission', PACKAGE / 'code/admit_candidate.py', '--ancestor-package', ancestor, + '--ancestor-replay', baseline, '--dependency', dependency, '--checker-package', checker, + '--native', native, '--owners', owners / 'OWNERS.json.gz', + '--comparison-record', comparison, '--out', admission) + admitted = read(admission / 'RESULT.json') + for field in ('kappa', 'kappa_decimal', 'complex_lower', 'bit_lower', 'global_profile', 'gain'): + need(admitted[field] == expected[field], 'Frozen admitted result: ' + field) + result = dict(status='PASS_COLD_RANK7_END_PAIR_COMPLETE_ADMISSION', + kappa=admitted['kappa'], kappa_decimal=admitted['kappa_decimal'], gain=admitted['gain'], + baseline_kappa=admitted['baseline_kappa'], global_profile=admitted['global_profile'], + sources=pins, word_sha256=payload_sha(word), owners_sha256=payload_sha(owners / 'OWNERS.json.gz'), + native_pins=native_result['pins'], seconds=time.monotonic()-started, stages=stages, + scope='Fresh pinned ancestor, X4 and U8 constructions, minimal end recipe, independent point audit, ' + 'targeted controls, exact owner reconstruction, complete native and strict global admission. ' + 'The cited PR589 bit circuit is repriced. Inherited uniform and analytic assumptions remain.') + dump(work / 'RESULT.json', result) + outputs = ['RESULT.json', 'STAGES.json', 'POINT-AUDIT.json', 'FULL-WORD.json.gz', + 'baseline741/RESULT.json', 'baseline741/RECEIPT.json', 'baseline741/ANCESTRAL-LINK.json', + 'baseline799-construction/CONTROLS.json', 'owners/OWNERS.json.gz', + 'baseline869-construction/FULL-WORD.json.gz', 'baseline869-construction/controls/controls.json', + 'native/RESULT.json', 'native/RECEIPT.json', 'admission/RESULT.json', 'admission/RECEIPT.json'] + dump(work / 'RECEIPT.json', dict(status=result['status'], + artifacts={name: sha(work / name) for name in outputs}, + package_inputs={str(path.relative_to(PACKAGE)): sha(path) for path in PACKAGE.rglob('*') + if path.is_file() and '__pycache__' not in path.parts})) + print(result['status'], result['kappa_decimal'], flush=True) + + +if __name__ == '__main__': + main()