Mister-Meeseeks · Oct 14, 2026 · Oct 14, 2026
diff --git a/research/test-suite-locale-reads/README.md b/research/test-suite-locale-reads/README.md
@@ -0,0 +1,166 @@
+# test-suite-locale-reads
+
+**No kappa, no certificate change.** One reproducible environment failure
+repaired: the repository's documented test command fails on a native Windows
+cp1252 host because `tests/test_research_current_status.py` reads
+`docs/research/current-status.md` with the locale codec.
+
+## Task card
+
+- **Task ID / scope key:** `verifier-engineering` / `portable-replay`
+  (scope key `831cc1a3c8f9ff01b197`, chooser OS nonce
+  `c7669648496a57422bfa478f1d356cad`, priority revision 3).
+- **Base commit:** `0b1cccf86e2f5132ed286a26ef0a681a495740ea` (selected-record
+  SHA-256 `10b9e340fd8e354a37a0c4813c90e5de1dd42044b5f6aca4a3fc5a42b685fa32`).
+- **Concrete object and question:** the suite's own document reads. On Windows
+  with the Git-for-Windows defaults, `python3 -B -m unittest discover -s tests
+  -p 'test_research_*.py'` (without `PYTHONUTF8`) fails with three errors in
+  `test_research_current_status.CurrentStatusTests`, all from
+  `STATUS.read_text()` decoding `docs/research/current-status.md` with cp1252
+  while the document contains byte `0x9d` (in U+201D). Can this be repaired
+  minimally and pinned by a regression?
+- **Scope status:** refined, within the card's premise ("one reproducible
+  environment failure repaired") and its allowed variation (one
+  interpreter/environment mismatch). Reason recorded in [TASK.json](TASK.json).
+- **Existing restriction being changed:** none in the mathematics. The prior
+  Windows-encoding work (merged #1299/#1293-rgb line, credited below) covered
+  `scripts/research_package.py`, `scripts/find_overlap.py` and
+  `tests/test_research_onboarding.py`; its static guard
+  (`TextEncodingTests.SCRIPTS`) covers five onboarding scripts only, and does
+  not cover the suite's other document readers.
+- **Method; parameter range:** reproduce on native Windows cp1252 (Python
+  3.13.15), fix the module's text reads to explicit UTF-8, add a static
+  locale-independence guard (the same AST pattern the onboarding tests use) and
+  a load-bearing byte control, and register a bounded replay that forces a
+  legacy codec (cp1252 on Windows, the C locale elsewhere).
+- **Nearest overlapping PRs:** https://github.com/CrocSwap/integer-mult-bounds/pull/1702
+  (makes the CP1252 overlap regression in `test_research_onboarding.py`
+  self-contained; different file and object, no conflict with this change) and
+  the merged #1299 line (predecessor). The gzip/canonical-payload variation of
+  the card is under active concurrent work
+  (https://github.com/CrocSwap/integer-mult-bounds/pull/1559,
+  https://github.com/CrocSwap/integer-mult-bounds/pull/1824) and was not
+  duplicated. Overlap refreshed at 2026-10-14 11:10Z and again before
+  submission.
+- **Optimistic full cost budget:** trivial runtime (the suite runs in about
+  15 s). Required observable improvement: the documented command exits 0 with
+  and without `PYTHONUTF8` on native Windows, and the new guard fails on the
+  base-commit module.
+- **Success evidence; falsification; cap:** see below. Falsification test: the
+  static guard reports exactly the base module's bare reads (lines 10 and 26),
+  and a bare locale read of the document fails in a forced-legacy child. One
+  session, one pivot (the historical-failure candidate was killed first).
+
+## Scope and evidence
+
+**What was reproduced (before).** Windows 11, cp1252, Git 2.55.0.windows.5
+(`core.autocrlf=true`), CPython 3.13.15. At base `0b1cccf8`, without
+`PYTHONUTF8`:
+
+```text
+ERROR: test_current_section_matches_selected_record ... setUp
+  File ".../tests/test_research_current_status.py", line 26, in setUp
+    self.text = STATUS.read_text()
+UnicodeDecodeError: 'charmap' codec can't decode byte 0x9d in position 1014
+Ran 42 tests in 13.787s — FAILED (errors=3, skipped=2)
+```
+
+The same command with `PYTHONUTF8=1`, and everything under CPython 3.12,
+passed. The three errors are one root cause: a locale-dependent read of a UTF-8
+document.
+
+**Obstruction recorded (killed first candidate).** The historical Windows
+failures cited in many PR bodies no longer reproduce at base:
+`test_research_onboarding` runs green natively (Python 3.13.15: 18 tests with 2
+environment-gated skips; Python 3.12: all 18 pass, `core.autocrlf=true`),
+including the formerly failing scaffold-pin and
+`os.killpg` tests. Credit for that repair goes to the merged
+#1299/#1293-rgb line ("UTF-8 metadata/pipe handling and Windows child-tree
+timeout fallback", recorded in `research/community-efficient400-audit`) and to
+the feedback report in
+https://github.com/CrocSwap/integer-mult-bounds/discussions/1200#discussioncomment-18910079,
+which identified the same byte (`0x9d`) in `research_package.py`. This package
+fixes the instance that line missed and pins it.
+
+**What is established (after).** The module's reads now pass explicit
+`encoding="utf-8"`. Two regression tests were added in the same module, in the
+style of `test_research_onboarding.TextEncodingTests` ("a UTF-8 CI host cannot
+observe locale-codec bugs, so check the calls statically"): an AST guard
+requiring `encoding=` on every text read/write in the module, and a
+load-bearing control asserting the document contains `e2 80 9d` and is
+undecodable as cp1252 (deterministic on every host). Evidence:
+
+```text
+python -B -m unittest discover -s tests -p 'test_research_*.py'   # no PYTHONUTF8, 3.13.15
+Ran 44 tests in 14.672s — OK (skipped=2)
+python3.12 -B -m unittest discover -s tests -p 'test_research_*.py'
+Ran 44 tests in 16.541s — OK
+```
+
+Failure controls (both run in this session): the static guard applied to the
+base-commit module reports missing `encoding=` at exactly lines 10 and 26; a
+bare `read_text()` of the document in a `PYTHONUTF8=0` child fails with the
+same `UnicodeDecodeError` on this host. The registered replay
+(`verify.py`) reruns all of this with a forced legacy codec.
+
+**Limitations (not checked).** No mathematics: no kappa, certificate, frozen
+pin, `upstream/`, selected-record or cold-replay change; the acceptance gate
+"preserve immutable source hashes and proof-relevant content" is untouched
+because no comparison logic changed. The roughly 100 remaining
+locale-dependent text calls in `tests/` and the one-off `scripts/` generators
+were inventoried but not swept (open question 1). The registered replay forces
+the legacy codec only for the repaired module; whole-suite behavior under a
+true POSIX ASCII locale is unmeasured (open question 2). macOS and Linux hosts
+were not executed here. [SOURCES.json](SOURCES.json) pins
+`tests/test_research_current_status.py` at its **base-commit** bytes as
+provenance (the defective version this PR repairs); it cannot be a live input
+under the pin contract because this contribution changes it.
+`docs/research/current-status.md` is the live input and matches its pin.
+
+## Reproduce
+
+From repo root, on any host:
+
+```sh
+python3 scripts/research_package.py --package research/test-suite-locale-reads --check-only
+python3.12 scripts/research_package.py --package research/test-suite-locale-reads
+```
+
+The second command runs `verify.py`: the cp1252 load-bearing control, the bare
+legacy-locale read failure control, the module's unittest under a forced legacy
+codec and the default environment, and the documented suite command in the
+default environment. The native-Windows reproduction of the original failure
+needs no `PYTHONUTF8`; on POSIX the replay forces `LC_ALL=C` with
+`PYTHONCOERCECLOCALE=0` and `PYTHONUTF8=0`.
+
+## Open next questions
+
+1. **Sweep the remaining locale-dependent reads in `tests/`** (about 100 calls
+   across about 40 modules). Why it matters: only `current-status.md` carries
+   cp1252-undefined bytes today, so only one module crashes; the next
+   typographic quote added to any read document recreates the same bogus
+   errors elsewhere. First step: extend the static AST guard to all
+   `tests/*.py` and fix the call sites that read documents already containing
+   non-ASCII bytes. Blocker: mechanical churn across frozen-era test modules
+   wants maintainer buy-in.
+2. **Whole-suite guarantee under a POSIX ASCII locale.** Why it matters:
+   "green on Windows cp1252" and "green on `LC_ALL=C`" are different claims;
+   the registered replay only forces the legacy codec for the repaired module.
+   First step: run the full discover command under `PYTHONUTF8=0
+   PYTHONCOERCECLOCALE=0 LC_ALL=C` on Linux and triage failures. Blocker: this
+   session ran on Windows only.
+
+## Credit
+
+Prepared by Aaron Wang, with help from opencode. New work is Apache-2.0.
+
+Predecessor credit: the merged #1299/#1293-rgb Windows encoding line
+(UTF-8 metadata and pipe handling, child-tree timeout fallback, cp1252
+simulations and the `TextEncodingTests` static-guard pattern, summarized in
+`research/community-efficient400-audit`); the failure report and environment
+details from
+https://github.com/CrocSwap/integer-mult-bounds/discussions/1200#discussioncomment-18910079;
+the `docs/research/current-status.md` consistency tests themselves
+(`tests/test_research_current_status.py`, authored upstream of this base). This
+package adds the instance their guard missed, a regression in the module that
+failed, and a replayable legacy-locale check. No mathematical claim is made.
diff --git a/research/test-suite-locale-reads/SOURCES.json b/research/test-suite-locale-reads/SOURCES.json
@@ -0,0 +1,30 @@
+{
+  "schema_version": 1,
+  "base_commit": "0b1cccf86e2f5132ed286a26ef0a681a495740ea",
+  "files": [
+    {
+      "path": "certificates/selected-result.json",
+      "usage": "provenance",
+      "git_blob": "ae6a99f28aa468649eb66a361e764d769363bf11",
+      "sha256": "10b9e340fd8e354a37a0c4813c90e5de1dd42044b5f6aca4a3fc5a42b685fa32"
+    },
+    {
+      "path": "docs/research/agent-task-catalog.json",
+      "usage": "provenance",
+      "git_blob": "06c96f5579bb67ed1b86d334e78fbed7da6945f1",
+      "sha256": "6686ad58f1fbdbc87134a78901b60fd53a20d3984355e91defbbebc9e9fa8075"
+    },
+    {
+      "path": "docs/research/current-status.md",
+      "usage": "input",
+      "git_blob": "d73b4ff9da41f044e793422db4318be17e234a6c",
+      "sha256": "adb54af005eb11edaf84d782fe729338680ff908f6d4fbf39ba1062d2e7d8a77"
+    },
+    {
+      "path": "tests/test_research_current_status.py",
+      "usage": "provenance",
+      "git_blob": "356f49df3c7f8a3d6266322fabc8f47cc4666f93",
+      "sha256": "9698ecb1f989a2a8282aecaabb8e13e433fc5dfdd0423b6a3ca6e690757c2b4a"
+    }
+  ]
+}
diff --git a/research/test-suite-locale-reads/TASK.json b/research/test-suite-locale-reads/TASK.json
@@ -0,0 +1,67 @@
+{
+  "schema_version": 1,
+  "task_id": "portable-replay",
+  "priority_revision": 3,
+  "base_commit": "0b1cccf86e2f5132ed286a26ef0a681a495740ea",
+  "original_card": {
+    "id": "portable-replay",
+    "question": "Can one reproducible environment or artifact-comparison failure be repaired?",
+    "starting_points": [
+      "research/gzip-container-reproducibility/README.md",
+      "docs/ci-verification.md"
+    ],
+    "methods": [
+      "Reproduce one portability failure and add a minimal cross-environment fix.",
+      "Audit one canonical-payload comparison for omitted proof-relevant fields."
+    ],
+    "acceptance_gate": "Preserve immutable source hashes and proof-relevant content; a different compressed byte stream alone is not a mathematical mismatch.",
+    "first_step": {
+      "command": [
+        "python3",
+        "-B",
+        "scripts/research_baseline.py"
+      ],
+      "expected": "PASS_RESEARCH_BASELINE_BINDINGS_ONLY",
+      "inspect": "Read the gzip-container reproducibility examples and distinguish frozen source bytes from regenerated payloads.",
+      "scope": "Positive control / source-bound orientation only; this is not a new construction or full cold replay."
+    },
+    "first_experiment": "Reproduce one specific mismatch or audit one canonical comparison for omitted proof-relevant fields.",
+    "bottleneck_link": "Portable exact checks prevent environment failures from blocking valid contributions.",
+    "allowed_variations": [
+      "One interpreter/version mismatch or one payload field comparison."
+    ],
+    "scope_change_triggers": [
+      "Removing immutable-source pins because generated compressed bytes differ."
+    ],
+    "overlap_terms": [
+      "gzip container",
+      "reproducibility",
+      "canonical payload"
+    ]
+  },
+  "assignment_seed": "c7669648496a57422bfa478f1d356cad",
+  "scope_status": "refined",
+  "actual_scope": "Repair one reproducible environment failure: on a native Windows cp1252 host the documented check python3 -B -m unittest discover -s tests -p 'test_research_*.py' errors in tests/test_research_current_status.py because its text reads use the locale codec while docs/research/current-status.md carries byte 0x9d (U+201D). Fix the reads to explicit UTF-8, add a static locale-independence guard and a load-bearing byte control in the same module, and register a bounded legacy-locale replay.",
+  "scope_change_reason": "Refined within the card's premise (one reproducible environment failure repaired) and its allowed variation (one interpreter/environment mismatch). The card's obvious first candidates were killed first: the historical Windows failures cited by many PR bodies (autocrlf temp-fixture pin mismatch in test_research_onboarding.PackageTests, POSIX-only os.killpg, cp1252 metadata reads in research_package.py/find_overlap.py) no longer reproduce at base 0b1cccf8 on native Windows (test_research_onboarding runs green: Python 3.13.15 has 18 tests with 2 environment-gated skips, Python 3.12 passes all 18 with core.autocrlf=true; the merged #1299/#1293-rgb line already repaired them). What remains is the suite's own document reads, which that line's five-script static guard (test_research_onboarding.TextEncodingTests.SCRIPTS) does not cover. The gzip/canonical-payload variation of the card is under active concurrent work (https://github.com/CrocSwap/integer-mult-bounds/pull/1559, https://github.com/CrocSwap/integer-mult-bounds/pull/1824) and was deliberately not duplicated.",
+  "bottleneck_link": "Portable exact checks prevent environment failures from blocking valid contributions. The documented suite command is the check every reviewer asks contributors to run; on the most common desktop configuration (Windows, Git-for-Windows defaults) it reports three bogus errors unless the contributor knows to set PYTHONUTF8=1, which is how several PR bodies came to disclaim 'pre-existing Windows failures' instead of getting a clean signal.",
+  "budget_minutes": 60,
+  "stop_rule": "One repair of tests/test_research_current_status.py plus one bounded package and one focused PR. No sweep of the ~100 remaining locale-dependent text calls in tests/ or the scripts/ one-off generators (inventoried as an open question). No frozen-pin, upstream, selected-record or certificate changes. One pivot maximum: the pivot from the killed historical-failure candidate to the suite's document reads is the one pivot used.",
+  "outcome": "completed",
+  "evidence": "Native Windows 11 (cp1252, Git 2.55 autocrlf=true), CPython 3.13.15. BEFORE (base 0b1cccf8): python -B -m unittest discover -s tests -p 'test_research_*.py' without PYTHONUTF8 -> 'Ran 42 tests ... FAILED (errors=3)', all three in test_research_current_status.CurrentStatusTests.setUp, UnicodeDecodeError: 'charmap' codec can't decode byte 0x9d in position 1014. Same command with PYTHONUTF8=1 -> OK (skipped=2); python3.12 -> OK. AFTER (this change): same command without PYTHONUTF8 -> 'Ran 44 tests ... OK (skipped=2)' on Python 3.13.15 and 'Ran 44 tests ... OK' on Python 3.12, with and without PYTHONUTF8. Failure controls (run recorded): the base-commit module carries bare reads at lines 10 and 26 under the new static guard (guard reports exactly those lines); a bare locale read of docs/research/current-status.md in a PYTHONUTF8=0 child fails with the same UnicodeDecodeError on this host, and the module's new test asserts the load-bearing byte sequence e2 80 9d and cp1252 undecodability on every host. Registered replay: python3.12 scripts/research_package.py --package research/test-suite-locale-reads passes; metadata check --check-only passes. scripts/research_baseline.py reports PASS_RESEARCH_BASELINE_BINDINGS_ONLY (card first step, unchanged).",
+  "author": "Aaron Wang",
+  "ai_assistance": "with help from opencode",
+  "open_questions": [
+    {
+      "question": "Should the remaining locale-dependent text reads in tests/ (~100 calls across ~40 modules) be swept the way test_research_onboarding.TextEncodingTests guards the five onboarding scripts?",
+      "why_it_matters": "Only tests/test_research_current_status.py crashes today because only docs/research/current-status.md carries cp1252-undefined bytes among the documents the suite reads; any typographic quote added to formal/lean/SOURCES.md, a workflow yml or a vendored .tex reintroduces the same bogus errors in another module.",
+      "first_step": "Extend the static AST guard to all tests/*.py, triage which reads target documents that already contain non-ASCII bytes (a byte scan of the repo finds 1536 non-ASCII files, mostly under research/), and fix those call sites first.",
+      "blocked_by": "Nothing technical; the mechanical churn across frozen-era test modules wants maintainer buy-in before a sweep PR."
+    },
+    {
+      "question": "Can the whole suite, not just tests/test_research_current_status.py, be guaranteed under a forced legacy codec on POSIX?",
+      "why_it_matters": "The registered replay forces the C locale only for the repaired module; other modules' reads could still fail under a true ASCII locale on Linux even though they pass under cp1252, so 'green on Windows' and 'green on LC_ALL=C' are different claims.",
+      "first_step": "Run the full discover command under PYTHONUTF8=0 PYTHONCOERCECLOCALE=0 LC_ALL=C on a Linux host and triage whatever fails.",
+      "blocked_by": "This session ran on Windows only; the POSIX C-locale behavior of the unmodified modules is unmeasured."
+    }
+  ]
+}
diff --git a/research/test-suite-locale-reads/VERIFY.json b/research/test-suite-locale-reads/VERIFY.json
@@ -0,0 +1,21 @@
+{
+  "schema_version": 1,
+  "status": "ready",
+  "python": "3.12",
+  "requirements": [],
+  "inputs": [
+    "docs/research/current-status.md",
+    "tests/test_research_current_status.py"
+  ],
+  "commands": [
+    {
+      "argv": [
+        "{python}",
+        "-B",
+        "{package}/verify.py"
+      ],
+      "timeout_seconds": 300,
+      "checks": "Locale-independence of tests/test_research_current_status.py: cp1252 and bare-read failure controls on docs/research/current-status.md, the module's unittest run under a forced legacy codec (cp1252 on Windows, C locale elsewhere) and under the default environment, and the documented suite command in the default environment. No frozen-pin, replay or selected-record checks."
+    }
+  ]
+}