Measure code placement instead of arguing about it (#240)

[kurok]

Part of #240 — deliverables 1 and 2. Deliverable 3 (the alignment-flag decision) needs dispatches that cannot run until this merges; see What is not here.

Branched from master (c403a69).

Why this went from housekeeping to the critical path today

#240 was filed as priority: medium. It stopped being that this morning.

Three consecutive PRs failed the benchmark gate on test__fast_mail_parser___parse_qp_message:

PR	what it changes	parse_qp_message on x86
#246	the headers dict cache	qp_message_metadata +7.3%
#247	the parse_many thread scheduler	+19.0%
#248	a header-value fast path	+9.2% → +7.5% → +12.9%

#247 is the control that settles what this is. It touches src/fast_mail_parser.rs and src/mail_parser.rs only — nothing within reach of quoted-printable decoding. Its own benchmark works perfectly (parse_many_small_page −55.6%), qp_message_metadata is −0.3% flat, and parse_qp_message reads +19.0% with per-round minima of 0.279 / 0.280 / 0.278. A thread-pool change cannot slow a decoder it never calls, and that is not noise.

On #248 I spent three attempts patching it with #[inline(never)] — first on the code I had changed, then on qp::decode_robust itself. Against an unmoving 0.246 ms base the three binaries read 0.268, 0.265, 0.277. It does not converge. That is what this PR is a response to: the repo has no instrument for the thing that is failing its gate, so every response to it is a guess.

What this adds

.github/workflows/layout-ab.yml (dispatch-only) builds the same checkout salts times, each with RUSTFLAGS="-C metadata=layout<k>" and its own CARGO_TARGET_DIR, then measures every side interleaved in one job. -C metadata is hashed into the crate's StableCrateId — the same thing a version bump perturbs (#204) — so the sides do identical work and differ only in where it landed. Unlike the version-string sed it keeps the wheel version and filename identical and needs no git checkout Cargo.toml afterwards.

.github/scripts/layout_spread.py reports, per group and benchmark, the per-salt medians, the spread, the control floor, and a sensitive/not classification; with two groups it adds a cost(b) table for the alignment candidate. It imports read_mins, read_machine and is_control from ab_median.py rather than copying them.

Two guards, because an instrument that quietly measures nothing is the failure mode here:

• every side's .so sha256 must be pairwise distinct — a salt that produced identical bytes perturbed nothing, and the run would report a reassuring 0% that means only that;
• reports with no control benchmark are refused outright, because without the pure-Python floor every spread is uninterpretable.

The classification rule is deliberately conservative: layout-sensitive iff the spread clears both the control floor and 3%. Three, because #205 measured the version-bump effect at 3.2–4.4% on the gate's own CPU — below that a four-salt sweep cannot separate placement from the residual it sits in, and claiming otherwise would give this a resolution it does not have.

Tests

tests/test_layout_spread.py pins cases (a)–(g) from the issue plus two more — 9 tests, all passing:

case	pins
(a)	identical sides → 0.0% spread, nothing sensitive
(b)	+10% treatment over a flat control → sensitive
(c)	same +10% but a 12% control → not sensitive (floor rule)
(d)	+2.5% over a 0.1% floor → not sensitive (3% rule)
(e)	two groups → cost table, correct to 0.1%
(f)	the CPU line is reported
(g)	no control benchmark → nonzero exit with ::error::
+	test__threaded___* is never classified sensitive, matching the gate
+	a misnamed report is refused rather than silently mis-parsed

Cases (c) and (d) are the ones that matter: they are the two ways this could wrongly tell someone their code is fine when the runner was just noisy, or wrongly send them hunting a layout ghost.

What is not here, and why

The dispatches. workflow_dispatch only works for workflows present on the default branch, so layout-ab.yml cannot be run until this merges. The acceptance criteria ask for two plain dispatches plus two per candidate flag with their summaries pasted here; I cannot produce those yet, and I would rather say so than imply the instrument has been exercised on real hardware. It has not — only its analysis path is tested, against synthetic reports.

The alignment-flag decision (deliverable 3) is conditional on those numbers, so it is not here either. .cargo/config.toml is untouched and the shipped wheels are byte-identical to master.

CONTRIBUTING.md no longer says "nobody has measured how sensitive they are" — but it does not yet claim a measured spread either. It records today's three failures as the evidence that the effect is live, and points at the dispatch command. When the dispatches run, that paragraph should be rewritten again with the real per-benchmark numbers, and ab_median.py's significant-verdict prose should quote the measured bound.

Suggested order

Merge this, then dispatch twice plain. If the spread on parse_qp_message is large — which today's three failures suggest — that number explains #246 and #247 outright, and they can be judged against it rather than against a 7% threshold that turns out to be narrower than the instrument's own scatter.

Checks run locally

pytest tests --ignore=tests/benchmark (820 passed, 3 skipped), ruff check . clean, the workflow YAML parses and its inputs are as documented. No src/, vendor/, fast_mail_parser/, __init__.pyi or tests/test_contract.py change; aarch64/apple build flags untouched.

[kurok]

Field evidence for why this workflow is needed, gathered since opening it.

I rebased #246 and #247 onto c403a69 — no code change to either, just a new base and therefore a new binary layout. Both had been failing the gate. Result:

• Cache the headers dict per object, collapse the six getters, and freeze the result classes (#231) #246 went from failing (+7.3% on parse_qp_message_metadata) to passing 15/15.
• Size parse_many's workers by bytes, and let the caller be one (#232) #247's failing benchmark moved. parse_qp_message went from +19.0% to −6.9%; the gate now fails on parse_tree_lazy_untouched at +15.2%, with parse_many_metadata +11.9%, parse_lazy_untouched +11.1% and parse_tree +9.3% — against a 0.5% control floor.

So on one revision, one CPU, unchanged source: placement alone moves the parse benchmarks ±15%, and which benchmark trips the 7% threshold is drawn fresh each build.

That is the spread this workflow measures directly rather than by inference, and it says the gate is currently trying to resolve 7% inside a band twice that wide. Two plain dispatches would turn the estimate above into a number, per benchmark, with a control floor under it — and would tell you whether the threshold, the benchmark set, or the build flags are what needs to change.

Nothing here changes any of that yet; this PR only adds the ability to find out.