From 05722d4cc5d7f7112f845286a8828e60b2587579 Mon Sep 17 00:00:00 2001 From: Claude Date: Wed, 16 Sep 2026 22:07:48 +0000 Subject: [PATCH 01/13] probe: does the tail descent pay for the mask-ALGEBRA ops, end to end? MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Probe-first, before building anything. The 5-8x descent win from #311 was measured TAIL-ONLY. That says nothing about whether the tail is a meaningful fraction of a whole `mask_and` call, and the answer decides whether a `U64x4`/`U64x2` facade surface across six backend files is worth building. ANSWER: yes, decisively, and it holds on the portable v3 tier as well as on v4 — the bitwise descent needs only AVX2 and SSE2, no AVX-512VL. That covers 8 of the 11 mask-algebra tail sites; only `mask_ternlog` needs VL. Tail cost, ISOLATED (AVX-512 host): body k padded descent ratio 8 1 8.41 ns 0.58 ns 14.4x 8 7 14.74 ns 1.76 ns 8.4x 64 2 10.30 ns 0.60 ns 17.3x 64 7 16.88 ns 3.35 ns 5.0x The padded tail costs 6-17 ns against a body of ~3.5 ns at 8 words, so today the tail is routinely LARGER THAN THE WORK IT TRAILS. Several descent values land below the noise of the body they were differenced against, i.e. its tail is indistinguishable from having no tail. And 7 of 8 possible mask sizes have a tail (`n % 8 != 0`). TWO MEASUREMENT ERRORS THIS PROBE MADE AND THEN CORRECTED — both recorded in the module docs, because each one produced a plausible number that was not about the variable under test: 1. CONFOUNDED BODIES. The first version gave the arms different bodies (`as_chunks::<8>()` + nested loop vs a flat `step_by(4)` loop) and reported the difference as a tail result. It is not: at n = 256/1024/4096, where `n % 8 == 0` and there is NO TAIL, it still showed D/P of 0.84/0.76/0.72. A tail strategy cannot move a call with no tail. Fixed by giving both arms ONE shared `body()`. 2. THE RESIDUAL SURVIVED, so I tested cache order — and order was NOT the cause: both orders agree (0.698 vs 0.717 at n=1024). Something about how the two functions compile still differs at `tail == 0`, which means every raw ratio carries that unknown alongside the tail. So the headline table CANNOT be read as a tail result on its own, and the probe says so in its own docs. The isolating measurement is difference-in-differences: `t(base + k) - t(base)` at fixed body size cancels whatever each arm's body compiled to and leaves the k-word tail alone. That is the table quoted above. Both arms are gated bit-identical against each other AND against a scalar reference before any timing, so a faster-but-wrong descent cannot be reported as a win. `black_box` on inputs and outputs; both timing orders reported. Gates: clippy -D warnings clean on native and v3; fmt clean; runs green on both tiers. Co-Authored-By: Claude Opus 5 Claude-Session: https://claude.ai/code/session_01DCfrD5y19cvFc4AoyydXYv --- examples/mask_algebra_tail_probe.rs | 263 ++++++++++++++++++++++++++++ 1 file changed, 263 insertions(+) create mode 100644 examples/mask_algebra_tail_probe.rs diff --git a/examples/mask_algebra_tail_probe.rs b/examples/mask_algebra_tail_probe.rs new file mode 100644 index 00000000..ff0c4771 --- /dev/null +++ b/examples/mask_algebra_tail_probe.rs @@ -0,0 +1,263 @@ +//! Does the tail DESCENT pay for the mask-ALGEBRA ops, end to end? +//! +//! # The question, and why it is not already answered +//! +//! `examples/ternlogq_tail_descent_probe.rs` measured the descent **tail only** +//! and found it 5-8x faster than zero-padding a `U64x8`. That result is real +//! and it is about one operation. What it does NOT say is whether the tail is a +//! meaningful FRACTION of a whole `mask_and` / `mask_or` / `mask_xor` call: +//! over `n` words the body runs `n/8` register ops and the tail runs ONE, so at +//! `n = 1024` a 5x-faster tail moves ~1/128 of the work and the end-to-end +//! effect should be invisible. At `n < 8` the tail IS the whole call. +//! +//! Somewhere between those two the descent stops mattering. This probe finds +//! where, so the decision to build a `U64x4`/`U64x2` facade surface across six +//! backend files rests on a measurement rather than on the tail-only number. +//! +//! # Arms +//! +//! - **P — padded**: exactly what `mask_and` does today. `as_chunks::<8>()` +//! body, then both operands zero-padded into a `U64x8` and the first +//! `tail.len()` lanes copied back out. +//! - **D — descent**: same body, then the remainder walked greedily 4 -> 2 -> 1 +//! with `_mm256_and_si256` / `_mm_and_si128` / scalar `&`. Every lane live, +//! no zero-init, no `copy_from_slice` of a padded array. +//! +//! Both arms produce identical output; the probe asserts that before timing, so +//! a faster-but-wrong descent cannot be reported as a win. +//! +//! # The confound this probe had, and why the arms now share a body +//! +//! The first version gave the two arms DIFFERENT bodies — `as_chunks::<8>()` +//! with a nested 2x256-bit loop for P, a flat `step_by(4)` loop for D — and +//! then reported the difference as a tail result. It is not: at `n = 256`, +//! `1024` and `4096`, where `n % 8 == 0` and there is NO TAIL AT ALL, it still +//! showed `D/P` of 0.844, 0.763 and 0.716. A tail strategy cannot move a call +//! that has no tail, so those rows were measuring body shape. +//! +//! Both arms now call ONE shared `body()` and differ only after it. Any +//! remaining `D/P != 1` at `n % 8 == 0` would be noise or layout, never the +//! variable under test — which makes those rows the probe's own control. +//! +//! # Why this needs no AVX-512 +//! +//! `_mm256_and_si256` is AVX2 and `_mm_and_si128` is SSE2. The bitwise descent +//! for and/or/xor/andnot needs **no AVX-512VL** at all — only the `ternlog` +//! descent does, because `_mm256_ternarylogic_epi64` is a VL instruction. So +//! this arm runs on the portable v3 baseline as well as on v4, and the result +//! applies to 8 of the 11 mask-algebra tail sites. +//! +//! Run: `cargo run --release --example mask_algebra_tail_probe` + +fn main() { + #[cfg(not(target_arch = "x86_64"))] + { + println!("mask_algebra_tail_probe: x86_64 only (uses AVX2/SSE2 intrinsics); skipping"); + } + #[cfg(target_arch = "x86_64")] + { + if !std::arch::is_x86_feature_detected!("avx2") { + println!("mask_algebra_tail_probe: needs AVX2 at runtime; skipping"); + return; + } + // SAFETY: avx2 confirmed present by the runtime check above; every + // callee below is `#[target_feature(enable = "avx2")]` and every access + // it makes is bounded by the slice lengths asserted equal in `run`. + unsafe { imp::run() } + } +} + +#[cfg(target_arch = "x86_64")] +mod imp { + use std::arch::x86_64::*; + use std::hint::black_box; + use std::time::Instant; + + /// The SHARED body, identical in both arms: full 4-word registers over the + /// first `n & !7` words. Factored out deliberately — see the module note on + /// the confound this removes. + #[target_feature(enable = "avx2")] + unsafe fn body(a: &[u64], b: &[u64], dst: &mut [u64], upto: usize) { + for i in (0..upto).step_by(4) { + // SAFETY: i + 4 <= upto <= n = a.len() = b.len() = dst.len(). + let va = _mm256_loadu_si256(a.as_ptr().add(i).cast()); + let vb = _mm256_loadu_si256(b.as_ptr().add(i).cast()); + _mm256_storeu_si256(dst.as_mut_ptr().add(i).cast(), _mm256_and_si256(va, vb)); + } + } + + /// P — shared body, then the CURRENT tail: both operands zero-padded into a + /// `[u64; 8]`, one full-width op, first `t` lanes copied back. + #[target_feature(enable = "avx2")] + unsafe fn and_padded(a: &[u64], b: &[u64], dst: &mut [u64]) { + let n = a.len(); + let done = n & !7; + body(a, b, dst, done); + let t = n - done; + if t != 0 { + let mut pa = [0u64; 8]; + let mut pb = [0u64; 8]; + pa[..t].copy_from_slice(&a[done..]); + pb[..t].copy_from_slice(&b[done..]); + let mut out = [0u64; 8]; + for k in 0..2 { + // SAFETY: pa/pb/out are [u64; 8]; k in 0..2 so the 256-bit + // access at u64 index 0 or 4 stays inside 64 bytes. + let va = _mm256_loadu_si256(pa.as_ptr().add(k * 4).cast()); + let vb = _mm256_loadu_si256(pb.as_ptr().add(k * 4).cast()); + _mm256_storeu_si256(out.as_mut_ptr().add(k * 4).cast(), _mm256_and_si256(va, vb)); + } + dst[done..].copy_from_slice(&out[..t]); + } + } + + /// D — the SAME shared body, then the remainder walked greedily 4 -> 2 -> 1. + /// Every lane live, no zero-init, no padded copy. + #[target_feature(enable = "avx2")] + unsafe fn and_descent(a: &[u64], b: &[u64], dst: &mut [u64]) { + let n = a.len(); + let done = n & !7; + body(a, b, dst, done); + let mut i = done; + while n - i >= 4 { + // SAFETY: loop condition guarantees i + 4 <= n. + let va = _mm256_loadu_si256(a.as_ptr().add(i).cast()); + let vb = _mm256_loadu_si256(b.as_ptr().add(i).cast()); + _mm256_storeu_si256(dst.as_mut_ptr().add(i).cast(), _mm256_and_si256(va, vb)); + i += 4; + } + while n - i >= 2 { + // SAFETY: loop condition guarantees i + 2 <= n. SSE2, 128-bit. + let va = _mm_loadu_si128(a.as_ptr().add(i).cast()); + let vb = _mm_loadu_si128(b.as_ptr().add(i).cast()); + _mm_storeu_si128(dst.as_mut_ptr().add(i).cast(), _mm_and_si128(va, vb)); + i += 2; + } + while i < n { + dst[i] = a[i] & b[i]; + i += 1; + } + } + + fn bench(f: unsafe fn(&[u64], &[u64], &mut [u64]), a: &[u64], b: &[u64], dst: &mut [u64], iters: u32) -> f64 { + // SAFETY: caller only passes the two avx2 fns above, under the runtime + // avx2 check in `main`; lengths are equal (asserted in `run`). + unsafe { + for _ in 0..(iters / 8).max(1) { + f(black_box(a), black_box(b), black_box(dst)); + } + let t = Instant::now(); + for _ in 0..iters { + f(black_box(a), black_box(b), black_box(dst)); + } + let e = t.elapsed(); + black_box(&dst[0]); + e.as_secs_f64() * 1e9 / f64::from(iters) + } + } + + pub unsafe fn run() { + println!("mask_algebra_tail_probe: mask_and, padded tail vs 4->2->1 descent"); + println!("arch=x86_64 avx2=true (AVX2/SSE2 only — no AVX-512, no VL)\n"); + println!( + "{:>7} {:>6} {:>9} {:>9} {:>6} {:>6} verdict", + "n", "tail", "P padded", "D descent", "D/P", "D/P'" + ); + + for &n in &[1usize, 3, 7, 8, 9, 15, 16, 31, 64, 256, 1024, 4096] { + let a: Vec = (0..n) + .map(|i| 0xF0F0_5555_AAAA_1111u64 ^ (i as u64).wrapping_mul(0x9E37_79B9)) + .collect(); + let b: Vec = (0..n) + .map(|i| 0x0FF0_1234_5678_9ABCu64 ^ (i as u64).wrapping_mul(0x85EB_CA6B)) + .collect(); + let mut dp = vec![0u64; n]; + let mut dd = vec![0u64; n]; + assert_eq!(a.len(), b.len()); + + // Gate on equality BEFORE timing: a faster wrong arm is not a win. + and_padded(&a, &b, &mut dp); + and_descent(&a, &b, &mut dd); + assert_eq!(dp, dd, "n={n}: descent must be bit-identical to padded"); + let expect: Vec = a.iter().zip(&b).map(|(x, y)| x & y).collect(); + assert_eq!(dp, expect, "n={n}: padded arm must equal the scalar reference"); + + let iters = if n <= 64 { 2_000_000 } else { 200_000 }; + + // BOTH ORDERS. The arm timed second inherits a warm cache from the + // first, and at n >= 256 the three buffers exceed L1 — so a + // single-order probe systematically favours whichever arm runs + // last. Reporting both makes that bias visible instead of letting + // it masquerade as the effect under test. + let p1 = bench(and_padded, &a, &b, &mut dp, iters); + let d1 = bench(and_descent, &a, &b, &mut dd, iters); + let d2 = bench(and_descent, &a, &b, &mut dd, iters); + let p2 = bench(and_padded, &a, &b, &mut dp, iters); + + let r1 = d1 / p1; + let r2 = d2 / p2; + let spread = (r1 - r2).abs(); + let verdict = if spread > 0.15 { + "ORDER-DEPENDENT — not a result" + } else if r1.max(r2) < 0.90 { + "descent WINS" + } else if r1.min(r2) > 1.10 { + "descent LOSES" + } else { + "inert" + }; + println!("{n:>7} {:>6} {p1:>9.2} {d1:>9.2} {r1:>6.3} {r2:>6.3} {verdict}", n % 8); + } + // ── The isolating measurement: difference-in-differences ────────── + // + // The table above cannot be read as a tail result on its own. At + // `n % 8 == 0` both arms execute the SAME code (shared body, then a + // `t == 0` branch neither takes), yet they do not time the same — and + // the both-orders columns rule out cache order as the cause. Something + // about how the two functions compile differs, so every ratio above + // carries that unknown alongside the tail. + // + // Differencing removes it. For a fixed body size `base` (a multiple of + // 8), `t(base + k) - t(base)` is the cost of a k-word tail and nothing + // else: the body work is identical in both terms and cancels, per arm, + // whatever each arm's body compiled to. + println!("\n── tail cost ISOLATED: t(base + k) - t(base), body held constant ──"); + println!("{:>6} {:>3} {:>10} {:>10} {:>7}", "base", "k", "P tail ns", "D tail ns", "P/D"); + for &base in &[8usize, 64] { + let mk = |n: usize, seed: u64| -> Vec { + (0..n) + .map(|i| seed ^ (i as u64).wrapping_mul(0x9E37_79B9)) + .collect() + }; + let a0 = mk(base, 0xF0F0_5555_AAAA_1111); + let b0 = mk(base, 0x0FF0_1234_5678_9ABC); + let mut d0 = vec![0u64; base]; + let it = 2_000_000; + let p0 = bench(and_padded, &a0, &b0, &mut d0, it); + let q0 = bench(and_descent, &a0, &b0, &mut d0, it); + for k in 1..8usize { + let n = base + k; + let a = mk(n, 0xF0F0_5555_AAAA_1111); + let b = mk(n, 0x0FF0_1234_5678_9ABC); + let mut d = vec![0u64; n]; + let pk = bench(and_padded, &a, &b, &mut d, it); + let qk = bench(and_descent, &a, &b, &mut d, it); + let pt = pk - p0; + let dt = qk - q0; + let ratio = if dt > 0.0 { pt / dt } else { f64::NAN }; + println!("{base:>6} {k:>3} {pt:>10.2} {dt:>10.2} {ratio:>7.2}"); + } + } + println!("P/D > 1 means the padded tail costs that many times the descent's."); + println!("A NEGATIVE D-tail is not an error: it means the descent's tail is BELOW the"); + println!("run-to-run noise of the body it was differenced against — indistinguishable"); + println!("from having no tail at all. The ratio is NaN there because the denominator is"); + println!("noise, and reporting a number would invent precision the measurement lacks."); + + println!("\nns per call (P and D columns are the P-first pass)."); + println!("D/P = descent/padded with PADDED timed first."); + println!("D/P' = the same ratio with DESCENT timed first."); + println!("The two must agree; if they do not, the number is cache order, not the tail."); + println!("tail = n % 8, the remainder the two arms handle differently."); + } +} From a88d57c79528a2363e322e4e0f109e741bf9dac2 Mon Sep 17 00:00:00 2001 From: Claude Date: Wed, 16 Sep 2026 22:21:01 +0000 Subject: [PATCH 02/13] probe: measure against the REAL mask_and, with a measured noise floor MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Three review findings, all correct, all about the measurement rather than the conclusion. Each had produced a plausible number that was not about the variable under test. ## codex P2 — the padded arm was a STRAWMAN on v4 The comparator hand-rolled two 256-bit ops and called itself "exactly what mask_and does today". On an AVX-512 host that is false: production `mask_and` goes through `U64x8` and issues ONE `_mm512_and_si512`. So every native/v4 number compared the descent against something slower than the real thing. Fixed by calling `ndarray::simd::mask_and` itself. That also retires an earlier error permanently — there is no hand-written body left to diverge from production, so the "confounded bodies" defect cannot recur. ## codex P2 — noise handling was ASYMMETRIC A negative tail delta was reported as noise; an equally noise-sized POSITIVE delta was accepted and divided into a ratio. That is where the old table's 23x, 37x and 56x came from: dividing by a number indistinguishable from zero. Now the noise floor is MEASURED (max |repeat - repeat| over 14 same-input pairs) and applied to |delta|, both signs alike. A ratio prints only when BOTH terms clear it; when the padded tail clears and the descent's does not, the row says `D~noise`, which is the honest statement and is itself the finding. ## coderabbit — the isolating section dropped the order control The first table alternated arm order precisely because a fixed order biases the comparison; the difference-in-differences loop, which produces the HEADLINE, did not. Every sample now alternates, and each point is the median of five. ## The conclusion survived, measured against production Noise floor 1.35 ns (native/v4), 0.27 ns (v3). native/v4 P tail 12.8-23.7 ns D tail 0.65-3.69 ns 14 of 14 widths v3 P tail 8.3-22.3 ns D tail 0.06-1.56 ns ratios 10-36x Half the descent rows land below the noise floor: its tail is indistinguishable from having no tail. The padded tail is 8-24 ns against a body of ~3.5 ns at 8 words, so it is routinely LARGER THAN THE WORK IT TRAILS, and 7 of 8 mask sizes have one. Fixing the strawman made the case STRONGER, not weaker — production's padded tail costs more than my hand-rolled comparator did, not less. Gates: clippy -D warnings clean on native and v3; fmt clean; runs green on both tiers with the gate asserting both arms bit-identical to a scalar reference before any timing. Co-Authored-By: Claude Opus 5 Claude-Session: https://claude.ai/code/session_01DCfrD5y19cvFc4AoyydXYv --- examples/mask_algebra_tail_probe.rs | 420 +++++++++++++++------------- 1 file changed, 225 insertions(+), 195 deletions(-) diff --git a/examples/mask_algebra_tail_probe.rs b/examples/mask_algebra_tail_probe.rs index ff0c4771..3f54165c 100644 --- a/examples/mask_algebra_tail_probe.rs +++ b/examples/mask_algebra_tail_probe.rs @@ -1,58 +1,71 @@ //! Does the tail DESCENT pay for the mask-ALGEBRA ops, end to end? //! -//! # The question, and why it is not already answered +//! # The question //! //! `examples/ternlogq_tail_descent_probe.rs` measured the descent **tail only** -//! and found it 5-8x faster than zero-padding a `U64x8`. That result is real -//! and it is about one operation. What it does NOT say is whether the tail is a -//! meaningful FRACTION of a whole `mask_and` / `mask_or` / `mask_xor` call: -//! over `n` words the body runs `n/8` register ops and the tail runs ONE, so at -//! `n = 1024` a 5x-faster tail moves ~1/128 of the work and the end-to-end -//! effect should be invisible. At `n < 8` the tail IS the whole call. +//! and found it 5-8x faster than zero-padding a `U64x8`. That is one operation. +//! It does NOT say whether the tail is a meaningful FRACTION of a whole +//! `mask_and` call: over `n` words the body runs `n/8` register ops and the +//! tail runs ONE, so at large `n` a faster tail should be invisible, while at +//! `n < 8` the tail IS the whole call. Somewhere between, it stops mattering. //! -//! Somewhere between those two the descent stops mattering. This probe finds -//! where, so the decision to build a `U64x4`/`U64x2` facade surface across six -//! backend files rests on a measurement rather than on the tail-only number. +//! The answer decides whether a `U64x4`/`U64x2` facade surface across six +//! backend files is worth building. //! -//! # Arms +//! # The arms //! -//! - **P — padded**: exactly what `mask_and` does today. `as_chunks::<8>()` -//! body, then both operands zero-padded into a `U64x8` and the first -//! `tail.len()` lanes copied back out. -//! - **D — descent**: same body, then the remainder walked greedily 4 -> 2 -> 1 -//! with `_mm256_and_si256` / `_mm_and_si128` / scalar `&`. Every lane live, -//! no zero-init, no `copy_from_slice` of a padded array. +//! - **P — production.** `ndarray::simd::mask_and`, called directly. Not a +//! re-implementation of it: the real function, so whatever `U64x8` resolves +//! to on this build (one `_mm512_and_si512` under AVX-512, two 256-bit ops +//! under v3) is what gets measured. +//! - **D — descent.** The SAME facade `U64x8` body, then the remainder walked +//! greedily 4 -> 2 -> 1 with `_mm256_and_si256` / `_mm_and_si128` / scalar. +//! Every lane live, no zero-init, no `copy_from_slice` of a padded array. //! -//! Both arms produce identical output; the probe asserts that before timing, so -//! a faster-but-wrong descent cannot be reported as a win. +//! Both are gated bit-identical against each other and against a scalar +//! reference before any timing, so a faster-but-wrong descent cannot be +//! reported as a win. //! -//! # The confound this probe had, and why the arms now share a body +//! # Why the descent needs no AVX-512VL //! -//! The first version gave the two arms DIFFERENT bodies — `as_chunks::<8>()` -//! with a nested 2x256-bit loop for P, a flat `step_by(4)` loop for D — and -//! then reported the difference as a tail result. It is not: at `n = 256`, -//! `1024` and `4096`, where `n % 8 == 0` and there is NO TAIL AT ALL, it still -//! showed `D/P` of 0.844, 0.763 and 0.716. A tail strategy cannot move a call -//! that has no tail, so those rows were measuring body shape. +//! `_mm256_and_si256` is AVX2 and `_mm_and_si128` is SSE2. Only +//! `_mm256_ternarylogic_epi64` is a VL instruction. So the bitwise descent runs +//! on the portable v3 baseline as well as on v4, and the result applies to 8 of +//! the 11 mask-algebra tail sites; only `mask_ternlog` needs the VL gate. //! -//! Both arms now call ONE shared `body()` and differ only after it. Any -//! remaining `D/P != 1` at `n % 8 == 0` would be noise or layout, never the -//! variable under test — which makes those rows the probe's own control. +//! # Three measurement errors this probe made, and what fixed them //! -//! # Why this needs no AVX-512 +//! Recorded because each produced a plausible number that was not about the +//! variable under test. The first two were caught here, the last three in +//! review on #315. //! -//! `_mm256_and_si256` is AVX2 and `_mm_and_si128` is SSE2. The bitwise descent -//! for and/or/xor/andnot needs **no AVX-512VL** at all — only the `ternlog` -//! descent does, because `_mm256_ternarylogic_epi64` is a VL instruction. So -//! this arm runs on the portable v3 baseline as well as on v4, and the result -//! applies to 8 of the 11 mask-algebra tail sites. +//! 1. **Confounded bodies.** The arms originally had DIFFERENT bodies +//! (`as_chunks::<8>()` + a nested loop vs a flat `step_by(4)` loop) and the +//! difference was reported as a tail result. It is not: at `n % 8 == 0`, +//! with NO TAIL AT ALL, it still showed `D/P` of 0.84 / 0.76 / 0.72. A tail +//! strategy cannot move a call that has no tail. +//! 2. **Cache order** was then tested as the explanation and RULED OUT — both +//! orders agreed. So a residual remained that was neither tail nor order. +//! 3. **The padded arm was a STRAWMAN on v4** (codex, #315). It hand-rolled two +//! 256-bit ops while production `mask_and` issues one `_mm512_and_si512`, so +//! the AVX-512 numbers compared the descent against something slower than +//! the real thing. Fixed by calling the real `mask_and`, which also retires +//! error 1 permanently: there is no hand-written body left to diverge. +//! 4. **Asymmetric noise handling** (codex, #315). A negative tail delta was +//! reported as noise, while an equally noise-sized POSITIVE delta was +//! accepted and divided into a ratio — which is where the old table's 23x, +//! 37x and 56x came from. Now a noise floor is MEASURED and applied to both +//! signs alike. +//! 5. **The isolating section dropped the order control** (coderabbit, #315). +//! The first table alternated orders; the difference-in-differences loop, +//! which produces the headline, did not. Now every sample alternates. //! //! Run: `cargo run --release --example mask_algebra_tail_probe` fn main() { #[cfg(not(target_arch = "x86_64"))] { - println!("mask_algebra_tail_probe: x86_64 only (uses AVX2/SSE2 intrinsics); skipping"); + println!("mask_algebra_tail_probe: x86_64 only (the descent arm uses AVX2/SSE2); skipping"); } #[cfg(target_arch = "x86_64")] { @@ -60,204 +73,221 @@ fn main() { println!("mask_algebra_tail_probe: needs AVX2 at runtime; skipping"); return; } - // SAFETY: avx2 confirmed present by the runtime check above; every - // callee below is `#[target_feature(enable = "avx2")]` and every access - // it makes is bounded by the slice lengths asserted equal in `run`. - unsafe { imp::run() } + imp::run(); } } #[cfg(target_arch = "x86_64")] mod imp { + use ndarray::simd::{mask_and, U64x8}; use std::arch::x86_64::*; use std::hint::black_box; use std::time::Instant; - /// The SHARED body, identical in both arms: full 4-word registers over the - /// first `n & !7` words. Factored out deliberately — see the module note on - /// the confound this removes. - #[target_feature(enable = "avx2")] - unsafe fn body(a: &[u64], b: &[u64], dst: &mut [u64], upto: usize) { - for i in (0..upto).step_by(4) { - // SAFETY: i + 4 <= upto <= n = a.len() = b.len() = dst.len(). - let va = _mm256_loadu_si256(a.as_ptr().add(i).cast()); - let vb = _mm256_loadu_si256(b.as_ptr().add(i).cast()); - _mm256_storeu_si256(dst.as_mut_ptr().add(i).cast(), _mm256_and_si256(va, vb)); - } + /// P — the PRODUCTION path, called not re-implemented. + fn and_production(a: &[u64], b: &[u64], dst: &mut [u64]) { + mask_and(a, b, dst); } - /// P — shared body, then the CURRENT tail: both operands zero-padded into a - /// `[u64; 8]`, one full-width op, first `t` lanes copied back. - #[target_feature(enable = "avx2")] - unsafe fn and_padded(a: &[u64], b: &[u64], dst: &mut [u64]) { + /// D — the same facade `U64x8` body as production, then a 4 -> 2 -> 1 + /// descent over the remainder instead of a zero-padded full-width op. + /// + /// The body is written through the facade (`U64x8`) so it compiles to + /// whatever backend this build selected, exactly as `mask_and`'s body does. + /// Only the tail differs. + fn and_descent(a: &[u64], b: &[u64], dst: &mut [u64]) { let n = a.len(); let done = n & !7; - body(a, b, dst, done); - let t = n - done; - if t != 0 { - let mut pa = [0u64; 8]; - let mut pb = [0u64; 8]; - pa[..t].copy_from_slice(&a[done..]); - pb[..t].copy_from_slice(&b[done..]); - let mut out = [0u64; 8]; - for k in 0..2 { - // SAFETY: pa/pb/out are [u64; 8]; k in 0..2 so the 256-bit - // access at u64 index 0 or 4 stays inside 64 bytes. - let va = _mm256_loadu_si256(pa.as_ptr().add(k * 4).cast()); - let vb = _mm256_loadu_si256(pb.as_ptr().add(k * 4).cast()); - _mm256_storeu_si256(out.as_mut_ptr().add(k * 4).cast(), _mm256_and_si256(va, vb)); + for i in (0..done).step_by(8) { + let va = U64x8::from_slice(&a[i..i + 8]); + let vb = U64x8::from_slice(&b[i..i + 8]); + (va & vb).copy_to_slice(&mut dst[i..i + 8]); + } + // SAFETY: avx2 was confirmed by the runtime check in `main`, so the + // 256-bit and 128-bit intrinsics below are legal on this CPU. Every + // access is bounded by the loop conditions against `n`, which equals + // `a.len() == b.len() == dst.len()` (asserted by the caller). + unsafe { + let mut i = done; + while n - i >= 4 { + let va = _mm256_loadu_si256(a.as_ptr().add(i).cast()); + let vb = _mm256_loadu_si256(b.as_ptr().add(i).cast()); + _mm256_storeu_si256(dst.as_mut_ptr().add(i).cast(), _mm256_and_si256(va, vb)); + i += 4; + } + while n - i >= 2 { + let va = _mm_loadu_si128(a.as_ptr().add(i).cast()); + let vb = _mm_loadu_si128(b.as_ptr().add(i).cast()); + _mm_storeu_si128(dst.as_mut_ptr().add(i).cast(), _mm_and_si128(va, vb)); + i += 2; + } + while i < n { + dst[i] = a[i] & b[i]; + i += 1; } - dst[done..].copy_from_slice(&out[..t]); } } - /// D — the SAME shared body, then the remainder walked greedily 4 -> 2 -> 1. - /// Every lane live, no zero-init, no padded copy. - #[target_feature(enable = "avx2")] - unsafe fn and_descent(a: &[u64], b: &[u64], dst: &mut [u64]) { - let n = a.len(); - let done = n & !7; - body(a, b, dst, done); - let mut i = done; - while n - i >= 4 { - // SAFETY: loop condition guarantees i + 4 <= n. - let va = _mm256_loadu_si256(a.as_ptr().add(i).cast()); - let vb = _mm256_loadu_si256(b.as_ptr().add(i).cast()); - _mm256_storeu_si256(dst.as_mut_ptr().add(i).cast(), _mm256_and_si256(va, vb)); - i += 4; + type Arm = fn(&[u64], &[u64], &mut [u64]); + + fn time_once(f: Arm, a: &[u64], b: &[u64], dst: &mut [u64], iters: u32) -> f64 { + for _ in 0..(iters / 8).max(1) { + f(black_box(a), black_box(b), black_box(dst)); } - while n - i >= 2 { - // SAFETY: loop condition guarantees i + 2 <= n. SSE2, 128-bit. - let va = _mm_loadu_si128(a.as_ptr().add(i).cast()); - let vb = _mm_loadu_si128(b.as_ptr().add(i).cast()); - _mm_storeu_si128(dst.as_mut_ptr().add(i).cast(), _mm_and_si128(va, vb)); - i += 2; + let t = Instant::now(); + for _ in 0..iters { + f(black_box(a), black_box(b), black_box(dst)); } - while i < n { - dst[i] = a[i] & b[i]; - i += 1; + let e = t.elapsed(); + black_box(&dst[0]); + e.as_secs_f64() * 1e9 / f64::from(iters) + } + + fn median(v: &mut [f64]) -> f64 { + v.sort_by(|x, y| x.partial_cmp(y).expect("timings are finite")); + let m = v.len() / 2; + if v.len() % 2 == 1 { + v[m] + } else { + (v[m - 1] + v[m]) / 2.0 } } - fn bench(f: unsafe fn(&[u64], &[u64], &mut [u64]), a: &[u64], b: &[u64], dst: &mut [u64], iters: u32) -> f64 { - // SAFETY: caller only passes the two avx2 fns above, under the runtime - // avx2 check in `main`; lengths are equal (asserted in `run`). - unsafe { - for _ in 0..(iters / 8).max(1) { - f(black_box(a), black_box(b), black_box(dst)); - } - let t = Instant::now(); - for _ in 0..iters { - f(black_box(a), black_box(b), black_box(dst)); + /// Time both arms on the same input, ALTERNATING which runs first on every + /// sample, and return each arm's median. + /// + /// Alternation is the control for two different biases at once: the arm + /// timed second inherits a warm cache, and a fixed order lets clock drift + /// accumulate into one arm. Both were real risks here — the first table's + /// two-order columns exist because of them — and the isolating section + /// below originally lacked this control entirely (coderabbit, #315). + fn time_pair(a: &[u64], b: &[u64], iters: u32, samples: usize) -> (f64, f64) { + let n = a.len(); + let mut dp = vec![0u64; n]; + let mut dd = vec![0u64; n]; + let mut ps = Vec::with_capacity(samples); + let mut ds = Vec::with_capacity(samples); + for s in 0..samples { + if s % 2 == 0 { + ps.push(time_once(and_production, a, b, &mut dp, iters)); + ds.push(time_once(and_descent, a, b, &mut dd, iters)); + } else { + ds.push(time_once(and_descent, a, b, &mut dd, iters)); + ps.push(time_once(and_production, a, b, &mut dp, iters)); } - let e = t.elapsed(); - black_box(&dst[0]); - e.as_secs_f64() * 1e9 / f64::from(iters) } + (median(&mut ps), median(&mut ds)) } - pub unsafe fn run() { - println!("mask_algebra_tail_probe: mask_and, padded tail vs 4->2->1 descent"); - println!("arch=x86_64 avx2=true (AVX2/SSE2 only — no AVX-512, no VL)\n"); - println!( - "{:>7} {:>6} {:>9} {:>9} {:>6} {:>6} verdict", - "n", "tail", "P padded", "D descent", "D/P", "D/P'" - ); - - for &n in &[1usize, 3, 7, 8, 9, 15, 16, 31, 64, 256, 1024, 4096] { - let a: Vec = (0..n) - .map(|i| 0xF0F0_5555_AAAA_1111u64 ^ (i as u64).wrapping_mul(0x9E37_79B9)) - .collect(); - let b: Vec = (0..n) - .map(|i| 0x0FF0_1234_5678_9ABCu64 ^ (i as u64).wrapping_mul(0x85EB_CA6B)) - .collect(); - let mut dp = vec![0u64; n]; - let mut dd = vec![0u64; n]; - assert_eq!(a.len(), b.len()); - - // Gate on equality BEFORE timing: a faster wrong arm is not a win. - and_padded(&a, &b, &mut dp); - and_descent(&a, &b, &mut dd); - assert_eq!(dp, dd, "n={n}: descent must be bit-identical to padded"); - let expect: Vec = a.iter().zip(&b).map(|(x, y)| x & y).collect(); - assert_eq!(dp, expect, "n={n}: padded arm must equal the scalar reference"); + fn mk(n: usize, seed: u64) -> Vec { + (0..n) + .map(|i| seed ^ (i as u64).wrapping_mul(0x9E37_79B9)) + .collect() + } - let iters = if n <= 64 { 2_000_000 } else { 200_000 }; + fn gate(n: usize) { + let a = mk(n, 0xF0F0_5555_AAAA_1111); + let b = mk(n, 0x0FF0_1234_5678_9ABC); + let mut dp = vec![0u64; n]; + let mut dd = vec![0u64; n]; + and_production(&a, &b, &mut dp); + and_descent(&a, &b, &mut dd); + let expect: Vec = a.iter().zip(&b).map(|(x, y)| x & y).collect(); + assert_eq!(dp, expect, "n={n}: production mask_and must equal the scalar reference"); + assert_eq!(dd, expect, "n={n}: descent must equal the scalar reference"); + } - // BOTH ORDERS. The arm timed second inherits a warm cache from the - // first, and at n >= 256 the three buffers exceed L1 — so a - // single-order probe systematically favours whichever arm runs - // last. Reporting both makes that bias visible instead of letting - // it masquerade as the effect under test. - let p1 = bench(and_padded, &a, &b, &mut dp, iters); - let d1 = bench(and_descent, &a, &b, &mut dd, iters); - let d2 = bench(and_descent, &a, &b, &mut dd, iters); - let p2 = bench(and_padded, &a, &b, &mut dp, iters); + pub fn run() { + let avx512 = std::arch::is_x86_feature_detected!("avx512f"); + println!("mask_algebra_tail_probe: production `mask_and` vs a 4->2->1 tail descent"); + println!("arch=x86_64 avx2=true avx512f={avx512}"); + println!("P is the REAL `ndarray::simd::mask_and`; D shares its facade U64x8 body.\n"); - let r1 = d1 / p1; - let r2 = d2 / p2; - let spread = (r1 - r2).abs(); - let verdict = if spread > 0.15 { - "ORDER-DEPENDENT — not a result" - } else if r1.max(r2) < 0.90 { - "descent WINS" - } else if r1.min(r2) > 1.10 { - "descent LOSES" - } else { - "inert" - }; - println!("{n:>7} {:>6} {p1:>9.2} {d1:>9.2} {r1:>6.3} {r2:>6.3} {verdict}", n % 8); + for n in 1..=16usize { + gate(n); } - // ── The isolating measurement: difference-in-differences ────────── + for &n in &[31usize, 64, 65, 256, 1024] { + gate(n); + } + + // ── Step 1: measure the NOISE FLOOR ─────────────────────────────── + // + // Both the baseline and the tail points are timings, so their + // difference carries both their noise. Repeating the SAME measurement + // and taking the spread of the differences gives the magnitude below + // which a tail delta means nothing — in EITHER direction. // - // The table above cannot be read as a tail result on its own. At - // `n % 8 == 0` both arms execute the SAME code (shared body, then a - // `t == 0` branch neither takes), yet they do not time the same — and - // the both-orders columns rule out cache order as the cause. Something - // about how the two functions compile differs, so every ratio above - // carries that unknown alongside the tail. + // The old version rejected only negative deltas and divided by the + // positive ones however small, which manufactured ratios of 23x, 37x + // and 56x out of noise (codex, #315). The floor is applied to |delta|. + let base_probe = 64usize; + let a0 = mk(base_probe, 0xF0F0_5555_AAAA_1111); + let b0 = mk(base_probe, 0x0FF0_1234_5678_9ABC); + let iters = 2_000_000u32; + let mut deltas = Vec::new(); + for _ in 0..7 { + let (p1, d1) = time_pair(&a0, &b0, iters, 2); + let (p2, d2) = time_pair(&a0, &b0, iters, 2); + deltas.push((p1 - p2).abs()); + deltas.push((d1 - d2).abs()); + } + let floor = deltas.iter().cloned().fold(0.0f64, f64::max); + println!("noise floor (max |repeat - repeat| over 14 same-input pairs): {floor:.2} ns"); + println!("A tail delta whose magnitude is below this is reported as `~noise`, either sign.\n"); + + // ── Step 2: the isolating measurement ───────────────────────────── // - // Differencing removes it. For a fixed body size `base` (a multiple of - // 8), `t(base + k) - t(base)` is the cost of a k-word tail and nothing - // else: the body work is identical in both terms and cancels, per arm, - // whatever each arm's body compiled to. - println!("\n── tail cost ISOLATED: t(base + k) - t(base), body held constant ──"); - println!("{:>6} {:>3} {:>10} {:>10} {:>7}", "base", "k", "P tail ns", "D tail ns", "P/D"); + // `t(base + k) - t(base)` at a fixed body size is the cost of a k-word + // tail and nothing else: the body work is identical in both terms and + // cancels per arm, whatever that arm's body compiled to. + println!("── tail cost ISOLATED: t(base + k) - t(base), body held constant ──"); + println!("{:>6} {:>3} {:>11} {:>11} {:>8}", "base", "k", "P tail ns", "D tail ns", "P/D"); + let mut wins = 0usize; + let mut rows = 0usize; for &base in &[8usize, 64] { - let mk = |n: usize, seed: u64| -> Vec { - (0..n) - .map(|i| seed ^ (i as u64).wrapping_mul(0x9E37_79B9)) - .collect() - }; - let a0 = mk(base, 0xF0F0_5555_AAAA_1111); - let b0 = mk(base, 0x0FF0_1234_5678_9ABC); - let mut d0 = vec![0u64; base]; - let it = 2_000_000; - let p0 = bench(and_padded, &a0, &b0, &mut d0, it); - let q0 = bench(and_descent, &a0, &b0, &mut d0, it); + let ab = mk(base, 0xF0F0_5555_AAAA_1111); + let bb = mk(base, 0x0FF0_1234_5678_9ABC); + let (p0, d0) = time_pair(&ab, &bb, iters, 5); for k in 1..8usize { let n = base + k; let a = mk(n, 0xF0F0_5555_AAAA_1111); let b = mk(n, 0x0FF0_1234_5678_9ABC); - let mut d = vec![0u64; n]; - let pk = bench(and_padded, &a, &b, &mut d, it); - let qk = bench(and_descent, &a, &b, &mut d, it); + let (pk, dk) = time_pair(&a, &b, iters, 5); let pt = pk - p0; - let dt = qk - q0; - let ratio = if dt > 0.0 { pt / dt } else { f64::NAN }; - println!("{base:>6} {k:>3} {pt:>10.2} {dt:>10.2} {ratio:>7.2}"); + let dt = dk - d0; + rows += 1; + let ps = if pt.abs() < floor { + format!("{pt:>8.2} ~n") + } else { + format!("{pt:>11.2}") + }; + let ds = if dt.abs() < floor { + format!("{dt:>8.2} ~n") + } else { + format!("{dt:>11.2}") + }; + // A ratio is only meaningful when BOTH terms clear the floor. + // Below it, the honest answer is that the tail is too cheap to + // measure — which is itself the finding, not a missing number. + let rs = if pt.abs() < floor { + " n/a".to_string() + } else if dt.abs() < floor { + wins += 1; + " D~noise".to_string() + } else if dt > 0.0 { + if pt / dt > 2.0 { + wins += 1; + } + format!("{:>8.2}", pt / dt) + } else { + " n/a".to_string() + }; + println!("{base:>6} {k:>3} {ps} {ds} {rs}"); } } - println!("P/D > 1 means the padded tail costs that many times the descent's."); - println!("A NEGATIVE D-tail is not an error: it means the descent's tail is BELOW the"); - println!("run-to-run noise of the body it was differenced against — indistinguishable"); - println!("from having no tail at all. The ratio is NaN there because the denominator is"); - println!("noise, and reporting a number would invent precision the measurement lacks."); - - println!("\nns per call (P and D columns are the P-first pass)."); - println!("D/P = descent/padded with PADDED timed first."); - println!("D/P' = the same ratio with DESCENT timed first."); - println!("The two must agree; if they do not, the number is cache order, not the tail."); - println!("tail = n % 8, the remainder the two arms handle differently."); + println!("\n`~n` = magnitude below the noise floor, i.e. indistinguishable from no tail."); + println!("`D~noise` = the padded tail is measurable and the descent's is not."); + println!("A ratio is printed only when BOTH terms clear the floor."); + println!("\ndescent clearly cheaper on {wins} of {rows} tail widths."); } } From 34bf8d03c86344e39984a54d8f581f4b4d0d4f92 Mon Sep 17 00:00:00 2001 From: Claude Date: Wed, 16 Sep 2026 22:24:18 +0000 Subject: [PATCH 03/13] probe: declare mask_algebra_tail_probe as std-gated The `--no-default-features` CI row builds every example, and the probe's rewrite to measure against the REAL `ndarray::simd::mask_and` made it depend on a module that is `#[cfg(feature = "std")]`. Every sibling probe already carries a `[[example]] required-features = ["std"]` block for exactly this reason; this one was added without one. Co-Authored-By: Claude Opus 5 Claude-Session: https://claude.ai/code/session_01DCfrD5y19cvFc4AoyydXYv --- Cargo.toml | 4 ++++ 1 file changed, 4 insertions(+) diff --git a/Cargo.toml b/Cargo.toml index c525bab3..456647c0 100644 --- a/Cargo.toml +++ b/Cargo.toml @@ -67,6 +67,10 @@ required-features = ["std"] name = "ternlogq_tail_descent_probe" required-features = ["std"] +[[example]] +name = "mask_algebra_tail_probe" +required-features = ["std"] + [[example]] name = "ternlogq_sparse_reapply_probe" required-features = ["std"] From 2936d2000a53da86b1c95a6c7c72f8e88bce6ee6 Mon Sep 17 00:00:00 2001 From: Claude Date: Wed, 16 Sep 2026 22:25:51 +0000 Subject: [PATCH 04/13] probe: document the four timing/gate helpers Each doc says what the helper GUARDS, not what it does: why the median rather than the mean (one preemption moves a mean by more than the effect), why `black_box` wraps the output as well as the inputs, why the operand generator must not produce an all-zero tail group, and why the bit-identity gate runs before any width's timing is believed. Co-Authored-By: Claude Opus 5 Claude-Session: https://claude.ai/code/session_01DCfrD5y19cvFc4AoyydXYv --- examples/mask_algebra_tail_probe.rs | 18 ++++++++++++++++++ 1 file changed, 18 insertions(+) diff --git a/examples/mask_algebra_tail_probe.rs b/examples/mask_algebra_tail_probe.rs index 3f54165c..2a8e778b 100644 --- a/examples/mask_algebra_tail_probe.rs +++ b/examples/mask_algebra_tail_probe.rs @@ -130,6 +130,11 @@ mod imp { type Arm = fn(&[u64], &[u64], &mut [u64]); + /// Median-free single timing of one arm: warm up over an eighth of the + /// budget, then time `iters` back-to-back calls and return ns/call. + /// + /// `black_box` wraps BOTH the inputs and one output word, so the optimizer + /// can neither hoist the call out of the loop nor delete the write. fn time_once(f: Arm, a: &[u64], b: &[u64], dst: &mut [u64], iters: u32) -> f64 { for _ in 0..(iters / 8).max(1) { f(black_box(a), black_box(b), black_box(dst)); @@ -143,6 +148,10 @@ mod imp { e.as_secs_f64() * 1e9 / f64::from(iters) } + /// Median of a timing sample, the robust centre for this probe. + /// + /// Deliberately not the mean: a scheduler preemption lands as one huge + /// outlier, which moves a mean by more than the effect being measured. fn median(v: &mut [f64]) -> f64 { v.sort_by(|x, y| x.partial_cmp(y).expect("timings are finite")); let m = v.len() / 2; @@ -179,12 +188,21 @@ mod imp { (median(&mut ps), median(&mut ds)) } + /// Deterministic pseudo-random operand of `n` words. + /// + /// The odd Weyl multiplier gives every word a different bit pattern, so a + /// tail group cannot accidentally be all-zero and time as free. fn mk(n: usize, seed: u64) -> Vec { (0..n) .map(|i| seed ^ (i as u64).wrapping_mul(0x9E37_79B9)) .collect() } + /// Correctness gate: both arms must be BIT-IDENTICAL to a scalar `&` + /// reference at this width before any timing of that width is believed. + /// + /// A descent that silently skipped its tail would be faster and wrong; the + /// gate is what stops the speed number from being reported anyway. fn gate(n: usize) { let a = mk(n, 0xF0F0_5555_AAAA_1111); let b = mk(n, 0x0FF0_1234_5678_9ABC); From 6aaadc3d77c22f9f34a779acbe007ff24c8495a7 Mon Sep 17 00:00:00 2001 From: Claude Date: Wed, 16 Sep 2026 22:54:13 +0000 Subject: [PATCH 05/13] probe: the tail needs a fixed TRIP COUNT, not intrinsics MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit The previous conclusion — build a `U64x4`/`U64x2` facade surface across six backend files plus an `avx512vl` gate — rested on comparing an intrinsic descent against a ZERO-PADDED full-width op. That comparison cannot decide what to build, because it never tested the cheap alternative. Two arms added, so it now does: S the same facade body, tail as `for i in done..n` F the same facade body, tail as FIXED-WIDTH steps in plain Rust Measured on a resolvable v4 run (noise floor 1.32 ns, 34 of 98 widths resolvable), as a share of the padded tail's cost removed: S 79.6% D 82.5% F 83.2% D - S = +2.9 pts intrinsics vs a variable-length loop D - F = -0.7 pts intrinsics vs portable fixed steps So the descent's advantage is over the TRIP COUNT, not the instruction set. `done..n` has a trip count unknown at compile time and LLVM emits a scalar loop with a branch per word; `for j in 0..4` compiles to a single `vandps ymm`. The new `narrow_bitop_codegen_probe` shows that directly, and shows something stronger: the facade's own `U64x4 &` emits assembly the assembler ALIASES to the hand-written loop (`probe_u64x4_and = probe_scalar4_and`), so the "scalar polyfill" was never costing anything. Consequence: the narrow facade type and the VL gate are not justified. The tail should be fixed-width steps in plain Rust — no backend edits, no raw intrinsics (so no exception to the all-SIMD-from-the-facade invariant), and it reaches the NEON, wasm and scalar tails an x86 descent never could. Four measurement corrections, each from a number this probe itself produced: - The count of widths where D beats S is thresholded by the noise floor and the floor is a draw: consecutive runs gave 2 of 14 and 10 of 14 from floors of 1.35 ns and 0.64 ns. Replaced by pooled fractions. - A per-pass median over 14 widths is itself noise; it swung D - S across [-27, +34] points. Pooled across all passes instead, with the per-pass spread printed beside it. - A tail that timed NEGATIVE inflated `(pt - dt)/pt` above 1 and pooled out as "D removes 121.3% of the padded cost". Widths now qualify only when all four tail costs clear the floor. This is the asymmetric-noise error codex caught in the ratio column, reintroduced one statistic later. - The header printed `is_x86_feature_detected!`, the HOST cpu, so a v3 build on this machine announced `avx512f=true`. It now prints the BUILD tier from `cfg!(target_feature)` alongside it. When the tails do not clear the floor the probe reports INCONCLUSIVE rather than a verdict — and says why that still favours the cheap option: D, S and F land within a point or two of each other, so the burden of proof is on the expensive one and it has not been met. Co-Authored-By: Claude Opus 5 Claude-Session: https://claude.ai/code/session_01DCfrD5y19cvFc4AoyydXYv --- Cargo.toml | 4 + examples/mask_algebra_tail_probe.rs | 416 +++++++++++++++++++++---- examples/narrow_bitop_codegen_probe.rs | 75 +++++ 3 files changed, 434 insertions(+), 61 deletions(-) create mode 100644 examples/narrow_bitop_codegen_probe.rs diff --git a/Cargo.toml b/Cargo.toml index 456647c0..6881d86d 100644 --- a/Cargo.toml +++ b/Cargo.toml @@ -71,6 +71,10 @@ required-features = ["std"] name = "mask_algebra_tail_probe" required-features = ["std"] +[[example]] +name = "narrow_bitop_codegen_probe" +required-features = ["std"] + [[example]] name = "ternlogq_sparse_reapply_probe" required-features = ["std"] diff --git a/examples/mask_algebra_tail_probe.rs b/examples/mask_algebra_tail_probe.rs index 2a8e778b..445da7c9 100644 --- a/examples/mask_algebra_tail_probe.rs +++ b/examples/mask_algebra_tail_probe.rs @@ -128,6 +128,74 @@ mod imp { } } + /// S — the same facade body, then the tail as a PLAIN SCALAR LOOP. + /// + /// The arm that could make the whole descent unnecessary. `D` beats `P` + /// because padding to full width does real work on words nobody wants, + /// not because intrinsics are magic — and a bounded scalar tail loop is + /// not obviously slow: measured on this tree (`narrow_bitop_codegen_probe`), + /// a fixed `for i in 0..4` over `&[u64; 4]` emits exactly + /// `vmovups/vandps ymm/vmovups`, and the 2-lane form the `xmm` equivalent. + /// LLVM already descends a trivially-bounded loop. + /// + /// If S ties D, the facade needs NO new narrow type, no `avx512vl` gate + /// and no six-backend edit: the tail is a `while i < n` and portability + /// comes free. That is the cheapest possible shape, so it has to be + /// falsified before the expensive one is built. + fn and_scalar_tail(a: &[u64], b: &[u64], dst: &mut [u64]) { + let n = a.len(); + let done = n & !7; + for i in (0..done).step_by(8) { + let va = U64x8::from_slice(&a[i..i + 8]); + let vb = U64x8::from_slice(&b[i..i + 8]); + (va & vb).copy_to_slice(&mut dst[i..i + 8]); + } + for i in done..n { + dst[i] = a[i] & b[i]; + } + } + + /// F — the synthesis, and the arm that should make this whole question + /// cheap: the same 4 -> 2 -> 1 descent as `D`, written in PLAIN RUST with + /// FIXED-SIZE steps and not one intrinsic. + /// + /// The mechanism `S` was missing. `S`'s tail is `for i in done..n`, a + /// loop whose trip count is unknown at compile time, so LLVM emits a + /// scalar loop with a branch per word. `F`'s steps are `[u64; 4]` and + /// `[u64; 2]` — trip counts fixed at compile time, which + /// `narrow_bitop_codegen_probe` measured compiling to a single + /// `vandps ymm` and `vandps xmm` respectively. + /// + /// If F ties D, the descent needs no narrow facade type, no `avx512vl` + /// gate and no backend edits: it is a portable helper that every backend + /// already compiles correctly, and it reaches the tail sites on NEON and + /// wasm too, which an x86 intrinsic descent never could. + fn and_fixed_descent(a: &[u64], b: &[u64], dst: &mut [u64]) { + let n = a.len(); + let done = n & !7; + for i in (0..done).step_by(8) { + let va = U64x8::from_slice(&a[i..i + 8]); + let vb = U64x8::from_slice(&b[i..i + 8]); + (va & vb).copy_to_slice(&mut dst[i..i + 8]); + } + let mut i = done; + if n - i >= 4 { + for j in 0..4 { + dst[i + j] = a[i + j] & b[i + j]; + } + i += 4; + } + if n - i >= 2 { + for j in 0..2 { + dst[i + j] = a[i + j] & b[i + j]; + } + i += 2; + } + if i < n { + dst[i] = a[i] & b[i]; + } + } + type Arm = fn(&[u64], &[u64], &mut [u64]); /// Median-free single timing of one arm: warm up over an eighth of the @@ -153,6 +221,14 @@ mod imp { /// Deliberately not the mean: a scheduler preemption lands as one huge /// outlier, which moves a mean by more than the effect being measured. fn median(v: &mut [f64]) -> f64 { + // NaN, not a panic: an empty sample is a real outcome here, not a bug. + // The qualification filter can reject every width in a pass when the + // machine is noisy, and "nothing was resolvable" is the finding — a + // probe that crashes instead of reporting it would look like broken + // code rather than an unresolvable measurement. + if v.is_empty() { + return f64::NAN; + } v.sort_by(|x, y| x.partial_cmp(y).expect("timings are finite")); let m = v.len() / 2; if v.len() % 2 == 1 { @@ -162,30 +238,38 @@ mod imp { } } - /// Time both arms on the same input, ALTERNATING which runs first on every - /// sample, and return each arm's median. + /// Time all three arms on the same input, ROTATING which runs first on + /// every sample, and return each arm's median. /// /// Alternation is the control for two different biases at once: the arm /// timed second inherits a warm cache, and a fixed order lets clock drift /// accumulate into one arm. Both were real risks here — the first table's /// two-order columns exist because of them — and the isolating section - /// below originally lacked this control entirely (coderabbit, #315). - fn time_pair(a: &[u64], b: &[u64], iters: u32, samples: usize) -> (f64, f64) { + /// below originally lacked this control entirely (coderabbit, #315). With + /// four arms the alternation became a rotation: a two-way swap would + /// have pinned the other two arms to fixed positions. + fn time_arms(a: &[u64], b: &[u64], iters: u32, samples: usize) -> [f64; 4] { let n = a.len(); - let mut dp = vec![0u64; n]; - let mut dd = vec![0u64; n]; - let mut ps = Vec::with_capacity(samples); - let mut ds = Vec::with_capacity(samples); + let mut d: [Vec; 4] = [vec![0; n], vec![0; n], vec![0; n], vec![0; n]]; + let arms: [Arm; 4] = [and_production, and_descent, and_scalar_tail, and_fixed_descent]; + let mut acc: [Vec; 4] = [ + Vec::with_capacity(samples), + Vec::with_capacity(samples), + Vec::with_capacity(samples), + Vec::with_capacity(samples), + ]; for s in 0..samples { - if s % 2 == 0 { - ps.push(time_once(and_production, a, b, &mut dp, iters)); - ds.push(time_once(and_descent, a, b, &mut dd, iters)); - } else { - ds.push(time_once(and_descent, a, b, &mut dd, iters)); - ps.push(time_once(and_production, a, b, &mut dp, iters)); + // Rotate which arm goes first. With more than two arms a swap is no + // longer enough: a fixed order gives arm 0 a cold cache on every + // sample and the last arm a warm one, forever. + for step in 0..4 { + let k = (s + step) % 4; + let t = time_once(arms[k], a, b, &mut d[k], iters); + acc[k].push(t); } } - (median(&mut ps), median(&mut ds)) + let [mut p, mut dd, mut ss, mut ff] = acc; + [median(&mut p), median(&mut dd), median(&mut ss), median(&mut ff)] } /// Deterministic pseudo-random operand of `n` words. @@ -210,15 +294,29 @@ mod imp { let mut dd = vec![0u64; n]; and_production(&a, &b, &mut dp); and_descent(&a, &b, &mut dd); + let mut ds = vec![0u64; n]; + and_scalar_tail(&a, &b, &mut ds); + let mut df = vec![0u64; n]; + and_fixed_descent(&a, &b, &mut df); let expect: Vec = a.iter().zip(&b).map(|(x, y)| x & y).collect(); assert_eq!(dp, expect, "n={n}: production mask_and must equal the scalar reference"); assert_eq!(dd, expect, "n={n}: descent must equal the scalar reference"); + assert_eq!(ds, expect, "n={n}: scalar-tail arm must equal the scalar reference"); + assert_eq!(df, expect, "n={n}: fixed-step descent must equal the scalar reference"); } pub fn run() { - let avx512 = std::arch::is_x86_feature_detected!("avx512f"); + // Two DIFFERENT facts, and conflating them is the trap this workspace + // keeps paying for. `is_x86_feature_detected!` asks the HOST CPU what + // it can do; `cfg!(target_feature)` reports what this BUILD was + // compiled for. Under `--config .cargo/config-v3.toml` on an AVX-512 + // host the first says true and the second says false — so a probe that + // prints only the runtime bit lets a v3 measurement be read as v4. + let host_avx512 = std::arch::is_x86_feature_detected!("avx512f"); + let built_avx512 = cfg!(target_feature = "avx512f"); println!("mask_algebra_tail_probe: production `mask_and` vs a 4->2->1 tail descent"); - println!("arch=x86_64 avx2=true avx512f={avx512}"); + println!("arch=x86_64 BUILT-FOR avx512f={built_avx512} (host cpu avx512f={host_avx512})"); + println!("the BUILT-FOR bit is the tier these timings belong to, not the host bit."); println!("P is the REAL `ndarray::simd::mask_and`; D shares its facade U64x8 body.\n"); for n in 1..=16usize { @@ -244,13 +342,15 @@ mod imp { let iters = 2_000_000u32; let mut deltas = Vec::new(); for _ in 0..7 { - let (p1, d1) = time_pair(&a0, &b0, iters, 2); - let (p2, d2) = time_pair(&a0, &b0, iters, 2); + let [p1, d1, s1, f1] = time_arms(&a0, &b0, iters, 2); + let [p2, d2, s2, f2] = time_arms(&a0, &b0, iters, 2); deltas.push((p1 - p2).abs()); deltas.push((d1 - d2).abs()); + deltas.push((s1 - s2).abs()); + deltas.push((f1 - f2).abs()); } let floor = deltas.iter().cloned().fold(0.0f64, f64::max); - println!("noise floor (max |repeat - repeat| over 14 same-input pairs): {floor:.2} ns"); + println!("noise floor (max |repeat - repeat| over 28 same-input repeats): {floor:.2} ns"); println!("A tail delta whose magnitude is below this is reported as `~noise`, either sign.\n"); // ── Step 2: the isolating measurement ───────────────────────────── @@ -259,53 +359,247 @@ mod imp { // tail and nothing else: the body work is identical in both terms and // cancels per arm, whatever that arm's body compiled to. println!("── tail cost ISOLATED: t(base + k) - t(base), body held constant ──"); - println!("{:>6} {:>3} {:>11} {:>11} {:>8}", "base", "k", "P tail ns", "D tail ns", "P/D"); + println!( + "{:>6} {:>3} {:>11} {:>11} {:>11} {:>11}", + "base", "k", "P tail ns", "D tail ns", "S tail ns", "F tail ns" + ); let mut wins = 0usize; let mut rows = 0usize; - for &base in &[8usize, 64] { - let ab = mk(base, 0xF0F0_5555_AAAA_1111); - let bb = mk(base, 0x0FF0_1234_5678_9ABC); - let (p0, d0) = time_pair(&ab, &bb, iters, 5); - for k in 1..8usize { - let n = base + k; - let a = mk(n, 0xF0F0_5555_AAAA_1111); - let b = mk(n, 0x0FF0_1234_5678_9ABC); - let (pk, dk) = time_pair(&a, &b, iters, 5); - let pt = pk - p0; - let dt = dk - d0; - rows += 1; - let ps = if pt.abs() < floor { - format!("{pt:>8.2} ~n") - } else { - format!("{pt:>11.2}") - }; - let ds = if dt.abs() < floor { - format!("{dt:>8.2} ~n") - } else { - format!("{dt:>11.2}") - }; - // A ratio is only meaningful when BOTH terms clear the floor. - // Below it, the honest answer is that the tail is too cheap to - // measure — which is itself the finding, not a missing number. - let rs = if pt.abs() < floor { - " n/a".to_string() - } else if dt.abs() < floor { - wins += 1; - " D~noise".to_string() - } else if dt > 0.0 { - if pt / dt > 2.0 { + // REPLICATE the whole sweep. A single pass gave D - F of -2.0, -8.2, + // -2.3, -4.4 and -0.4 points on v4 but -1.6, +3.1 and +5.3 on v3: the + // sign is not stable, so any verdict read off one pass is a draw, not + // a result. Repeating inside the probe is what lets it report the + // SPREAD instead of inviting the reader to trust whichever pass ran. + const REPEATS: usize = 7; + let mut ds_pts = Vec::with_capacity(REPEATS); + let mut df_pts = Vec::with_capacity(REPEATS); + let mut ds_gap = Vec::new(); + // Fraction of the padded tail's cost that each strategy removes. This + // is the statistic the verdict rests on, because the obvious one — a + // COUNT of widths where D beats S — is thresholded by the noise floor + // and the floor is itself a draw: consecutive runs of this probe gave + // 2 of 14 and 10 of 14 from floors of 1.35 ns and 0.64 ns. A count + // that flips with the floor cannot decide a six-file change; the + // magnitudes it was thresholding were stable the whole time. + let mut s_frac = Vec::new(); + let mut d_frac = Vec::new(); + let mut f_frac = Vec::new(); + // How many widths were actually resolvable. The verdict is gated on + // this: a tail costs ~1 ns and the floor on a busy machine is ~2 ns, + // so on some runs NOTHING qualifies and the only honest output is to + // say the question was not answered. + let mut qualified = 0usize; + for rep in 0..REPEATS { + s_frac.clear(); + d_frac.clear(); + f_frac.clear(); + for &base in &[8usize, 64] { + let ab = mk(base, 0xF0F0_5555_AAAA_1111); + let bb = mk(base, 0x0FF0_1234_5678_9ABC); + let [p0, d0, s0, f0] = time_arms(&ab, &bb, iters, 9); + for k in 1..8usize { + let n = base + k; + let a = mk(n, 0xF0F0_5555_AAAA_1111); + let b = mk(n, 0x0FF0_1234_5678_9ABC); + let [pk, dk, sk, fk] = time_arms(&a, &b, iters, 9); + let pt = pk - p0; + let dt = dk - d0; + let st = sk - s0; + let ft = fk - f0; + rows += 1; + ds_gap.push(st - dt); + // A width contributes to the fraction statistics only + // when ALL FOUR of its tail costs are measurable AND + // positive. Without this, a tail that timed NEGATIVE (the + // longer array came out faster — pure noise) gives + // (pt - dt)/pt > 1, and pooling those produced a "D + // removes 121.3% of the padded cost", which is not a + // quantity. Same asymmetric-noise error codex caught in + // the ratio column, reintroduced one statistic later: + // rejecting a negative DENOMINATOR is not enough when a + // negative NUMERATOR inflates instead. + if pt >= floor && dt >= floor && st >= floor && ft >= floor { + qualified += 1; + s_frac.push((pt - st) / pt); + d_frac.push((pt - dt) / pt); + f_frac.push((pt - ft) / pt); + } + let ps = if pt.abs() < floor { + format!("{pt:>8.2} ~n") + } else { + format!("{pt:>11.2}") + }; + let ds = if dt.abs() < floor { + format!("{dt:>8.2} ~n") + } else { + format!("{dt:>11.2}") + }; + let ss = if st.abs() < floor { + format!("{st:>8.2} ~n") + } else { + format!("{st:>11.2}") + }; + let fs = if ft.abs() < floor { + format!("{ft:>8.2} ~n") + } else { + format!("{ft:>11.2}") + }; + // "Cheaper than the padded tail" counts a width when the + // padded cost is measurable and the descent's is either below + // the floor or more than 2x smaller. A ratio is deliberately + // NOT printed: below the floor the denominator is noise, and a + // number there would invent precision the measurement lacks. + if pt.abs() >= floor && (dt.abs() < floor || (dt > 0.0 && pt / dt > 2.0)) { wins += 1; } - format!("{:>8.2}", pt / dt) - } else { - " n/a".to_string() - }; - println!("{base:>6} {k:>3} {ps} {ds} {rs}"); + if rep == 0 { + println!("{base:>6} {k:>3} {ps} {ds} {ss} {fs}"); + } + } + } + let sp = median(&mut s_frac.clone()) * 100.0; + let dp = median(&mut d_frac.clone()) * 100.0; + let fp = median(&mut f_frac.clone()) * 100.0; + if (dp - sp).is_finite() { + ds_pts.push(dp - sp); + } + if (dp - fp).is_finite() { + df_pts.push(dp - fp); + } + if rep == 0 { + println!("\n(table above is pass 1 of {REPEATS}; the verdict uses all {REPEATS})"); } } println!("\n`~n` = magnitude below the noise floor, i.e. indistinguishable from no tail."); - println!("`D~noise` = the padded tail is measurable and the descent's is not."); - println!("A ratio is printed only when BOTH terms clear the floor."); - println!("\ndescent clearly cheaper on {wins} of {rows} tail widths."); + println!("P = production padded tail, D = x86 intrinsic descent,"); + println!("S = variable-length scalar loop, F = portable fixed-step descent."); + println!("\ndescent clearly cheaper than the PADDED tail on {wins} of {rows} tail widths."); + + // ── Step 3: the question that decides what gets BUILT ───────────── + println!("\n── what does the tail actually need: intrinsics, or a fixed trip count? ──"); + let med_gap = median(&mut ds_gap); + println!("median (S - D) within pass 1: {med_gap:.2} ns [noise floor {floor:.2} ns]"); + + let lo = |v: &[f64]| { + if v.is_empty() { + f64::NAN + } else { + v.iter().cloned().fold(f64::INFINITY, f64::min) + } + }; + let hi = |v: &[f64]| { + if v.is_empty() { + f64::NAN + } else { + v.iter().cloned().fold(f64::NEG_INFINITY, f64::max) + } + }; + let s_all = median(&mut s_frac.clone()) * 100.0; + let d_all = median(&mut d_frac.clone()) * 100.0; + let f_all = median(&mut f_frac.clone()) * 100.0; + let ds_med = d_all - s_all; + let df_med = d_all - f_all; + // NaN reaches these lines whenever the filter rejected every width, so + // it prints as `n/a` rather than as a number: a bare `NaN%` reads like + // a broken probe when it actually means "not resolvable on this run". + let pc = |x: f64| { + if x.is_finite() { + format!("{x:.1}%") + } else { + "n/a".to_string() + } + }; + let pt_ = |x: f64| { + if x.is_finite() { + format!("{x:>5.1}") + } else { + " n/a".to_string() + } + }; + println!("\npooled over all {REPEATS} passes -- S {} D {} F {}", pc(s_all), pc(d_all), pc(f_all)); + println!("\nShare of the padded tail's cost removed, differenced per pass."); + println!("Percentage-point gaps, pooled, with the per-pass spread beside them:"); + println!( + " D - S = {} [{}, {}] intrinsic descent vs a variable-length loop", + pt_(ds_med), + pt_(lo(&ds_pts)), + pt_(hi(&ds_pts)) + ); + println!( + " D - F = {} [{}, {}] intrinsic descent vs PORTABLE fixed steps", + pt_(df_med), + pt_(lo(&df_pts)), + pt_(hi(&df_pts)) + ); + + // The decision is not "does D ever win" — the question is whether what + // intrinsics buy over PORTABLE fixed steps is worth a narrow type on + // six backend files plus an `avx512vl` gate. Compare the two gaps: if + // D - F is small beside D - S, the mechanism was the trip count and + // not the instruction set, and the portable form is the one to build. + // A verdict is only available when the baseline comparison itself is + // positive and clear. If the intrinsic descent does not measurably + // beat even a variable-length loop on this machine, nothing here can + // adjudicate the finer D-vs-F question, and saying so is the result. + let resolvable = qualified * 4 >= rows && ds_med.is_finite() && df_med.is_finite(); + println!( + "widths where all four tails were measurable: {qualified} of {rows} ({}resolvable)", + if resolvable { "" } else { "NOT " } + ); + if !resolvable { + println!( + "\nVERDICT: INCONCLUSIVE — and that is the honest result, not a failed run.\n\ + A k-word tail costs on the order of 1 ns; the noise floor here measured\n\ + {floor:.2} ns. Only {qualified} of {rows} widths had all four tail costs resolvable\n\ + above it, so the D-vs-F comparison has no support in this data and any\n\ + number printed for it would be arithmetic on noise.\n\ + Re-run on a quiet machine, or with larger `iters`, before reading a\n\ + verdict into the gaps below.\n\ + What DOES survive, because it is an order of magnitude larger than the\n\ + floor: every tail strategy removes most of the PADDED tail's 8-20 ns,\n\ + which is the finding this probe was built to establish.\n\ + \n\ + And inconclusive is not neutral about what to BUILD. D, S and F came\n\ + out within a point or two of each other, so the burden of proof sits\n\ + on the expensive option and it has not been met: nothing here supports\n\ + a `U64x4`/`U64x2` narrow type across six backend files plus an\n\ + `avx512vl` gate. Take the cheapest form that captures the padded-tail\n\ + win — fixed-width steps in plain Rust — which needs no backend edits\n\ + and no raw intrinsics, and so reaches the NEON, wasm and scalar tails\n\ + that an x86 descent never could." + ); + } else if ds_med <= 2.0 { + println!( + "\nVERDICT: INCONCLUSIVE on this run. The intrinsic descent beat a plain\n\ + variable-length tail loop by only {ds_med:.1} points, so this machine is too\n\ + noisy right now to resolve the smaller D-vs-F gap ({df_med:.1} points).\n\ + Re-run on a quiet machine before reading anything into either.\n\ + What IS solid here and does not depend on that comparison: all three\n\ + tail strategies remove ~80-95% of the PADDED tail's cost, which is the\n\ + finding this probe was built for." + ); + } else if df_med < ds_med / 2.0 { + println!( + "\nVERDICT: the mechanism is the TRIP COUNT, not the instruction set.\n\ + Against a variable-length `for i in done..n` tail the intrinsic descent\n\ + is worth a median {ds_med:.1} points; against fixed-width steps written in\n\ + plain Rust it is worth {df_med:.1}. A `for j in 0..4` has a trip count known at\n\ + compile time and compiles to a single `vandps ymm`\n\ + (`narrow_bitop_codegen_probe`, where the facade's own `U64x4 &`\n\ + emits assembly the assembler ALIASES to the hand-written loop);\n\ + `done..n` does not and cannot.\n\ + So the `U64x4`/`U64x2` facade surface and the `avx512vl` gate are NOT\n\ + justified. Write the tail as fixed-width steps: no backend edits,\n\ + no raw intrinsics — hence no exception to the all-SIMD-from-the-facade\n\ + invariant — and it reaches the NEON, wasm and scalar tails, which an\n\ + x86 intrinsic descent never could." + ); + } else { + println!( + "\nVERDICT: intrinsics buy a median {df_med:.1} points over portable fixed steps,\n\ + a real fraction of the {ds_med:.1} they buy over a variable-length loop.\n\ + The facade narrow-type surface has measured support. Build phase 1." + ); + } } } diff --git a/examples/narrow_bitop_codegen_probe.rs b/examples/narrow_bitop_codegen_probe.rs new file mode 100644 index 00000000..092509d0 --- /dev/null +++ b/examples/narrow_bitop_codegen_probe.rs @@ -0,0 +1,75 @@ +//! Does the facade's NARROW u64 type already descend to one instruction? +//! +//! On the x86 backends `U64x4` is a scalar-storage polyfill — `pub struct +//! U64x4(pub [u64; 4])` (`simd_avx2.rs`'s `avx2_int_type!`), with `BitAnd` +//! implemented as a 4-iteration loop. The mask-algebra tail descent needs a +//! real 256-bit `and`, and the obvious reading of that struct is that the +//! facade cannot supply one without being retyped to `__m256i`. +//! +//! That reading is an ASSUMPTION about codegen, and this probe is here to +//! refuse it. `[u64; 4]` is `repr(align(64))` and the loop is trivially +//! unrollable, so LLVM may already be emitting a single `vpand ymm` — in +//! which case the tail descent needs a NAME on the facade and nothing more, +//! and the multi-backend retype is unnecessary work. +//! +//! Read the emitted assembly, not this comment: +//! +//! ```sh +//! cargo rustc --release --example narrow_bitop_codegen_probe -- --emit asm +//! ``` +//! +//! and grep the four probe symbols for `vpand` / `vandps` / `vpternlog`. + +use ndarray::simd::{U64x4, U64x8}; + +/// 256-bit AND through the facade's narrow type. Probe symbol. +#[inline(never)] +#[unsafe(no_mangle)] +pub extern "C" fn probe_u64x4_and(a: &[u64; 4], b: &[u64; 4], out: &mut [u64; 4]) { + let r = U64x4::from_array(*a) & U64x4::from_array(*b); + *out = r.to_array(); +} + +/// 512-bit AND through the facade's wide type — the known-good reference. +/// If THIS does not emit a single wide op the probe is measuring the build, +/// not the type. +#[inline(never)] +#[unsafe(no_mangle)] +pub extern "C" fn probe_u64x8_and(a: &[u64; 8], b: &[u64; 8], out: &mut [u64; 8]) { + let r = U64x8::from_slice(a) & U64x8::from_slice(b); + r.copy_to_slice(out); +} + +/// The hand-written scalar the polyfill's loop is supposed to beat (or match). +#[inline(never)] +#[unsafe(no_mangle)] +pub extern "C" fn probe_scalar4_and(a: &[u64; 4], b: &[u64; 4], out: &mut [u64; 4]) { + for i in 0..4 { + out[i] = a[i] & b[i]; + } +} + +/// Two-lane scalar AND — the last rung of a descent, and the width at which +/// auto-vectorization is least likely to be worth LLVM's while. +#[inline(never)] +#[unsafe(no_mangle)] +pub extern "C" fn probe_scalar2_and(a: &[u64; 2], b: &[u64; 2], out: &mut [u64; 2]) { + for i in 0..2 { + out[i] = a[i] & b[i]; + } +} + +fn main() { + let a4 = [0xF0F0u64; 4]; + let b4 = [0x00FFu64; 4]; + let mut o4 = [0u64; 4]; + probe_u64x4_and(&a4, &b4, &mut o4); + probe_scalar4_and(&a4, &b4, &mut o4); + let a8 = [0xF0F0u64; 8]; + let b8 = [0x00FFu64; 8]; + let mut o8 = [0u64; 8]; + probe_u64x8_and(&a8, &b8, &mut o8); + let mut o2 = [0u64; 2]; + probe_scalar2_and(&[1, 2], &[3, 3], &mut o2); + println!("probe: {o4:?} {o8:?} {o2:?}"); +} From fc009d1054cab663ec5831a5c4f5160d4ca1bba8 Mon Sep 17 00:00:00 2001 From: Claude Date: Wed, 16 Sep 2026 22:55:52 +0000 Subject: [PATCH 06/13] blackboard (15): the tail wanted a trip count; cancel the U64x4/U64x2 build Records the decision from PR #315 and the four measurement errors behind it, including the one that matters most for future probes: the asymmetric-noise error codex caught in the ratio column was reintroduced one statistic later, because rejecting a negative denominator is not enough when a negative numerator inflates instead. Co-Authored-By: Claude Opus 5 Claude-Session: https://claude.ai/code/session_01DCfrD5y19cvFc4AoyydXYv --- .claude/blackboard.md | 95 +++++++++++++++++++++++++++++++++++++++++++ 1 file changed, 95 insertions(+) diff --git a/.claude/blackboard.md b/.claude/blackboard.md index b0318ed9..4a0d2ee3 100644 --- a/.claude/blackboard.md +++ b/.claude/blackboard.md @@ -1,3 +1,98 @@ +## 2026-09-16 (15) — the mask-algebra tail wanted a fixed TRIP COUNT, not intrinsics; the `U64x4`/`U64x2` + `avx512vl` build is CANCELLED + +PR #315 (branch `claude/mask-algebra-tail-probe`). The planned step (b) — give +`U64x4`/`U64x2` a real surface across six backend files, add a `tail_descend` +helper, gate `mask_ternlog`'s tail behind `avx512vl` — is **not justified**, and +the probe that was written to support it is what killed it. + +### What the earlier measurement actually compared + +`D` (an x86 intrinsic 4→2→1 descent) against `P` (the zero-padded full-width op +production runs today). D won by 5–17×, which is real. It is also **the wrong +comparison for deciding what to build**, because the alternative it did not +test was free. Two arms were added: + +| arm | tail | +|---|---| +| S | `for i in done..n` | +| F | FIXED-WIDTH steps, plain Rust, zero intrinsics | + +Resolvable v4 run (floor 1.32 ns, 34 of 98 widths resolvable), as the share of +the padded tail's cost removed: **S 79.6% · D 82.5% · F 83.2%**, i.e. +`D − S = +2.9 pts` and **`D − F = −0.7 pts`**. + +**The mechanism is the trip count.** `done..n` is unknown at compile time, so +LLVM emits a scalar loop with a branch per word. `for j in 0..4` compiles to a +single `vandps ymm`. Intrinsics contribute nothing on top of that. + +### The codegen half, and it is the stronger half + +`examples/narrow_bitop_codegen_probe.rs`. On the x86 backends `U64x4` is +`pub struct U64x4(pub [u64; 4])` — the `avx2_int_type!` scalar polyfill — and +the obvious reading is that the facade cannot supply a real 256-bit `and` +without being retyped to `__m256i`. That reading is false, and the assembler +says so outright: + +``` +probe_u64x4_and = probe_scalar4_and + vmovups (%rsi), %ymm0 + vandps (%rdi), %ymm0, %ymm0 + vmovups %ymm0, (%rdx) +``` + +The two functions produced **bit-identical machine code and were merged into +one symbol**. So the scalar polyfill never cost anything, and retyping it would +buy nothing. A struct's storage type is not its codegen. + +### The decision + +Write the tail as fixed-width steps in plain Rust. It needs **no backend edits +and no raw intrinsics** — so `simd_masking_ops.rs` keeps the `cfg(target_arch)`-free +property its own header claims (`:117`), there is no exception to the +all-SIMD-from-the-facade invariant, and it covers **all 11** algebra tails +rather than the 8 an x86 descent could reach, because NEON, wasm and scalar get +it for free. `avx512vl` stays unused in `src/` — this would have been its first +dependency anywhere in the tree. + +### Four measurement errors, all mine, each caught by a number the probe printed + +Kept because the failure modes generalize past this probe. + +1. **Confounded bodies.** The arms had different bodies, so at `n % 8 == 0` — + *no tail at all* — D/P still read 0.72–0.84. A tail strategy cannot move a + call with no tail. Shared one `body()`, then isolated with + difference-in-differences, `t(base+k) − t(base)` at fixed body size. +2. **A floor-thresholded COUNT is not a statistic.** "Widths where D beats S" + gave **2 of 14** then **10 of 14** on consecutive runs, off floors of + 1.35 ns and 0.64 ns. The magnitudes being thresholded were stable + throughout. A verdict that flips with the floor cannot decide a six-file + change. +3. **A per-pass median over 14 widths is itself noise** — it swung `D − S` + across **[−27, +34]** points. Pool across passes for the headline; print the + per-pass spread beside it so the scatter is visible rather than implied. +4. **A negative tail inflates a fraction.** `(pt − dt)/pt > 1` when `dt < 0`, + and pooling those produced *"D removes 121.3% of the padded cost"* — not a + quantity. This is the SAME asymmetric-noise error codex caught in the ratio + column on this very PR, **reintroduced one statistic later**: rejecting a + negative *denominator* is not enough when a negative *numerator* inflates + instead. Widths now qualify only when all four tail costs clear the floor. + +### Two rules worth carrying forward + +- **A probe must be able to say INCONCLUSIVE.** A k-word tail costs ~1 ns and + the floor on a busy machine is ~2 ns, so some runs genuinely cannot resolve + the question. It now reports that instead of manufacturing a verdict — and + notes that inconclusive is *not neutral about what to build*: D, S and F land + within a point or two, so the burden of proof is on the expensive option and + it has not been met. +- **`is_x86_feature_detected!` is the HOST, `cfg!(target_feature)` is the + BUILD.** The header printed the first, so a `--config .cargo/config-v3.toml` + build on this AVX-512 machine announced `avx512f=true`. That is exactly the + read-the-arm's-own-report failure entry (14) warns about, committed by the + session that wrote entry (14). It now prints both, labelled. + +--- + ## 2026-09-16 (14) — the default `target-cpu` now MEASURES THE HOST; a config that a caller can silently REPLACE was never a guarantee Three findings, one root cause, PR #313 (branch `claude/c64-6502-falsifier-shztkk`). From fa4532a8552dd0e7e8ef0a910400486019dc713b Mon Sep 17 00:00:00 2001 From: Claude Date: Wed, 16 Sep 2026 22:59:58 +0000 Subject: [PATCH 07/13] probe: answer the codegen bar the padded tail was chosen for, on aarch64 MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit The padded tail is not an oversight. `simd_masking_ops.rs:102-107` records it as a deliberate 2026-09-14 choice made for codegen UNIFORMITY: an exact-length scalar tail "fully unrolled ... on aarch64 into 7 x (and, orr) on GPRs ... in a facade op whose contract is `packed on every backend`", and became `vpmaskmovq` on AVX2. The previous commit recommended a peel while addressing only half of that rationale, on x86 only. The same header is explicit that "no throughput comparison against the old peel has been made", so a tail proposal has two bars to clear: 1. Throughput — what `mask_algebra_tail_probe` measures, and the gap the header names. The padded tail costs 8-20 ns against a ~3.5 ns body at 8 words. 2. Packed on every backend — now measured on the backend the objection was about. Emitting asm needs no linker and no qemu: rustup target add aarch64-unknown-linux-gnu rustc --target aarch64-unknown-linux-gnu -O --emit asm probe.rs A FIXED-width peel on aarch64: probe_fixed4_and: ldp q0, q3, [x1] / and v0.16b, v0.16b, v1.16b and v1.16b, v3.16b, v2.16b / stp q0, q1, [x2] probe_fixed2_and: ldr q0, [x0] / and v0.16b, v1.16b, v0.16b / str q0, [x2] Two NEON `and`s on 128-bit `v` registers, and one. Packed — not the GPR unroll the header warns about. The distinction the 2026-09-14 witness could not draw is between an EXACT-LENGTH tail, whose trip count is a runtime value, and a FIXED-WIDTH step, whose trip count is a constant. Only the first degenerates to GPRs, which is also why arm S loses to arm F in the throughput probe — one mechanism explains both measurements. So the recommendation is not that padding was wrong. Padding bought a real property with a real measurement behind it; fixed-width steps keep that property and stop paying 8-20 ns for it. Co-Authored-By: Claude Opus 5 Claude-Session: https://claude.ai/code/session_01DCfrD5y19cvFc4AoyydXYv --- examples/mask_algebra_tail_probe.rs | 29 ++++++++++++++++++++ examples/narrow_bitop_codegen_probe.rs | 37 ++++++++++++++++++++++++++ 2 files changed, 66 insertions(+) diff --git a/examples/mask_algebra_tail_probe.rs b/examples/mask_algebra_tail_probe.rs index 445da7c9..9af36722 100644 --- a/examples/mask_algebra_tail_probe.rs +++ b/examples/mask_algebra_tail_probe.rs @@ -12,6 +12,35 @@ //! The answer decides whether a `U64x4`/`U64x2` facade surface across six //! backend files is worth building. //! +//! # What the padded tail is actually FOR — read this before proposing a peel +//! +//! The zero-padded tail is not an oversight and not a speed choice. It was +//! chosen deliberately (`simd_masking_ops.rs:102-107`, codegen witness +//! 2026-09-14) for codegen UNIFORMITY: an exact-length scalar tail "fully +//! unrolled ... on aarch64 into 7 x (and, orr) on GPRs ... in a facade op +//! whose contract is `packed on every backend`", and became `vpmaskmovq` on +//! AVX2. Padding makes both arms one shape — packed body, packed tail. +//! +//! That header is also explicit that "no throughput comparison against the old +//! peel has been made". THAT is the gap this probe fills. So any tail proposal +//! clears two bars, and being faster is only the first: +//! +//! 1. **Throughput** — measured here. The padded tail costs 8-20 ns against a +//! body of ~3.5 ns at 8 words, so it is routinely larger than the work it +//! trails, and 7 of 8 mask sizes have one. +//! 2. **Packed on every backend** — measured in `narrow_bitop_codegen_probe`, +//! including on the backend the objection was about. A FIXED-WIDTH peel +//! emits `and v0.16b` on aarch64 and `vandps ymm`/`xmm` on x86. The +//! distinction the 2026-09-14 witness could not draw is between an +//! EXACT-LENGTH tail, whose trip count is a runtime value, and a +//! FIXED-WIDTH step, whose trip count is a constant. Only the first +//! degenerates to GPRs — which is also exactly why arm `S` below loses to +//! arm `F`. +//! +//! So the recommendation is not "padding was wrong". Padding bought a real +//! property with a real measurement behind it; fixed-width steps keep that +//! property AND stop paying 8-20 ns for it. +//! //! # The arms //! //! - **P — production.** `ndarray::simd::mask_and`, called directly. Not a diff --git a/examples/narrow_bitop_codegen_probe.rs b/examples/narrow_bitop_codegen_probe.rs index 092509d0..3402f8e2 100644 --- a/examples/narrow_bitop_codegen_probe.rs +++ b/examples/narrow_bitop_codegen_probe.rs @@ -19,6 +19,43 @@ //! ``` //! //! and grep the four probe symbols for `vpand` / `vandps` / `vpternlog`. +//! +//! # The aarch64 half, and it is the one that matters for the tail decision +//! +//! `simd_masking_ops.rs` does NOT zero-pad its tails for speed. Its header +//! (`:102-107`) records the reason as codegen UNIFORMITY: with an exact-length +//! scalar tail, LLVM "fully unrolled it on aarch64 into 7 x (and, orr) on +//! GPRs ... in a facade op whose contract is `packed on every backend`", while +//! the same loop on AVX2 became `vpmaskmovq`. Padding made both arms one +//! shape. The same header is explicit that "no throughput comparison against +//! the old peel has been made". +//! +//! So a tail proposal has to clear TWO bars, and speed is only the first. +//! `mask_algebra_tail_probe` supplies the missing throughput comparison; this +//! probe answers the codegen one, on the backend the objection was about: +//! +//! ```sh +//! rustup target add aarch64-unknown-linux-gnu # asm needs no linker, no qemu +//! rustc --target aarch64-unknown-linux-gnu -O --emit asm probe.rs +//! ``` +//! +//! Measured 2026-09-16, a FIXED-width peel on aarch64: +//! +//! ```text +//! probe_fixed4_and: ldp q0, q3, [x1] / and v0.16b, v0.16b, v1.16b +//! and v1.16b, v3.16b, v2.16b / stp q0, q1, [x2] +//! probe_fixed2_and: ldr q0, [x0] / and v0.16b, v1.16b, v0.16b / str q0, [x2] +//! ``` +//! +//! Two NEON `and`s on 128-bit `v` registers, and one. **Packed** — not the +//! GPR unroll the header warns about. The distinction the header's measurement +//! could not draw is between an EXACT-LENGTH tail, whose trip count is a +//! runtime value, and a FIXED-WIDTH step, whose trip count is a constant. +//! Only the first degenerates to GPRs. +//! +//! That is what makes fixed-width steps admissible rather than merely faster: +//! they PRESERVE the property padding was chosen to protect, on x86 and +//! aarch64 alike, while the padded form pays 8-20 ns for it. use ndarray::simd::{U64x4, U64x8}; From 101d7c9e83752a275586ebf22fe62a14339021e1 Mon Sep 17 00:00:00 2001 From: Claude Date: Wed, 16 Sep 2026 23:07:24 +0000 Subject: [PATCH 08/13] probe: the "pooled over all passes" line was reading ONE pass MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Caught by re-reading my own diff, and it is the same defect class this probe exists to document: a label that outran what the code computed. `s_frac`/`d_frac`/`f_frac` were cleared at the start of every pass, so the headline medians — printed as "pooled over all 7 passes" — were computed from the LAST pass alone, while `qualified`, `rows` and `wins` on the same screen accumulated across all seven. Two different denominators under one heading. The intended pooling was written earlier and never landed: `cargo fmt` reindented the target between the edit being composed and applied, so the string replacement silently matched nothing. The passing lint and the plausible output gave no sign. Now each pass records `pass_start` and slices its own contribution for the spread line, while the vectors accumulate for the headline. Also reports the median ABSOLUTE tail cost per arm in nanoseconds on the qualified widths. A share or a ratio can blow up when either term approaches zero — which is how this probe previously printed "D removes 121.3% of the padded cost" — and a nanosecond cannot, so those four numbers are the least-processed form of the result and the thing to sanity-check the verdict against. And the qualification filter's DIRECTION is now stated in the source rather than assumed: requiring dt/st/ft >= floor keeps the widths where the non-padded tails are most expensive and discards the ones too cheap to measure, so it is conservative AGAINST the peel arms, and the shares are a lower bound on what a peel saves. For D-vs-F the requirement is symmetric in both arms, so it restricts the subset without tilting the difference. Co-Authored-By: Claude Opus 5 Claude-Session: https://claude.ai/code/session_01DCfrD5y19cvFc4AoyydXYv --- examples/mask_algebra_tail_probe.rs | 29 +++++++++++++++++++++++------ 1 file changed, 23 insertions(+), 6 deletions(-) diff --git a/examples/mask_algebra_tail_probe.rs b/examples/mask_algebra_tail_probe.rs index 9af36722..3c552be0 100644 --- a/examples/mask_algebra_tail_probe.rs +++ b/examples/mask_algebra_tail_probe.rs @@ -418,10 +418,23 @@ mod imp { // so on some runs NOTHING qualifies and the only honest output is to // say the question was not answered. let mut qualified = 0usize; + // Absolute tail cost in ns on the qualified widths, per arm. Ratios and + // shares can blow up when a denominator or numerator approaches zero; + // a nanosecond cannot. These four numbers are the least-processed form + // of the result and are what the verdict should be sanity-checked + // against. + let mut p_ns = Vec::new(); + let mut d_ns = Vec::new(); + let mut s_ns = Vec::new(); + let mut f_ns = Vec::new(); for rep in 0..REPEATS { - s_frac.clear(); - d_frac.clear(); - f_frac.clear(); + // Mark where this pass's contributions start. The vectors are NOT + // cleared: the headline medians POOL every pass, and only the + // spread line slices out one pass at a time. Clearing here — which + // is what this code did until it was caught by re-reading the diff + // — left the line labelled "pooled over all N passes" reading the + // LAST pass alone, a false label on the headline number. + let pass_start = s_frac.len(); for &base in &[8usize, 64] { let ab = mk(base, 0xF0F0_5555_AAAA_1111); let bb = mk(base, 0x0FF0_1234_5678_9ABC); @@ -449,6 +462,10 @@ mod imp { // negative NUMERATOR inflates instead. if pt >= floor && dt >= floor && st >= floor && ft >= floor { qualified += 1; + p_ns.push(pt); + d_ns.push(dt); + s_ns.push(st); + f_ns.push(ft); s_frac.push((pt - st) / pt); d_frac.push((pt - dt) / pt); f_frac.push((pt - ft) / pt); @@ -486,9 +503,9 @@ mod imp { } } } - let sp = median(&mut s_frac.clone()) * 100.0; - let dp = median(&mut d_frac.clone()) * 100.0; - let fp = median(&mut f_frac.clone()) * 100.0; + let sp = median(&mut s_frac[pass_start..].to_vec()) * 100.0; + let dp = median(&mut d_frac[pass_start..].to_vec()) * 100.0; + let fp = median(&mut f_frac[pass_start..].to_vec()) * 100.0; if (dp - sp).is_finite() { ds_pts.push(dp - sp); } From 3853fc5e779866f7902cb8ebeb7634deea95287b Mon Sep 17 00:00:00 2001 From: Claude Date: Wed, 16 Sep 2026 23:13:28 +0000 Subject: [PATCH 09/13] probe: address four codex findings; withdraw the D-vs-F claim MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit All four were real. One I had already found by re-reading my own diff (the pooling bug, fixed in 101d7c9e); the other three are fixed here, and two of them contradict claims I had written down. 1. P1 — ternlog was never tested before dropping the VL gate. Correct: every timing arm is two-input AND, and `mask_ternlog` is the ONE op whose narrow descent would reach `_mm256_ternarylogic_epi64`. `probe_fixed4_ternlog` measures a fixed-step arbitrary three-input truth table: v4: vpor / vpternlogq $32 / vpternlogq $236 3 logic ops, 2 of them VL v3: vandnps/vandps/vorps/vandps/vorps 5 logic ops, packed, no VL Two findings pointing opposite ways. The `avx512vl` GATE is unnecessary even here — LLVM reaches `vpternlogq` on a 256-bit ymm from plain Rust when the target allows it, and degrades to packed boolean ops when it does not, so naming VL in our source would duplicate the compiler's tier selection. But one intrinsic is ONE instruction where LLVM used three, so ternlog's throughput question is OPEN and the AND arms do not close it. 2. P2 — the qualification filter selects on the very quantities being compared, and my note about it had the direction backwards. I claimed it was conservative against the peel arms and that symmetry left D-F untilted. The second half is false: near the floor an arm qualifies only on draws where its OWN error pushed it upward, which compresses the observed D-F gap and favours exactly the "fixed steps tie intrinsics" reading I drew from it. Symmetry applies the conditioning to both arms rather than removing it. The filter stays as the lesser evil — unfiltered gave "D removes 121.3% of the padded cost" — but the source now says the sample is CENSORED and the gap is a lower bound. 3. P2 — the INCONCLUSIVE branch still recommended an architecture. It said D, S and F were "within a point or two" and picked F, a measurement claim drawn from data the same paragraph had just rejected, on runs where those gaps may be NaN. It now stops without the claim. Also lands the absolute-nanosecond report that 101d7c9e's message claimed and its diff did not contain — the print was lost to the same `cargo fmt` reindent that ate the pooling edit, while the vectors were still being filled, so `clippy -D warnings` saw a live `push` and said nothing. Measured: median ABSOLUTE tail cost on the qualified widths (ns): P 16.82 D 5.24 S 6.28 F 5.33 Ratios and shares blow up when either term approaches zero; a nanosecond cannot. These are the least-processed form of the result. With pooling correct this machine reports INCONCLUSIVE (7 of 98 widths resolvable against a 3.65 ns floor), so the earlier headline `D-S = +2.9, D-F = -0.7` is WITHDRAWN: it came from the single-pass binary. What survives is that the padded tail costs ~3x every alternative, which sits an order of magnitude above the floor. Co-Authored-By: Claude Opus 5 Claude-Session: https://claude.ai/code/session_01DCfrD5y19cvFc4AoyydXYv --- examples/mask_algebra_tail_probe.rs | 40 ++++++++++++---- examples/narrow_bitop_codegen_probe.rs | 64 ++++++++++++++++++++++++++ 2 files changed, 96 insertions(+), 8 deletions(-) diff --git a/examples/mask_algebra_tail_probe.rs b/examples/mask_algebra_tail_probe.rs index 3c552be0..7f13a8e3 100644 --- a/examples/mask_algebra_tail_probe.rs +++ b/examples/mask_algebra_tail_probe.rs @@ -562,6 +562,32 @@ mod imp { " n/a".to_string() } }; + println!( + "\nmedian ABSOLUTE tail cost on the {qualified} qualified widths (ns):\n\ + \x20 P {:.2} D {:.2} S {:.2} F {:.2}", + median(&mut p_ns.clone()), + median(&mut d_ns.clone()), + median(&mut s_ns.clone()), + median(&mut f_ns.clone()) + ); + // ⊘ The filter is a SELECTION on the very quantities being compared, + // and the first version of this note had its direction backwards. It + // said the filter is "conservative AGAINST D, S and F" because it + // keeps the widths where their tails are most expensive, and that for + // D-vs-F it "restricts the subset without tilting the difference" + // because the requirement is symmetric. The second half is false + // (codex P2, #315): when the true costs sit near the floor — exactly + // this regime — an arm qualifies only on draws where its OWN + // measurement error pushed it upward. That truncation compresses the + // observed D-F gap toward zero, which systematically favours the + // "fixed steps tie intrinsics" reading. Symmetry does not remove the + // conditioning, it applies it to both arms. + // + // The filter stays because unfiltered is worse — negative tails gave + // "D removes 121.3% of the padded cost" — but this is a CENSORED + // sample and the D-F gap it yields is a lower bound on any true + // difference. Resolving it properly needs precision, not filtering: + // more iterations, a quiet machine, or modelling the censored points. println!("\npooled over all {REPEATS} passes -- S {} D {} F {}", pc(s_all), pc(d_all), pc(f_all)); println!("\nShare of the padded tail's cost removed, differenced per pass."); println!("Percentage-point gaps, pooled, with the per-pass spread beside them:"); @@ -605,14 +631,12 @@ mod imp { floor: every tail strategy removes most of the PADDED tail's 8-20 ns,\n\ which is the finding this probe was built to establish.\n\ \n\ - And inconclusive is not neutral about what to BUILD. D, S and F came\n\ - out within a point or two of each other, so the burden of proof sits\n\ - on the expensive option and it has not been met: nothing here supports\n\ - a `U64x4`/`U64x2` narrow type across six backend files plus an\n\ - `avx512vl` gate. Take the cheapest form that captures the padded-tail\n\ - win — fixed-width steps in plain Rust — which needs no backend edits\n\ - and no raw intrinsics, and so reaches the NEON, wasm and scalar tails\n\ - that an x86 descent never could." + This branch deliberately stops here. An earlier version went on to\n\ + say D, S and F had landed \"within a point or two\" and to recommend F\n\ + anyway — a measurement claim drawn from the data the same paragraph\n\ + had just rejected, on a run where those gaps may be NaN or arbitrarily\n\ + large (codex P2, #315). Nothing here establishes that closeness, so\n\ + nothing here recommends an architecture." ); } else if ds_med <= 2.0 { println!( diff --git a/examples/narrow_bitop_codegen_probe.rs b/examples/narrow_bitop_codegen_probe.rs index 3402f8e2..76b25723 100644 --- a/examples/narrow_bitop_codegen_probe.rs +++ b/examples/narrow_bitop_codegen_probe.rs @@ -56,6 +56,36 @@ //! That is what makes fixed-width steps admissible rather than merely faster: //! they PRESERVE the property padding was chosen to protect, on x86 and //! aarch64 alike, while the padded form pays 8-20 ns for it. +//! +//! # Ternlog is NOT settled by the AND arms — measured, and it cuts both ways +//! +//! Every other arm here is two-input AND, which is not the question for +//! `mask_ternlog` (codex P1, #315): that is the one mask-algebra op whose +//! narrow descent would reach for `_mm256_ternarylogic_epi64`, a VL +//! instruction. `probe_fixed4_ternlog` asks what LLVM does with a fixed-step +//! arbitrary three-input truth table. Measured 2026-09-16: +//! +//! ```text +//! v4: vpor %ymm0, %ymm2, %ymm3 +//! vpternlogq $32, %ymm0, %ymm1, %ymm2 +//! vpternlogq $236, %ymm1, %ymm2, %ymm3 <- 3 logic ops, 2 of them VL +//! v3: vandnps / vandps / vorps / vandps / vorps <- 5 logic ops, packed, no VL +//! ``` +//! +//! Two findings, and they point opposite ways: +//! +//! - **The `avx512vl` GATE is unnecessary even for ternlog.** LLVM emits +//! `vpternlogq` on a 256-bit `ymm` from plain Rust when the target supports +//! it, and degrades to packed boolean ops when it does not. Tier selection +//! is the compiler's job; naming VL in our source would only duplicate it. +//! - **But an intrinsic descent would still be strictly better HERE.** One +//! `_mm256_ternarylogic_epi64` is ONE instruction; LLVM used three. That is +//! a real gap, and it exists for ternlog alone — the AND arms lower to a +//! single `vandps`. +//! +//! So the throughput question for `mask_ternlog`'s tail is **OPEN**, and the +//! AND measurements must not be read as closing it. What IS closed: the other +//! ten algebra tails, and the gate. use ndarray::simd::{U64x4, U64x8}; @@ -86,6 +116,38 @@ pub extern "C" fn probe_scalar4_and(a: &[u64; 4], b: &[u64; 4], out: &mut [u64; } } +/// A fixed-step 4-lane arbitrary THREE-input truth table — the case that +/// decides whether `mask_ternlog`'s tail can drop AVX-512VL. +/// +/// Every other probe here is two-input AND, and two-input AND is not the +/// question (codex P1, #315): `mask_ternlog` is the ONE mask-algebra op whose +/// narrow descent would reach for `_mm256_ternarylogic_epi64`, which IS a VL +/// instruction. If LLVM lowers this to several boolean ops instead of one +/// `vpternlogq`, then the VL gate survives for ternlog even though the AND / +/// OR / XOR / ANDNOT tails do not need it. +/// +/// `IMM` is the 8-bit truth table, matching `mask_ternlog`'s own convention: +/// bit `(a<<2)|(b<<1)|c` of `IMM` is the output for that input triple. Written +/// as the canonical sum-of-minterms so nothing but the truth table is assumed. +#[inline(never)] +#[unsafe(no_mangle)] +pub extern "C" fn probe_fixed4_ternlog(a: &[u64; 4], b: &[u64; 4], c: &[u64; 4], out: &mut [u64; 4]) { + const IMM: u64 = 0xE8; // majority(a, b, c) — a table with no 2-input shortcut + for i in 0..4 { + let (x, y, z) = (a[i], b[i], c[i]); + let mut r = 0u64; + for m in 0..8u32 { + if (IMM >> m) & 1 == 1 { + let mx = if m & 4 != 0 { x } else { !x }; + let my = if m & 2 != 0 { y } else { !y }; + let mz = if m & 1 != 0 { z } else { !z }; + r |= mx & my & mz; + } + } + out[i] = r; + } +} + /// Two-lane scalar AND — the last rung of a descent, and the width at which /// auto-vectorization is least likely to be worth LLVM's while. #[inline(never)] @@ -106,6 +168,8 @@ fn main() { let b8 = [0x00FFu64; 8]; let mut o8 = [0u64; 8]; probe_u64x8_and(&a8, &b8, &mut o8); + let c4 = [0x0F0Fu64; 4]; + probe_fixed4_ternlog(&a4, &b4, &c4, &mut o4); let mut o2 = [0u64; 2]; probe_scalar2_and(&[1, 2], &[3, 3], &mut o2); println!("probe: {o4:?} {o8:?} {o2:?}"); From 797b0d8104e0c79d26bce865aee83da15ebee3ae Mon Sep 17 00:00:00 2001 From: Claude Date: Wed, 16 Sep 2026 23:15:10 +0000 Subject: [PATCH 10/13] =?UTF-8?q?blackboard=20(16):=20correct=20(15)=20?= =?UTF-8?q?=E2=80=94=20its=20timing=20headline=20came=20from=20a=20buggy?= =?UTF-8?q?=20binary?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit (15) stands on its codegen half and is withdrawn on its timing half; both are kept, per the append-only rule. Withdrawn: `D - S = +2.9, D - F = -0.7`. Those medians were labelled "pooled over all 7 passes" and read the last pass alone. With pooling correct the same machine reports INCONCLUSIVE — 7 of 98 widths resolvable against a 3.65 ns floor. Replaced by the least-processed form, which has no ratio pathology: median ABSOLUTE tail cost, qualified widths (ns): P 16.82 D 5.24 S 6.28 F 5.33 The padded tail is ~3x every alternative, an order of magnitude above the floor. The ordering of D against F (0.09 ns apart) is not established. Also records the codex finding that changed the architecture answer: every timing arm was two-input AND, so they could never have settled the VL question for `mask_ternlog`, the one op whose narrow descent would use `_mm256_ternarylogic_epi64`. Measured, it splits — the gate is unnecessary for a better reason than (15) gave (LLVM reaches `vpternlogq` from plain Rust and degrades cleanly, so a cfg gate would duplicate its tier selection), while ternlog's throughput question is OPEN because one intrinsic is one instruction where LLVM used three. And the correction I had written backwards in the source: the qualification filter is a censored sample whose truncation compresses the D-F gap toward the tie I was claiming, not a conservative one. The rule carried forward: absence of support for the expensive option is a reason not to build it yet, never evidence that the cheap one is equal. Co-Authored-By: Claude Opus 5 Claude-Session: https://claude.ai/code/session_01DCfrD5y19cvFc4AoyydXYv --- .claude/blackboard.md | 100 ++++++++++++++++++++++++++++++++++++++++++ 1 file changed, 100 insertions(+) diff --git a/.claude/blackboard.md b/.claude/blackboard.md index 4a0d2ee3..0bc63007 100644 --- a/.claude/blackboard.md +++ b/.claude/blackboard.md @@ -1,3 +1,103 @@ +## 2026-09-16 (16) — ⊘ CORRECTS (15): its headline numbers came from a BUGGY binary, and the D-vs-F ordering is NOT established + +Same PR (#315), same day, four codex findings later. Entry (15) stands on its +codegen half and is **withdrawn on its timing half**. Read them together; (15) +is not deleted. + +### What is WITHDRAWN from (15) + +> `D − S = +2.9 pts` and **`D − F = −0.7 pts`** + +Those came from a binary whose medians, printed as *"pooled over all 7 passes"*, +were computed from **the last pass alone**: `s_frac`/`d_frac`/`f_frac` were +cleared at the top of every pass while `qualified`, `rows` and `wins` on the +same screen accumulated across all seven — two denominators under one heading, +and a run could call itself resolvable off a single final-pass observation. + +**The pooling had been written and silently lost.** `cargo fmt` reindented the +target between the replacement being composed and applied, so the string match +found nothing, and a passing `clippy -D warnings` plus plausible output gave no +sign. **Second instance of that exact failure in one session** — the other ate +an entire `println!` block while leaving its `Vec::push` calls alive, so the +lint saw a used variable and the commit message shipped a claim the diff did +not contain. *Assert on the anchor, and diff the commit against the message.* + +### What replaces it + +With pooling correct, this machine reports **INCONCLUSIVE**: 7 of 98 widths +resolvable against a 3.65 ns floor. The least-processed form of the result, +which has no ratio pathology at all: + +``` +median ABSOLUTE tail cost, qualified widths (ns): + P 16.82 D 5.24 S 6.28 F 5.33 +``` + +**The padded tail is ~3× every alternative** — an order of magnitude above the +floor, so that part is solid and is what the probe was built to establish. The +ordering of D against F (0.09 ns apart here) is **not** established, and the +probe now says INCONCLUSIVE rather than picking. + +### The codex finding that changed the architecture answer: TERNLOG + +Every timing arm was two-input AND, and `mask_ternlog` is the ONE algebra op +whose narrow descent would reach `_mm256_ternarylogic_epi64` — a VL +instruction. So the AND arms could never have settled the VL question, and +(15) overreached in saying they did. Measured on a fixed-step arbitrary +three-input truth table: + +``` +v4: vpor / vpternlogq $32 / vpternlogq $236 3 logic ops, 2 of them VL +v3: vandnps/vandps/vorps/vandps/vorps 5 logic ops, packed, no VL +``` + +Two halves, opposite directions: + +- **The `avx512vl` GATE is unnecessary — and for a better reason than (15) + gave.** LLVM reaches `vpternlogq` on a 256-bit `ymm` from PLAIN RUST when the + target allows it and degrades to packed boolean ops when it does not. Naming + VL in our source would duplicate tier selection the compiler already does. +- **Ternlog's THROUGHPUT question is OPEN.** One intrinsic is one instruction + where LLVM used three. That gap is ternlog-specific; the AND arms lower to a + single `vandps`. + +### The statistical correction, because I had it backwards IN THE SOURCE + +Widths contribute only when all four tail costs clear the floor (unfiltered, +negative tails gave *"D removes 121.3% of the padded cost"*). My comment claimed +this was *conservative against* the peel arms and that symmetry left D−F +untilted. **The second half is false.** Near the floor an arm qualifies only on +draws where its OWN error pushed it upward, so the observed D−F gap is +compressed toward zero — favouring exactly the "fixed steps tie intrinsics" +reading I drew from it. Symmetry applies the conditioning to both arms rather +than cancelling it. The filter stays as the lesser evil; the source now calls +the sample **censored** and the gap a **lower bound**, and names precision (more +iterations, a quiet machine) as the fix rather than more filtering. + +### And the overclaim in the INCONCLUSIVE branch + +It reported "no support" and then asserted D, S and F were "within a point or +two" and recommended F — a measurement claim drawn from data the same paragraph +had just rejected. Removed; that branch now recommends nothing. + +### Net standing position + +| claim | status | +|---|---| +| padded tail costs ~3× any alternative | **established** | +| `U64x4 &` is bit-identical to a hand-written loop (assembler MERGES the symbols) | **established** | +| fixed-width peels are packed on aarch64 too (`and v0.16b`) | **established** | +| `avx512vl` gate unnecessary, incl. ternlog | **established** (codegen) | +| narrow `U64x4`/`U64x2` facade type | **no measured support** — which is not the same as shown equivalent | +| F ties D | **NOT established** — needs a quiet machine | +| ternlog tail throughput | **OPEN** | + +The **rule** worth carrying: absence of support for the expensive option is a +reason not to build it yet, never evidence that the cheap one is equal. (15) +blurred those; this entry separates them. + +--- + ## 2026-09-16 (15) — the mask-algebra tail wanted a fixed TRIP COUNT, not intrinsics; the `U64x4`/`U64x2` + `avx512vl` build is CANCELLED PR #315 (branch `claude/mask-algebra-tail-probe`). The planned step (b) — give From 525f0523dec40d939e95a95063c5ad6964c097f0 Mon Sep 17 00:00:00 2001 From: Claude Date: Wed, 16 Sep 2026 23:16:55 +0000 Subject: [PATCH 11/13] probe: the (S - D) median was pooled and unfiltered, labelled per-pass MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit coderabbit, #315. `ds_gap` is declared outside the `rep` loop and never cleared, so the median covers every pass, and it is pushed before the qualification gate so it also covers every width — including the noise-dominated ones the percentage statistics deliberately exclude. The line called it "within pass 1". Both facts are now in the label. This is the same defect class as the pooling bug one screen up — a label outrunning what the code computes — which is the point worth keeping: finding one instance of that is not the same as finding them all. So this commit also audits the other twelve accumulators rather than waiting for the next review to find the next one. All thirteen (`ds_gap`, `s_frac`/`d_frac`/`f_frac`, `p_ns`/`d_ns`/`s_ns`/`f_ns`, `ds_pts`/`df_pts`, `qualified`, `rows`, `wins`) are declared before the loop and none is cleared, so every one pools; each label now says so, the per-pass spread is the only thing sliced by `pass_start`, and the noise-floor count (28) was checked against its 7 iterations x 4 arms rather than trusted. Co-Authored-By: Claude Opus 5 Claude-Session: https://claude.ai/code/session_01DCfrD5y19cvFc4AoyydXYv --- examples/mask_algebra_tail_probe.rs | 13 ++++++++++++- 1 file changed, 12 insertions(+), 1 deletion(-) diff --git a/examples/mask_algebra_tail_probe.rs b/examples/mask_algebra_tail_probe.rs index 7f13a8e3..15a78454 100644 --- a/examples/mask_algebra_tail_probe.rs +++ b/examples/mask_algebra_tail_probe.rs @@ -524,7 +524,18 @@ mod imp { // ── Step 3: the question that decides what gets BUILT ───────────── println!("\n── what does the tail actually need: intrinsics, or a fixed trip count? ──"); let med_gap = median(&mut ds_gap); - println!("median (S - D) within pass 1: {med_gap:.2} ns [noise floor {floor:.2} ns]"); + // `ds_gap` is declared outside the `rep` loop and never cleared, so this + // is pooled over EVERY pass, and it is pushed before the qualification + // gate so it covers EVERY width — including the noise-dominated ones the + // percentage statistics below exclude. Both facts belong in the label: + // it read "within pass 1" until coderabbit caught it on #315, the same + // defect class as the pooling bug one screen up (a label outrunning what + // the code computes) and a reminder that finding one instance of that + // is not the same as finding them all. + println!( + "median (S - D), pooled over all {REPEATS} passes and ALL widths (unfiltered): \ + {med_gap:.2} ns [noise floor {floor:.2} ns]" + ); let lo = |v: &[f64]| { if v.is_empty() { From 5385ebf831e8848ae39949299f9cd6218b131644 Mon Sep 17 00:00:00 2001 From: Claude Date: Wed, 16 Sep 2026 23:20:15 +0000 Subject: [PATCH 12/13] probe: mark the (S - D) median when it is below its own resolution MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Fifth instance of the defect class this PR keeps turning up — a number printed without the fact that the measurement cannot support it. Observed on the run that prompted this commit: the line printed `1.33 ns` against a noise floor of `1.57 ns`, i.e. a value indistinguishable from zero, as a bare figure, while every per-width figure on the screen above it already carries `~n` in exactly that situation. The line now carries the same `~noise` marker under the same condition (`med_gap.abs() < floor`). Worth stating as the general form, since one instance of this class was fixed three times in this PR before the class itself was named: finding a mislabelled statistic is not the same as finding the mislabelled statistics, and the cheap sweep — read every print against what its accumulator actually holds — is what catches the rest. Co-Authored-By: Claude Opus 5 Claude-Session: https://claude.ai/code/session_01DCfrD5y19cvFc4AoyydXYv --- examples/mask_algebra_tail_probe.rs | 8 +++++++- 1 file changed, 7 insertions(+), 1 deletion(-) diff --git a/examples/mask_algebra_tail_probe.rs b/examples/mask_algebra_tail_probe.rs index 15a78454..ee63627b 100644 --- a/examples/mask_algebra_tail_probe.rs +++ b/examples/mask_algebra_tail_probe.rs @@ -532,9 +532,15 @@ mod imp { // defect class as the pooling bug one screen up (a label outrunning what // the code computes) and a reminder that finding one instance of that // is not the same as finding them all. + // ...and mark it when it is below its own resolution, as every other + // figure here is. Measured on the run that prompted this: 1.33 ns + // against a 1.57 ns floor — a bare number there is the same defect one + // more time, a value printed without the fact that it cannot be + // distinguished from zero. println!( "median (S - D), pooled over all {REPEATS} passes and ALL widths (unfiltered): \ - {med_gap:.2} ns [noise floor {floor:.2} ns]" + {med_gap:.2} ns{} [noise floor {floor:.2} ns]", + if med_gap.abs() < floor { " ~noise" } else { "" } ); let lo = |v: &[f64]| { From 80b80e087032779dfd6743bf8dee55a90abfed68 Mon Sep 17 00:00:00 2001 From: Claude Date: Wed, 16 Sep 2026 23:31:47 +0000 Subject: [PATCH 13/13] probe: wrong unit, an unsupported bound, and a checked ternlog claim MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Three coderabbit findings on 5385ebf8. Two are errors the PREVIOUS correction introduced, which is the third generation of one defect class in this PR and the thing worth recording. 1. The unit was OBSERVATIONS, not widths. The sweep visits 2 bases x 7 k = 14 distinct widths and repeats them 7 times, so the counter tops out at 98 samples of 14 widths. "7 of 98 widths" invents 84 widths that do not exist. The measured numbers are unchanged; what was wrong is what they counted. 2. "the gap is a LOWER BOUND on any true difference" is withdrawn. Truncation compresses the observed gap toward zero — but only from whichever side the truly cheaper arm sits on, and which arm that is is exactly what this sample cannot say. A bound needs the sign first. Replaced with: selection-biased, direction and magnitude unresolved. That is strictly more honest and costs nothing, because the decision never rested on the size of D - F, only on the absence of support for the expensive option. 3. Asked for a rustdoc example on `probe_fixed4_ternlog`. Declined as asked, and done better: this file is in `examples/`, where rustdoc never runs, so a ``` block would be an untested assertion dressed as a verified one — the exact thing this PR spent its time removing. `main` now asserts the semantics instead, so the claim is checked on every run. That assertion earned its place immediately: it rejected the expected value I had written from doing the arithmetic in my head. I had 0xF0F0 & 0x0F0F as 0x0F00 when the nibbles do not overlap at all, so it is 0x0000, and majority is 0x00FF rather than 0x0FFF. A truth table transcribed one bit off would still emit plausible `vpternlogq` and survive a read of the assembly — and so would a doc example nobody executes. Blackboard (17) corrects (16) on both counts and tabulates all five instances of the class, with the mechanical sweep that finds them faster than review does: for every printed line, name the accumulator behind it, its UNIT and its SCOPE, then check the words against all three. Co-Authored-By: Claude Opus 5 Claude-Session: https://claude.ai/code/session_01DCfrD5y19cvFc4AoyydXYv --- .claude/blackboard.md | 65 ++++++++++++++++++++++++++ examples/mask_algebra_tail_probe.rs | 29 +++++++++--- examples/narrow_bitop_codegen_probe.rs | 26 +++++++++++ 3 files changed, 113 insertions(+), 7 deletions(-) diff --git a/.claude/blackboard.md b/.claude/blackboard.md index 0bc63007..6c539dce 100644 --- a/.claude/blackboard.md +++ b/.claude/blackboard.md @@ -1,3 +1,68 @@ +## 2026-09-16 (17) — ⊘ CORRECTS (16) twice: the unit was OBSERVATIONS not widths, and "lower bound" was one claim too many + +Both caught by coderabbit on the same PR (#315), both are errors (16) introduced +while correcting (15), and both are the SAME defect class (15) and (16) already +name — a label or a claim outrunning what the measurement supports. Third +generation of one mistake in one session, which is the finding worth keeping. + +### 1. "7 of 98 widths" — wrong unit, overstated the design 7× + +The sweep visits `2 bases × 7 k = 14` **distinct widths** and repeats them +`REPEATS = 7` times, so the counter tops out at **98 OBSERVATIONS of 14 +widths**. (16) reported "7 of 98 widths", which invents 84 widths that do not +exist. Corrected in both the probe's labels and here: + +``` +median ABSOLUTE tail cost, 7 qualified observations of 14 widths (ns): + P 16.82 D 5.24 S 6.28 F 5.33 +observations where all four tails were measurable: 7 of 98 + (14 distinct widths × 7 passes) +``` + +The measured numbers are unchanged; what was wrong is what they were counting. + +### 2. "the gap is a LOWER BOUND" — not established, withdrawn + +(16) said the censored D−F gap is "a **lower bound** on any true difference". +That is one claim too many. Truncation does compress the gap toward zero — but +only toward zero **from whichever side the truly cheaper arm sits on**, and +which arm that is is precisely what this sample cannot say. A bound needs the +sign first. So the honest statement, now in both places: + +> **selection-biased; direction and magnitude unresolved.** + +Note this is STRICTLY more honest than the claim it replaces, and it weakens +nothing that mattered: the decision never rested on the size of D−F, only on +the absence of support for the expensive option. + +### Why this keeps happening, stated so the next session can shortcut it + +Five instances, one class, three of them introduced *by the fix for the +previous one*: + +| # | the claim | what the code/design actually had | +|---|---|---| +| 1 | D/P ratio read as a tail result | arms with different bodies; `tail == 0` rows moved | +| 2 | "widths where D beats S: N of 14" | a count thresholded by a floor that is itself a draw | +| 3 | "pooled over all 7 passes" | the last pass alone | +| 4 | "D removes 121.3% of the padded cost" | a fraction inflated by a negative numerator | +| 5 | "7 of 98 **widths**" / "a **lower bound**" | 98 observations of 14 widths / no established sign | + +**The generalizable rule: correcting one statistic does not audit the ones +beside it, and the correction itself is a new claim needing the same check.** +The cheap sweep is mechanical — for every printed line, name the accumulator +behind it, its UNIT, and its SCOPE, then read whether the words match all +three. That sweep found instance 5's siblings in one pass (all thirteen +accumulators pool; none is cleared) where four rounds of review had found them +one at a time. + +Corollary for a verdict rather than a label: state only what the sample can +support. "No measured support for the expensive option" survives every one of +the five corrections above. "The cheap option is equal", "the gap is a lower +bound", and every ratio drawn from a near-zero denominator did not. + +--- + ## 2026-09-16 (16) — ⊘ CORRECTS (15): its headline numbers came from a BUGGY binary, and the D-vs-F ordering is NOT established Same PR (#315), same day, four codex findings later. Entry (15) stands on its diff --git a/examples/mask_algebra_tail_probe.rs b/examples/mask_algebra_tail_probe.rs index ee63627b..fca7ce68 100644 --- a/examples/mask_algebra_tail_probe.rs +++ b/examples/mask_algebra_tail_probe.rs @@ -418,7 +418,12 @@ mod imp { // so on some runs NOTHING qualifies and the only honest output is to // say the question was not answered. let mut qualified = 0usize; - // Absolute tail cost in ns on the qualified widths, per arm. Ratios and + // Absolute tail cost in ns on the qualified OBSERVATIONS, per arm. + // "Observation", not "width": the sweep visits 2 bases x 7 k = 14 + // distinct widths and repeats them REPEATS times, so the counter tops + // out at 98 samples of 14 widths. Calling 98 a width count overstates + // the design by 7x (coderabbit, #315) — the same wrong-unit defect as + // the pooled-vs-per-pass labels, one field over. Ratios and // shares can blow up when a denominator or numerator approaches zero; // a nanosecond cannot. These four numbers are the least-processed form // of the result and are what the verdict should be sanity-checked @@ -580,7 +585,7 @@ mod imp { } }; println!( - "\nmedian ABSOLUTE tail cost on the {qualified} qualified widths (ns):\n\ + "\nmedian ABSOLUTE tail cost, {qualified} qualified observations of 14 widths (ns):\n\ \x20 P {:.2} D {:.2} S {:.2} F {:.2}", median(&mut p_ns.clone()), median(&mut d_ns.clone()), @@ -601,10 +606,19 @@ mod imp { // conditioning, it applies it to both arms. // // The filter stays because unfiltered is worse — negative tails gave - // "D removes 121.3% of the padded cost" — but this is a CENSORED - // sample and the D-F gap it yields is a lower bound on any true - // difference. Resolving it properly needs precision, not filtering: - // more iterations, a quiet machine, or modelling the censored points. + // "D removes 121.3% of the padded cost" — but this is a CENSORED sample: + // SELECTION-BIASED, with the direction and magnitude of the bias + // UNRESOLVED. + // + // ⊘ An earlier version of this note called the observed D-F gap a + // "lower bound on any true difference". That is one claim too many + // (coderabbit, #315). Truncation does compress the gap toward zero — + // but only toward zero FROM WHICHEVER SIDE the truly cheaper arm sits + // on, and which arm that is is precisely what this sample cannot say. + // Asserting a bound requires knowing the sign first, so neither the + // sign nor the magnitude of the true difference is recoverable here. + // Resolving it needs precision, not filtering: more iterations, a + // quiet machine, or modelling the censored points. println!("\npooled over all {REPEATS} passes -- S {} D {} F {}", pc(s_all), pc(d_all), pc(f_all)); println!("\nShare of the padded tail's cost removed, differenced per pass."); println!("Percentage-point gaps, pooled, with the per-pass spread beside them:"); @@ -632,7 +646,8 @@ mod imp { // adjudicate the finer D-vs-F question, and saying so is the result. let resolvable = qualified * 4 >= rows && ds_med.is_finite() && df_med.is_finite(); println!( - "widths where all four tails were measurable: {qualified} of {rows} ({}resolvable)", + "observations where all four tails were measurable: {qualified} of {rows} \ + (14 distinct widths x {REPEATS} passes) ({}resolvable)", if resolvable { "" } else { "NOT " } ); if !resolvable { diff --git a/examples/narrow_bitop_codegen_probe.rs b/examples/narrow_bitop_codegen_probe.rs index 76b25723..edf5527b 100644 --- a/examples/narrow_bitop_codegen_probe.rs +++ b/examples/narrow_bitop_codegen_probe.rs @@ -129,6 +129,21 @@ pub extern "C" fn probe_scalar4_and(a: &[u64; 4], b: &[u64; 4], out: &mut [u64; /// `IMM` is the 8-bit truth table, matching `mask_ternlog`'s own convention: /// bit `(a<<2)|(b<<1)|c` of `IMM` is the output for that input triple. Written /// as the canonical sum-of-minterms so nothing but the truth table is assumed. +/// +/// `IMM = 0xE8` is bitwise MAJORITY — a bit is set where at least two of the +/// three inputs have it set: +/// +/// ```text +/// a = 0xF0F0, b = 0x00FF, c = 0x0F0F +/// a&b = 0x00F0 a&c = 0x0000 b&c = 0x000F -> 0x00FF +/// ``` +/// +/// This is deliberately NOT written as a rustdoc example. This file lives in +/// `examples/`, where rustdoc never runs, so a ``` block here would be an +/// untested assertion dressed as a verified one — the exact thing the rest of +/// this PR spent its time removing (coderabbit asked for a usage example, +/// #315). `main` asserts the triple above instead, so the claim is CHECKED on +/// every run rather than decorated. #[inline(never)] #[unsafe(no_mangle)] pub extern "C" fn probe_fixed4_ternlog(a: &[u64; 4], b: &[u64; 4], c: &[u64; 4], out: &mut [u64; 4]) { @@ -168,8 +183,19 @@ fn main() { let b8 = [0x00FFu64; 8]; let mut o8 = [0u64; 8]; probe_u64x8_and(&a8, &b8, &mut o8); + // Pin the ternlog arm's SEMANTICS, not just that it runs: `IMM = 0xE8` is + // bitwise majority, so a bit survives where at least two inputs set it. + // 0xF0F0/0x00FF/0x0F0F pairwise-AND to 0x00F0 | 0x0000 | 0x000F = 0x00FF. + // + // This assertion earned its place on the first run: it rejected 0x0FFF, + // which is what I had written from doing the arithmetic in my head (I had + // 0xF0F0 & 0x0F0F as 0x0F00 when the nibbles do not overlap at all, so it + // is 0x0000). A truth table transcribed one bit off would still emit + // plausible `vpternlogq` and survive a read of the assembly — and so would + // a doc example nobody executes. This fails instead. let c4 = [0x0F0Fu64; 4]; probe_fixed4_ternlog(&a4, &b4, &c4, &mut o4); + assert_eq!(o4, [0x00FFu64; 4], "IMM=0xE8 must be bitwise majority: 0xF0F0/0x00FF/0x0F0F -> 0x00FF"); let mut o2 = [0u64; 2]; probe_scalar2_and(&[1, 2], &[3, 3], &mut o2); println!("probe: {o4:?} {o8:?} {o2:?}");