model: the host union's ungated relu² form, for Bolt's routed experts

[midagedev]

What and why

The host union (model::moe) gets the ungated relu² form, so Bolt's routed experts (up [w,1024] → relu² → down [1024,w], no gate, inside the 1024 latent) can run on the host tier. It is an arm of the common union beside the gated SwiGLU form, not a Bolt copy: ExpertForm::Relu2 reads one stack where the gated form reads two (a UnionPass::Up first pass), and the combine is relu(up)². A NaN or an infinity in the up output is refused by name. The union scratch's expert width becomes a capacity, so one scratch serves a run's layers of differing widths (Bolt's ff runs 1536–2560 per layer). Prerequisite of the Bolt offload work (B3, #328); it changes no kernel.

Release notes: internal — host-tier prerequisite for offloading Bolt's routed experts; no user-visible change.

Signatures B3 calls

• HostLayer::experts_union_into<'x>(&self, src: R8Source<'_>, x: impl Into<Tensor2View<'x>>, lists: &[&[(u32, f32)]], out: &mut [f32], scratch: &mut UnionScratch) -> Result<(), ModelError> (unchanged shape; experts_step_union_into takes the same).
• A layer: arch::nemotron_h_moe::host::layer(src, &hp, l) -> Result<Option<HostLayer>, ModelError> — None for a layer that does not route (so a sparse Vec<Option<HostLayer>>), Err for a layer past the model. Or HostLayer::build_relu2(src, &Relu2Spec { up, down, n_expert, embd, ff }) (no gate field).
• Scratch: UnionScratch::new_routed(embd = 1024, ff = the widest layer's, cols, n_used = 9). A layer wider than the scratch, or another embd, is refused by name; a narrower layer gives the same bits as through a scratch made for it.
• An NVMe tier on a relu² layer is refused by name (TierError::Relu2Tiered) at the call (attach_tier itself does not refuse).

Code shape

• No second copy. The relu² combine, the first pass and the scratch capacity are in the common union (ops.rs, moe.rs). nemotron_h_moe/host.rs only wires the latent and each layer's shapes; the stack names come from the common arch::names::routed_stacks.
• Model code holds only architecture facts. host.rs: which layers route, each layer's own n_expert and ff, the latent, the relu² form. The common owner gained a form arm (ExpertForm), no per-model branch.
• Per-model copies. Added: none under arch/; arch/nemotron_h_moe/host.rs is the one new file and holds only the facts above. Removed from this PR's own earlier shape: two Bolt-only name functions in arch/nemotron_h_moe/names.rs, moe.rs's test relu2_file, and the tests' own copies of seeded, diff_cells, panic_text and the two layer-file writers (now tests/common/columns.rs, tests/common/panics.rs and one StackFile::write in tests/union.rs).
• Dead code is gone. The WIP's gate-name field and its form-vs-file refusal are removed: nothing in the tree could reach them.
• No silent failure. A layer past the model, a scratch narrower than the layer, another embd, a relu² layer with a tier, and a non-finite up value are each a named error; each has a gate clause and a mutant.
• Comments state what is true now. tools/check-comments.sh: ok.
• No relaxed gate. The union gate's existing refusal "scratch of other widths" still passes with the same text.
• cargo fmt clean; cargo clippy -p bloomery-model --all-targets: one warning, ops.rs collapsible_if (existing, from 9e894eb). Workspace just lint not run by me.

Duplication found or removed: see "Per-model copies" above. Still open and not touched here (other families, for a follow-up): the (gate, up, down) name triple in deepseek41/host.rs, glm5next/host.rs, mimo2/host.rs, qwen35moe/host.rs can take routed_stacks; three synthetic stack byte generators (tests/common/r8layer.rs, moe.rs tests stack_bytes, tests/union.rs q8_stack).

Gates (box, tools/gate-batch.sh --round-ledger, real tier, head a2c9c1c6d)

gate	result	box job
gate-union (8 tests: V4.1 ×3, V2-Lite, Q6_K/Q8_0/r8 synthetic, relu² new)	PASS	s2357
gate-nemotron-h-moe-host (new, Bolt file)	PASS	s2356
gate-ops (lib 384 + ops 8; group _into gate gains relu² and non-finite clauses)	PASS	s2365
gate-moe / gate-alloc / gate-mt / gate-r8 / gate-ds41-host	PASS	s2370 / s2369 / s2378 / s2379 / s2364
gate-qwen4exp-meta / gate-nemotron-h-moe-meta	PASS	s2384 / s2388

DONE total=10 red=0 tier=real skipped=0 wall=2606s laneA=2605s laneB=2076s. Every log shows Compiling bloomery-model.

Narrowing (just narrow origin/main, at 948cac02e)

96240135..948cac02: ptx-scan --no-jit of gate_glm5next_e2e, gate_kquant, gate_swap is identical on both sides (crates/model holds no device code), so 159 selected recipes narrow to 122. The narrowed list ran in the fixture tier:

Fixture tier, head 948cac02e over 962401358 (tools/gate-batch.sh --list narrowed.txt --round-ledger --tier fixture): 83 recipes ran and 48 are deferred to the real tier (29 real-only, 1 of a family with no fixture, 18 real-file; deferred.list).

• First pass: DONE total=131 red=28 tier=fixture wall=2665s (55 green). 21 of the 28 reds were my own doing: I exported BLOOMERY_Q_CARD=any into the batch, which box.sh refuses (rc 64) beside the recipes' own BLOOMERY_CARD pick. The 28 re-ran without it: DONE total=28 red=7 tier=fixture wall=2003s, so 76 of 83 are green.
• The 7 reds are not this change:
  • Files absent on the box: gate-gpu-nemotron-e2e and gate-gpu-nemotron-serve (the nemotron_h family has no fixture file, rc 66), gate-gemma-embedding-meta, gate-gemma-embedding2-meta, gate-gpu-gemma-embedding2-e2e (/models/embeddinggemma-2/embeddinggemma-2-Q8_0.gguf is missing).
  • gate-gpu-qwen4exp-e2e (job s2182) and gate-gpu-qwen4exp-mtp (s2197) fail one card-plan clause each on the 3090 (and on the A6000 in the first pass): card leg plan: 4 of 4 layers hold routed experts on the card, a layer's count Some(255)..Some(256) (want 256 on every layer: half the experts, the header's budget) FAIL. The same line, with Compiling bloomery-model in the log, fails on origin/main e195c9e6a (jobs s2230, s2235, BLOOMERY_GATE_CARD=3090, fixture tier). The plan code is not touched here.
• The two commits after the fixture run (bcb7332bf, a2c9c1c6d) change only crates/model host-tier source and tests (no device code). The ten owning gates above re-ran on them; the fixture batch was not re-run.

FAIL-first (one mutant each, red, then reverted; every log shows Compiling bloomery-model)

mutant	caught by
relu not squared / combine reads the wrong slot / first pass writes the wrong slot / leg reads two up blocks	union relu² clause
capacity check removed (both halves) / column-room half removed	union relu² refusal clauses
caller-side and claim-side combine copy the up (write_par, col_into)	union relu² / group gate (ne1 = 10 through matmul_q_group_cols_into)
warm_spans warms three stacks / tier refusal removed (one-column, union)	moe unit test
non-MoE layer is Err (the WIP's behaviour) / embd is the model's, not the latent	gate-nemotron-h-moe-host
routed_stacks up and down swapped in host::layer	gate-nemotron-h-moe-host (rc 101)
non-finite refusal removed (union / group gate) / refusal catches NaN only	union relu² overflow clause / group gate ±inf clauses

The col_into mutant survived the union clause and the ne1 = 1, 6 clauses (only the bench-only matmul_q_group_cols_into with an input past DEFER_MAX_COLS reaches it); the group gate now runs it at ten columns and that mutant is red.

GPU Evidence

• Box: 0, NVIDIA GeForce RTX 3090, 615.71.09 and 1, NVIDIA RTX A6000, 615.71.09 (nvidia-smi); AMD Ryzen Threadripper PRO 5975WX, Linux 6.8.0-142-generic. These gates run the host tier on the CPU (AVX2 qdot kernels); no device kernel changed, and no card kernel ran in them.
• Commits: gates ran on head a2c9c1c6d, whose merge base with main is db5c89e7c (main has since moved to c14c46115; the PR merges cleanly, the gates were not re-run on it). Narrow and fixture runs: base 962401358, head 948cac02e.
• Commands: tools/gate-batch.sh --list h1-gates.list --round-ledger (jobs above); just narrow origin/main; tools/gate-batch.sh --list narrowed.txt --round-ledger --tier fixture.
• Output (raw PASS lines, head a2c9c1c6d):

PASSED: union relu² — IQ2_XXS up and down, ungated: the union call at k = 1, 8, 9, 64 on two layers of different widths through one scratch made for the wider equal to experts_into and to the transcription on qdot's calls, bit for bit under both deferral arms and in their dispatches; the widths' refusals by name, and an overflowing up row refused by name
PASSED: nemotron_h_moe host — 40 MoE layers build as relu², the rest as none; layers 3 and 21 (the narrowest and widest ff) through one scratch for ff 2560: union equal to experts_into at k = 1, 9 under both deferral arms
test result: ok. 384 passed; 0 failed (gate-ops lib)
test result: ok. 8 passed; 0 failed (gate-ops ops)

Timing

No timed run was made. Derived, not measured: the form choice is one predictable branch per expert combine, and the relu² finiteness check is a branch-free OR over the up row, about 0.05–0.1 µs per 2560-wide combine against a derived ~6.6 µs per host expert (under 2 %). A same-lease just ab-decode A/B belongs to perf.

---

AI use: this change and its gates were written and reviewed by a Claude agent session (Claude Sonnet 5.5, the h1 worker of the Bolt offload team); the lead reviews and merges. This text was written by that bot.

🤖 Generated with Claude Code

[github-actions]

Verdict: MUST_FIX: relu² hides invalid activations; the change also duplicates a model identity and adds redundant parameters.

Must fix

1. Reject non-finite values before applying relu²

crates/model/src/ops.rs:1643: matmul_q_group_swiglu accepts a shape-valid GroupInput::Relu2 tensor containing NaN or negative infinity. This loop turns either into zero before quantization. With otherwise zero activations and finite down weights, the call returns zero instead of the named failure required by AGENTS.md.
Fix the shared combine now, before every host execution path inherits this behavior. Add a refusal clause for NaN and both infinities.

fn relu2_into(up: &[f32], out: &mut [f32]) {
    for (o, &u) in out.iter_mut().zip(up) {
        assert!(u.is_finite(), "relu2: non-finite activation value {u}");
        let r = u.max(0.0);
        *o = r * r;
    }
}

2. Reuse the reference model path

crates/model/tests/nemotron_h_moe_host.rs:32 repeats crates/refset/src/arch/nemotron_h_moe/mod.rs:22. The new gate creates another owner for the same model identity.
Reuse the owner now so a model-path change does not require another coordinated edit. Delete the local constant and import:

use refset::arch::nemotron_h_moe::MODEL as FILE;

3. Remove the redundant sizing parameters

crates/model/src/ops.rs:5046 takes slots and ff, but both callers pass dimensions already owned by this UnionCall. These new parameters add no choice.
Remove them while there are only two call sites. Replace both calls with self.gu_cells() and the definition with:

fn gu_cells(&self) -> usize {
    let cells = self.plan.slots() * self.up.n;
    match self.form {
        ExpertForm::Gated { .. } => 2 * cells,
        ExpertForm::Relu2 => cells,
    }
}

Non-blocking

⚠ Host dispatch timing evidence: crates/model/src/ops.rs:695 adds a form selection per expert combine: experts.len() selections per served host leg/token [derived from serve_resolved at moe.rs:1055–1075]. AGENTS.md requires a same-lease A/B for dispatch changes. Attach the gated-host A/B commands and output; the reported bit and dispatch-count gates do not measure speed.

Checked

Duplicates: compared the cited model identity and helper bodies; dryer reported zero pairs.
Complexity: crapper flagged new HostLayer::build_form at 22; its branches validate stacks and resolve sources at load time.
Silent failure: traced relu² into the down projection; quantization cannot detect the masked inputs.
Tests: the PR reports FAIL-first and host gates; runtime gates were not independently run during validation.
Gates: the scratch expert width becomes a capacity.
Escape catalog: no C1–C4 patterns matched.

Notes

The PR reports CPU host-tier gates and identical PTX scans. Its selected fixture batch remains FIXTURE_BATCH_PENDING.

_{🔨 Shingler, bloomery's review bot: approves a clean pull request and requests changes on a must-fix (how it works)}

Filed outside this PR: #357

[github-actions]

Reject NaN and negative infinity before max converts them to a valid zero activation.

Suggested change

	let r = u.max(0.0);
	assert!(u.is_finite(), "relu2: non-finite activation value {u}");
	let r = u.max(0.0);

[github-actions]

Since the last review:

The non-finite refusal remains open (item 1). This increment only updates gate selection; no source or test code changed.

Verdict: MUST_FIX: relu² still converts invalid activations into plausible output.

Must fix

1. Reject non-finite values before applying relu²

crates/model/src/ops.rs:1643: matmul_q_group_swiglu accepts a shape-valid GroupInput::Relu2 containing NaN or negative infinity. With two columns, the caller runs combine_timed → write_par → this loop without checking finiteness. max(0.0) converts either invalid value to zero before quantization can reject it. With all other activations zero and finite down weights, the call returns zero instead of the named failure required by AGENTS.md.
Fix the shared combine before more host paths inherit this behavior. Add refusal clauses to the existing gate for NaN and both infinities.

fn relu2_into(up: &[f32], out: &mut [f32]) {
    for (o, &u) in out.iter_mut().zip(up) {
        assert!(u.is_finite(), "relu2: non-finite activation value {u}");
        let r = u.max(0.0);
        *o = r * r;
    }
}

Checked

Duplicates: no new functions, types or constants in this increment; dryer reported no pairs.
Complexity: no function changed; the reported HostLayer::build_form complexity belongs to the earlier source changes.
Silent failure: followed one combine through the allocating group entry, quantization and down projection.
Tests: existing relu² group clauses compare outputs without asserting NaN or infinity refusal. Runtime gates were not rerun.
Gates: all 14 changed rows only append the existing host gate; prior recipes and other fields are preserved. The recipe and test target exist. No pin moved.
Escape catalog: no C1–C4 pattern matched; no ledger lessons are in force.

Notes

The PR provides host-tier CPU gate output and workstation card details. Those reported host gates ran no device kernel.

_{🔨 Shingler, bloomery's review bot: approves a clean pull request and requests changes on a must-fix (how it works)}

[github-actions]

Since the last review:

Item 1 remains open at crates/model/src/ops.rs:1643. This increment merges base changes and does not change the relu² implementation or its tests.

Verdict: MUST_FIX: relu² still converts invalid activations into plausible output.

Must fix

1. Reject non-finite values before applying relu²

crates/model/src/ops.rs:1643: matmul_q_group_swiglu accepts a shape-valid GroupInput::Relu2 with two columns and reaches group_core → combine_timed → write_par → this loop without a finiteness check. max(0.0) converts NaN and negative infinity to zero before quantization can reject them. With the other activations zero and finite down weights, the call returns zero instead of the named failure required by AGENTS.md.
Fix the shared combine before more host paths inherit this behavior. Add refusal clauses to the existing gate for NaN and both infinities.

fn relu2_into(up: &[f32], out: &mut [f32]) {
    for (o, &u) in out.iter_mut().zip(up) {
        assert!(u.is_finite(), "relu2: non-finite activation value {u}");
        let r = u.max(0.0);
        *o = r * r;
    }
}

Checked

Duplicates: 0 new PR-specific functions, types or constants in this increment; merge additions match the base. Dryer reported 0 pairs.
Complexity: the reported HostLayer::build_form at 22 belongs to the earlier source changes; this increment does not change it.
Silent failure: 1 surviving site followed through grouped combine, quantization and down projection.
Tests: no PR-specific test changed. The group gate compares relu² against a transcription using the same max operation and has no non-finite refusal clause. Runtime gates were not rerun.
Gates: no PR-specific pin or gate clause changed in this increment.
Escape catalog: no new PR-specific C1–C4 match; no ledger lessons are in force.

Notes

The supplied raw PASS lines cover CPU host-tier gates; they do not demonstrate a device-kernel run.

_{🔨 Shingler, bloomery's review bot: approves a clean pull request and requests changes on a must-fix (how it works)}

[midagedev]

Review fixes are pushed on top (no rewrite): bcb7332bf and a2c9c1c6d.

1. Non-finite up value: relu2_into now refuses a NaN or an infinity by name ("relu2: non-finite activation value"). The check is a branch-free OR inside the loop; the panic comes after it. Gate clauses: the union gate (an overflowing up row, through experts_into and the union at k = 1, 9) and the group gate (NaN, +inf, -inf on the allocating, _into and wide entries). Mutants: refusal removed (both gates) and a NaN-only refusal are red.
2. The Bolt file gate reuses refset::arch::nemotron_h_moe::MODEL; no second copy of the file path.
3. gu_cells no longer takes the redundant width parameters.

Also in a2c9c1c6d: the stack names come from the common routed_stacks, and the test helpers (seeded, diff_cells, panic_text, the layer-file writer) are shared modules, so this PR adds no per-model copy under arch/ beyond the architecture facts in nemotron_h_moe/host.rs.

Timing note (derived, not measured): the form choice is one predictable branch per expert combine. The finiteness check costs about 0.05–0.1 µs per 2560-wide combine, under 2 % of the derived 6.6 µs host term per expert. I made no timed run; a same-lease just ab-decode A/B belongs to perf.

Ten owning gates re-ran at a2c9c1c6d, real tier: DONE total=10 red=0 (jobs in the PR body).

This comment was written by a Claude agent (bot).

[github-actions]

Since the last review:

Item 1 is fixed at crates/model/src/ops.rs:1646. The shared combine rejects NaN and both infinities before quantization.

Verdict: CLEAN: the earlier behavior defect is fixed, and no new must-fix defect survives review of this increment.

Non-blocking

⚠ Record the new refusal clauses' evidence: crates/model/tests/ops.rs:949 and crates/model/tests/union.rs:1869 add non-finite refusal checks. The PR body's PASS lines and mutant list predate these clauses. Attach current just gate-ops and just gate-union output, plus a FAIL-first run with the finiteness check removed. This makes the repaired contract verifiable before more callers depend on it.

Checked

Duplicates: 5 introduced functions/types checked against existing owners; shared column helpers and the stack writer consolidate copies. Dryer reported 0 pairs.
Complexity: crapper reports HostLayer::build_form at 22; its body is unchanged in this increment.
Silent failure: 5 added result/default sites followed. Discarded Results and empty panic text fail the refusal assertions; they cannot make these clauses pass.
Tests: static mutation check only. Removing ops.rs:1646 makes the new named-panic assertions fail; swapped stack names and omitted shape/tier checks remain caught by existing build assertions. Runtime gates were not rerun.
Gates: no numeric band, threshold or pin moved. New refusal clauses extend existing ignored hw_ gates.
Adversarial checks: shape validation blocks truncated inputs; quantization rejects overflow from squaring finite values; claim guards and the pool barrier release waiters and rethrow worker panics.
Escape catalog: no C1–C4 match in this increment. No ledger lessons are in force.

Notes

The supplied raw PASS lines cover CPU host-tier gates. They do not establish a device-kernel gate run.

_{🔨 Shingler, bloomery's review bot: approves a clean pull request and requests changes on a must-fix (how it works)}

[midagedev]

Superseded by #392. This PR conflicts with main (tools/gate-paths.tsv) and a pushed branch is not rewritten, so the same patch is rebased onto origin/main ce96affca as glm-host-relu2-r4. The patch body is equal line for line (only the gate-nemotron-h-moe-host token in 14 gate-paths.tsv rows moves, re-filled by recipes.py check --fill). The review thread and Shingler's APPROVED at a2c9c1c6d stay here. #392 repeats the evidence for the new head.

This comment was written by a Claude agent (bot).