model: Bolt's plan places routed experts on the host tier (Experts::{Card,Placed,Host})

[midagedev]

What and why

Bolt's (nemotron_h_moe) plan can now put routed experts on the host tier. This is B2 of the offload row of #328. The common planner already makes that plan per layer. Only the family wrapper refused it.

hybrid.rs gains Experts::{Card, Placed, Host} and PlanInputs::plan_with. plan() is plan_with(.., Experts::Card): same plan, same error text. Placed keeps each routed layer's id prefix [0, n_l) on the card as its budget holds and the rest on the host. Host keeps none on the card. The context, tier and card-count refusals hold under every arm. The load still refuses host experts: the body has no host tier yet (B3).

Release notes: internal — the plan half of Bolt's --n-cpu-moe equivalent; no load uses it yet.

API for B3

• PlanInputs::plan_with(&machine, ctx, &levers, Experts::{Card, Placed, Host}), re-exported as nemotron_h_moe::place::Experts.
• Machine of a placed load: workstation::placed_on(RTX_3090, 88, workstation::SCRATCH + <the body's own scratch>). The body's scratch is the GPU side's to size. This PR counts none.
• The split is in the plan: plan.n_l[l]; each routed row is Card(0) ids [0, n_l) then Host/HostFile ids [n_l, E_l); plan.host_experts(l); plan.host.expert_bytes. Sum only over layers that have routed stacks: layer_experts(l) of any other layer is the model's largest count.
• No new lever. n_l follows from the bytes: BLOOMERY_CARD_BUDGET and --place gate already reach plan through PlanLevers.

Pinned facts (the real 24 GB header, ctx 4096, body scratch not counted)

• The 3090 holds all 13,824 routed experts: plan_gate(88) and placed_on(RTX_3090, 88, SCRATCH) both. Placed there is Debug-equal to Card.
• Half the card's expert bytes (budget B = 15,668,690,944 on the placed machine): 7,010 of 13,824 on the card, 6,814 on the host (8,549,646,336 B).
• 12 GiB card room (the 3060 host's card): 4,705 on the card (5,710,950,400 B), 9,119 on the host (11,345,461,248 B). Headroom 1,074,593,792 B. The per-layer sums equal the plan's totals.
• All on the host: 17,052,794,880 B.
• The spread rule gives each layer the same expert count, not the same fraction: at 12 GiB layer 1 holds 118 of 240 and layer 85 holds 117 of 496.

Gates and FAIL-first

Fixture clauses in nemotron_h_moe::spec (C1 to C4), nemotron_h::place (C5), and hw_nemotron_h_moe_plans_the_host_tier on the real header. The clauses of a split have one owner, tests/common/host_split.rs. Each mutant ran through tools/mutant-run.sh with --expect on the intended clause, then a clean run.

mutant	red
M1 swap the Placed and Host arms	C1, C2, C3
M2 the host refusal kept under every arm	C2, C3; hw alone
M3 placement.rs host experts counted at the model's largest layer	C2 (f), C1, C3, C4; hw alone
M4 Host calls plan_routed	C3
M5 placement.rs expert budget without the margin	C2 (a); hw alone
M6 the tier check only under Card	C4
M7 Host refused when nothing is routed	C5
M8 host segment bytes off by one	C2 (e)
M9 the down stack one expert short	C2 (c)

9 of 9 killed, clean runs green. M3, M5 and M8, M9 are temporary mutants of crates/placement; none is committed. Clause (d), the stacks' lists equal, is implied by (c): no mutant isolates it.

tools/gate-paths.tsv gets a row for crates/model/src/arch/hybrid.rs (second commit): before, narrow kept the full list (167 recipes, "no row maps it"); after, rows 111 -> 80 recipes (102 in all; the row holds 82 recipes since the merge of main). recipes.py check: 134 gate rows, rc 0.

GPU Evidence

Box: RTX 3090 and RTX A6000, driver 615.71.09, nvcc cuda_13.3, Linux 6.8.0-142. All jobs through the box queue.

Two trees. The jobs below ran at head 9b630364c on base 962401358 (origin/main then), except the lines marked [tip]. After them, origin/main moved to e195c9e6a; this PR merged it (e263e7f59) and fixed two call sites for main's Option<u64> set-aside (766b36916). The merge moved the hybrid.rs row of tools/gate-paths.tsv from 80 to 82 recipes. The 80-gate batch below ran on the pre-merge tree; the [tip] lines re-ran the four gates the merge could change.

[tip] at 766b36916, tools/gate-batch.sh --tier fixture --round-ledger, DONE total=4 red=2 wall=608s:

• gate-nemotron-h-moe-meta job s2246 rc 0: lib 15 passed, hw 2 passed.
• gate-nemotron-meta job s2251 rc 0: lib 36 passed, hw 1 passed.
• gate-qwen3-plan rc 101, not from this PR: error[E0308]: mismatched types at crates/gpu-gates/src/bin/shared/qwen3moe_place.rs:660 and :1364 (reserve_bytes() is Option<u64> since bd378cd6e; that file's reserves: line came with ea4b6228c). Both files are byte-identical to origin/main e195c9e6a.
• gate-gpu-qwen38-nodes-remote rc 101, not from this PR: Error: "BLOOMERY_TIER_NODES names 0 addresses, the gate one". The batch sets no remote node.

Second merge: origin/main 402f9abc6 (85410e2b4, plain push) brings #372 (686da9f16), which fixes the qwen3moe_place.rs compile error above. just check on the box at 85410e2b4: rc 0. gate-qwen3-plan and the two meta gates were not re-run at 85410e2b4; the row is still 82 recipes (recipes.py check rc 0).

• gate-nemotron-h-moe-meta job s2221 rc 0: lib 15 passed, hw 2 passed (hw_nemotron_h_moe_header, hw_nemotron_h_moe_plans_the_host_tier).
• gate-nemotron-meta job s2222 rc 0: lib 36 passed, hw 1 passed.
• 80-gate batch (the new row): tools/gate-batch.sh --list <the row> --tier fixture --round-ledger → DONE total=80 red=5 tier=fixture skipped=0 wall=5478s laneA=3243s laneB=3403s laneX=2073s deferred=120 real_only=20 real_file=16.
  • gate-gpu-nemotron-h-moe-e2e job s2065, A6000, rc 0: PASSED: gate_nemotron_h_moe_e2e the captured step holds the entries its layer kinds derive; eager and replay give bit-identical logits; a reset rewinds the ids and the logits bits; the scan state carries; the fault word stays clean; …
  • gate-gpu-nemotron-h-moe-serve job s2067, A6000, rc 0: PASSED: gate_nemotron_serve the seat plans its default and says so, refuses a second slot, loads at the context it named, …
• Nano on its real file (the fixture tier has no Nano fixture, rc 66 there): gate-gpu-nemotron-e2e job s2218, 3090, rc 0, PASSED: gate_nemotron_e2e …; gate-gpu-nemotron-serve job s2220, A6000, rc 0, PASSED: gate_nemotron_serve ….
• Red in the batch, not from this change:
  • gate-gpu-qwen4exp-e2e (s2063) and gate-gpu-qwen4exp-mtp (s2070), rc 1: card leg plan: … a layer's count Some(255)..Some(256) (want 256 …) FAIL. The same two lines fail on origin/main e195c9e6a with none of our commits (jobs s2219 and s2223).
  • gate-gpu-gemma-embedding2-e2e (s2073), rc 1: /models/embeddinggemma-2/embeddinggemma-2-Q8_0.gguf: No such file or directory. The box has no such file.
• Deferred by name in the fixture tier (20 real-only, 16 real-file; list in the batch's deferred.list): gate-gpu-p6, gate-gpu-e2e, gate-gpu-hybrid, gate-gpu-p8, gate-gpu-moe, gate-gpu-ds41-moe, gate-gpu-ds41-chain-ffn, gate-gpu-p10, gate-placement, gate-gpu-qwen3moe-router, gate-gpu-qwen35moe-verify, gate-gpu-qwen35moe-mtp, gate-gpu-qwen35-dspark, gate-gpu-minicpm-text, gate-gpu-clef-hidden, gate-gpu-qwen35-media, gate-gpu-qwen35moe-media, gate-gpu-clef-serve, gate-gpu-lev-serve, gate-gpu-qwen4exp-iqleg, gate-gpu-gemm, gate-gpu-gemma-embedding2-serve, gate-gpu-qwen3moe-e2e, gate-fixture, gate-gpu-head, gate-gpu-ds41-long, gate-gpu-ds41-draft, gate-gpu-ds41-chat, gate-gpu-nvtier, gate-gpu-qwen38-residency, gate-gpu-qwen3-serve, gate-gpu-p8b, gate-gpu-dspark-experts, gate-gpu-dspark-kv, gate-gpu-dspark-graph, gate-gpu-glm5next-serve.
• ptx-scan --no-jit of gate_glm5next_e2e, gate_kquant, gate_swap identical to the base (just narrow): no kernel moved.
• Static: cargo check -p bloomery-model --all-targets 0 warnings; clippy 12 warnings, all in other files; fmt-check and every tools/check-*.sh rc 0 except check-recipes.sh, where narrow-scan.sh --self-test (16 FAIL) fails the same way on origin/main's tools/.

Code shape

• No second copy: the split's clauses are one file (tests/common/host_split.rs) read by the fixture gate and the real-file gate. The placed machine has no wrapper here; qwen3moe's machine and counted_arena_bytes are a separate lift.
• Model code holds only architecture facts: hybrid.rs gains an enum and one function, no per-model branch. A file with no routed stacks plans the same under every arm (C5).
• No silent failure: Card keeps its named refusal of host experts. No comment carries a date or an issue number.
• No gate or lint was relaxed.

This PR was written by a bot (a Claude Sonnet 5.5 agent in the Claude Code harness) under the lead's direction.

Rebase onto main ce96aff (supersedes #371)

#371 had two merge commits and conflicted with main. This branch holds its three commits on main, no merge commit.

• tools/gate-paths.tsv: main's rows kept, the hybrid.rs row added; recipes.py check --fill added five gates newer recipes reach (gate-fixture-budget, gate-gpu-gemma-embedding-e2e, gate-gpu-gemma-embedding-serve, gate-gpu-qwen38-nodes-remote, gate-qwen3-plan); recipes.py check rc 0.
• The last commit's diff is byte-identical to 766b369 (md5 of both diffs equal).
• Box, gate-batch real tier --round-ledger: gate-placement rc 0, gate-nemotron-meta rc 0, gate-nemotron-h-moe-meta rc 0.
• Mac static (remote): combos rc 0; red only lint (68, the same count as main e0975c3) and check-recipes (tools/ref/dense-logits.py --self-test reads /proc on macOS, also red on main). Neither comes from this change.

Written by a bot (Claude agent) for the maintainer.

🤖 Generated with Claude Code

[github-actions]

Verdict: MUST_FIX: The new tests duplicate fixture and sizing facts and add an unnecessary forwarding module.

Must fix

1. Use the fixture helper for all fixture construction

crates/model/src/arch/nemotron_h_moe/spec.rs:608 repeats the setup at crates/model/src/arch/nemotron_h_moe/spec.rs:489 and crates/model/src/arch/nemotron_h_moe/spec.rs:564. This leaves three owners of the same header, tokenizer setting and read sequence (R14).
Reusing the helper now removes 20 lines before a fixture change needs three edits.

Replace the setup from let path = header_typed(...) through let inputs = ... in the_fixture_layout_plans_whole_card with:

let inputs = fixture_inputs("nhm-fixture");

Use this replacement in a_card_too_small_for_every_expert_is_refused:

let inputs = fixture_inputs("nhm-spill");

2. Give the partial-budget formula one owner

crates/model/src/arch/nemotron_h_moe/spec.rs:663 and crates/model/tests/nemotron_h_moe_meta.rs:424 copy the formula already at crates/model/src/arch/nemotron_h_moe/spec.rs:589. All three size the same dense floor plus half the expert bytes (R14).
A shared calculation removes more code than it adds and keeps future sizing changes in one place.

Add this to tests/common/host_split.rs; it uses the existing Card::floor_bytes owner:

/// The one card's budget for its dense floor and half its expert bytes.
pub fn half_expert_budget(plan: &Plan<'_>) -> u64 {
    let c = &plan.cards[0];
    let card = &plan.machine.cards[0];
    card.floor_bytes(
        c.dense_bytes + c.rounding_bytes + c.expert_bytes / 2,
        c.kv_bytes,
        c.stage_rows_bytes + card.margin_bytes,
    )
    .expect("bounded partial-plan budget")
}

In both fixture tests, replace the sizing setup after whole with:

let budget = super::host_split::half_expert_budget(&whole);
let mut small = machine;
small.cards[0].usable_bytes = budget;

In the real-file gate, replace let c = ... and the budget sum with:

let budget = host_split::half_expert_budget(card);

3. Remove the forwarding placement module

crates/model/tests/nemotron_h_moe_meta.rs:21 adds a module that only re-exports an existing module. A direct import supplies the same crate::placement path to the shared checker.
Removing it now avoids maintaining a second namespace with no separate behavior.

Delete the forwarding module and its comment. Replace the placement import with:

use model::placement::{self, Plan, PlanLevers, workstation};

4. Use one pin for the file's routed expert count

crates/model/tests/nemotron_h_moe_meta.rs:325 repeats the file total already pinned at crates/model/tests/nemotron_h_moe_meta.rs:263. Both tests describe the same model file (R14).
Using the new constant in both places prevents two independent count pins.

Replace the existing header-count condition with:

if u64::from(routed) != ROUTED || hp.n_expert != 496 {

Checked

Duplicates: 15 new functions, types and constants inspected; three duplicate facts and one unneeded module confirmed above.
Complexity: plan() delegates; plan_with() adds the expert-policy match after the existing guards.
Silent failure: cleanup errors do not supply plan values; missing expert lists fail the segment comparison; a refused Card plan still records a failure.
Tests: static inspection confirms assertions for context/tier/card-count guards, policy arms and per-layer split clauses. Gates were not run in this job.
Gates: no existing tolerance or pin was relaxed.
Escape catalog: C1–C4 do not match. No hot path changed.

Notes

GPU evidence lacks the required nvidia-smi output line and exact just gate-* command/output pairs.

_{🔨 Shingler, bloomery's review bot: approves a clean pull request and requests changes on a must-fix (how it works)}

[midagedev]

Merged for the 0.3.4 train. The four review items (fixture helper reuse in nemotron_h_moe/spec.rs, one owner for the partial-budget formula, the forwarding placement module, one pin for the routed expert count) are code-shape items; they go in a follow-up PR on main.

Written by a bot (Claude agent) for the maintainer.