model: Bolt's plan places routed experts on the host tier (Experts::{Card,Placed,Host})

[midagedev]

What and why

Bolt's (nemotron_h_moe) plan can now put routed experts on the host tier. This is B2 of the offload row of #328. The common planner already makes that plan per layer. Only the family wrapper refused it.

hybrid.rs gains Experts::{Card, Placed, Host} and PlanInputs::plan_with. plan() is plan_with(.., Experts::Card): same plan, same error text. Placed keeps each routed layer's id prefix [0, n_l) on the card as its budget holds and the rest on the host. Host keeps none on the card. The context, tier and card-count refusals hold under every arm. The load still refuses host experts: the body has no host tier yet (B3).

Release notes: internal — the plan half of Bolt's --n-cpu-moe equivalent; no load uses it yet.

API for B3

• PlanInputs::plan_with(&machine, ctx, &levers, Experts::{Card, Placed, Host}), re-exported as nemotron_h_moe::place::Experts.
• Machine of a placed load: workstation::placed_on(RTX_3090, 88, workstation::SCRATCH + <the body's own scratch>). The body's scratch is the GPU side's to size. This PR counts none.
• The split is in the plan: plan.n_l[l]; each routed row is Card(0) ids [0, n_l) then Host/HostFile ids [n_l, E_l); plan.host_experts(l); plan.host.expert_bytes. Sum only over layers that have routed stacks: layer_experts(l) of any other layer is the model's largest count.
• No new lever. n_l follows from the bytes: BLOOMERY_CARD_BUDGET and --place gate already reach plan through PlanLevers.

Pinned facts (the real 24 GB header, ctx 4096, body scratch not counted)

• The 3090 holds all 13,824 routed experts: plan_gate(88) and placed_on(RTX_3090, 88, SCRATCH) both. Placed there is Debug-equal to Card.
• Half the card's expert bytes (budget B = 15,668,690,944 on the placed machine): 7,010 of 13,824 on the card, 6,814 on the host (8,549,646,336 B).
• 12 GiB card room (the 3060 host's card): 4,705 on the card (5,710,950,400 B), 9,119 on the host (11,345,461,248 B). Headroom 1,074,593,792 B. The per-layer sums equal the plan's totals.
• All on the host: 17,052,794,880 B.
• The spread rule gives each layer the same expert count, not the same fraction: at 12 GiB layer 1 holds 118 of 240 and layer 85 holds 117 of 496.

Gates and FAIL-first

Fixture clauses in nemotron_h_moe::spec (C1 to C4), nemotron_h::place (C5), and hw_nemotron_h_moe_plans_the_host_tier on the real header. The clauses of a split have one owner, tests/common/host_split.rs. Each mutant ran through tools/mutant-run.sh with --expect on the intended clause, then a clean run.

mutant	red
M1 swap the Placed and Host arms	C1, C2, C3
M2 the host refusal kept under every arm	C2, C3; hw alone
M3 placement.rs host experts counted at the model's largest layer	C2 (f), C1, C3, C4; hw alone
M4 Host calls plan_routed	C3
M5 placement.rs expert budget without the margin	C2 (a); hw alone
M6 the tier check only under Card	C4
M7 Host refused when nothing is routed	C5
M8 host segment bytes off by one	C2 (e)
M9 the down stack one expert short	C2 (c)

9 of 9 killed, clean runs green. M3, M5 and M8, M9 are temporary mutants of crates/placement; none is committed. Clause (d), the stacks' lists equal, is implied by (c): no mutant isolates it.

tools/gate-paths.tsv gets a row for crates/model/src/arch/hybrid.rs (second commit): before, narrow kept the full list (167 recipes, "no row maps it"); after, rows 111 -> 80 recipes (102 in all; the row holds 82 recipes since the merge of main). recipes.py check: 134 gate rows, rc 0.

GPU Evidence

Box: RTX 3090 and RTX A6000, driver 615.71.09, nvcc cuda_13.3, Linux 6.8.0-142. All jobs through the box queue.

Two trees. The jobs below ran at head 9b630364c on base 962401358 (origin/main then), except the lines marked [tip]. After them, origin/main moved to e195c9e6a; this PR merged it (e263e7f59) and fixed two call sites for main's Option<u64> set-aside (766b36916). The merge moved the hybrid.rs row of tools/gate-paths.tsv from 80 to 82 recipes. The 80-gate batch below ran on the pre-merge tree; the [tip] lines re-ran the four gates the merge could change.

[tip] at 766b36916, tools/gate-batch.sh --tier fixture --round-ledger, DONE total=4 red=2 wall=608s:

• gate-nemotron-h-moe-meta job s2246 rc 0: lib 15 passed, hw 2 passed.
• gate-nemotron-meta job s2251 rc 0: lib 36 passed, hw 1 passed.
• gate-qwen3-plan rc 101, not from this PR: error[E0308]: mismatched types at crates/gpu-gates/src/bin/shared/qwen3moe_place.rs:660 and :1364 (reserve_bytes() is Option<u64> since bd378cd6e; that file's reserves: line came with ea4b6228c). Both files are byte-identical to origin/main e195c9e6a.
• gate-gpu-qwen38-nodes-remote rc 101, not from this PR: Error: "BLOOMERY_TIER_NODES names 0 addresses, the gate one". The batch sets no remote node.

Second merge: origin/main 402f9abc6 (85410e2b4, plain push) brings #372 (686da9f16), which fixes the qwen3moe_place.rs compile error above. just check on the box at 85410e2b4: rc 0. gate-qwen3-plan and the two meta gates were not re-run at 85410e2b4; the row is still 82 recipes (recipes.py check rc 0).

• gate-nemotron-h-moe-meta job s2221 rc 0: lib 15 passed, hw 2 passed (hw_nemotron_h_moe_header, hw_nemotron_h_moe_plans_the_host_tier).
• gate-nemotron-meta job s2222 rc 0: lib 36 passed, hw 1 passed.
• 80-gate batch (the new row): tools/gate-batch.sh --list <the row> --tier fixture --round-ledger → DONE total=80 red=5 tier=fixture skipped=0 wall=5478s laneA=3243s laneB=3403s laneX=2073s deferred=120 real_only=20 real_file=16.
  • gate-gpu-nemotron-h-moe-e2e job s2065, A6000, rc 0: PASSED: gate_nemotron_h_moe_e2e the captured step holds the entries its layer kinds derive; eager and replay give bit-identical logits; a reset rewinds the ids and the logits bits; the scan state carries; the fault word stays clean; …
  • gate-gpu-nemotron-h-moe-serve job s2067, A6000, rc 0: PASSED: gate_nemotron_serve the seat plans its default and says so, refuses a second slot, loads at the context it named, …
• Nano on its real file (the fixture tier has no Nano fixture, rc 66 there): gate-gpu-nemotron-e2e job s2218, 3090, rc 0, PASSED: gate_nemotron_e2e …; gate-gpu-nemotron-serve job s2220, A6000, rc 0, PASSED: gate_nemotron_serve ….
• Red in the batch, not from this change:
  • gate-gpu-qwen4exp-e2e (s2063) and gate-gpu-qwen4exp-mtp (s2070), rc 1: card leg plan: … a layer's count Some(255)..Some(256) (want 256 …) FAIL. The same two lines fail on origin/main e195c9e6a with none of our commits (jobs s2219 and s2223).
  • gate-gpu-gemma-embedding2-e2e (s2073), rc 1: /models/embeddinggemma-2/embeddinggemma-2-Q8_0.gguf: No such file or directory. The box has no such file.
• Deferred by name in the fixture tier (20 real-only, 16 real-file; list in the batch's deferred.list): gate-gpu-p6, gate-gpu-e2e, gate-gpu-hybrid, gate-gpu-p8, gate-gpu-moe, gate-gpu-ds41-moe, gate-gpu-ds41-chain-ffn, gate-gpu-p10, gate-placement, gate-gpu-qwen3moe-router, gate-gpu-qwen35moe-verify, gate-gpu-qwen35moe-mtp, gate-gpu-qwen35-dspark, gate-gpu-minicpm-text, gate-gpu-clef-hidden, gate-gpu-qwen35-media, gate-gpu-qwen35moe-media, gate-gpu-clef-serve, gate-gpu-lev-serve, gate-gpu-qwen4exp-iqleg, gate-gpu-gemm, gate-gpu-gemma-embedding2-serve, gate-gpu-qwen3moe-e2e, gate-fixture, gate-gpu-head, gate-gpu-ds41-long, gate-gpu-ds41-draft, gate-gpu-ds41-chat, gate-gpu-nvtier, gate-gpu-qwen38-residency, gate-gpu-qwen3-serve, gate-gpu-p8b, gate-gpu-dspark-experts, gate-gpu-dspark-kv, gate-gpu-dspark-graph, gate-gpu-glm5next-serve.
• ptx-scan --no-jit of gate_glm5next_e2e, gate_kquant, gate_swap identical to the base (just narrow): no kernel moved.
• Static: cargo check -p bloomery-model --all-targets 0 warnings; clippy 12 warnings, all in other files; fmt-check and every tools/check-*.sh rc 0 except check-recipes.sh, where narrow-scan.sh --self-test (16 FAIL) fails the same way on origin/main's tools/.

Code shape

• No second copy: the split's clauses are one file (tests/common/host_split.rs) read by the fixture gate and the real-file gate. The placed machine has no wrapper here; qwen3moe's machine and counted_arena_bytes are a separate lift.
• Model code holds only architecture facts: hybrid.rs gains an enum and one function, no per-model branch. A file with no routed stacks plans the same under every arm (C5).
• No silent failure: Card keeps its named refusal of host experts. No comment carries a date or an issue number.
• No gate or lint was relaxed.

This PR was written by a bot (a Claude Sonnet 5.5 agent in the Claude Code harness) under the lead's direction.

🤖 Generated with Claude Code

[github-actions]

Verdict: MUST_FIX: Two new test owners leave the same fixture setup and expert-count pin duplicated.

Must fix

1. Use the fixture reader in the existing fixture tests

crates/model/src/arch/nemotron_h_moe/spec.rs:608 repeats the setup at crates/model/src/arch/nemotron_h_moe/spec.rs:489 and crates/model/src/arch/nemotron_h_moe/spec.rs:564: the same architecture, keys, tokenizer, tensors and read sequence (R14).
Reusing the helper now removes two setup blocks before the fixture changes.
Replace each header_typed-through-place::read setup with the following line, in test order:

let inputs = fixture_inputs("nhm-fixture");

let inputs = fixture_inputs("nhm-spill");

2. Use ROUTED in the existing header pin

crates/model/tests/nemotron_h_moe_meta.rs:325 introduces an owner for the Bolt file's total expert count, but crates/model/tests/nemotron_h_moe_meta.rs:263 keeps the same literal as a separate pin (R14).
Sharing it now prevents the header and placement gates from disagreeing after a fixture update.
Replace the existing condition with:

if u64::from(routed) != ROUTED || hp.n_expert != 496 {

Non-blocking

⚠ Check the aggregate host byte total: crates/model/tests/common/host_split.rs:132, a static trace shows that replacing expert_bytes: host_experts at crates/placement/src/placement.rs:3770 with expert_bytes: 0 escapes the new assertions. The real-file gate prints the total, and HostNeed::of uses it for sizing.
Add before the return: assert_eq!(plan.host.expert_bytes, layers.iter().map(|l| l.host_bytes).sum::<u64>());

Checked

Duplicates: both setup blocks match fixture_inputs; the header literal and ROUTED pin the same total. Both fixes reuse existing owners and add no elements.
Complexity: plan_with adds an enum dispatch without model-specific branches.
Silent failure: ignored fixture cleanup supplies no plan value; a missing expert list fails the segment comparison; show's discarded failures are checked immediately afterward; the Card-error fallback retains a failing pin.
Tests: static mutation traces at hybrid.rs:602 (Placed→Host) fail the whole-fit and small-card tests; :611 (Host→plan_routed) fails the Host and real-file tests; :589 (tier guard Card-only) fails the changed refusal test; a Host refusal at :610 fails the no-routed-stacks test.
Gates: no existing pin, case or gate was relaxed or removed.
Escape catalog: no C1–C4 pattern matches the diff.
Behavior: zero context, tier cards and wrong card counts are refused before dispatch; final invariants check card and host room. No decode or prefill work changes.

Notes

Static review only; gates were not rerun. GPU evidence omits the nvidia-smi output and exact just gate-* invocations.

_{🔨 Shingler, bloomery's review bot: approves a clean pull request and requests changes on a must-fix (how it works)}

[github-actions]

Since the last review: the three earlier items fall outside the incremental review scope.

Verdict: CLEAN: No in-scope must-fix items remain.

Checked

Evidence: read all three findings' cited code. The two duplication reports are kind B; the host-byte assertion proposal identifies a coverage gap without a reachable behavior failure.
Scope: none of the three finding paths appears in the supplied incremental diff. The re-review rule excludes these non-kind-A findings.
Diff: all 19 files in the incremental patch are byte-identical to BASE_TREE.
Validation: static inspection only; gates were not rerun.

_{🔨 Shingler, bloomery's review bot: approves a clean pull request and requests changes on a must-fix (how it works)}

[midagedev]

Superseded by #393: the same three commits rebased onto main ce96aff without the merge commits (gate-paths row refilled; owning gates green). I will close this PR when #393 merges.

Written by a bot (Claude agent) for the maintainer.

[midagedev]

Superseded by #393 (merged).

Written by a bot (Claude agent) for the maintainer.