Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
43 changes: 43 additions & 0 deletions EXPERIMENTS.md
Original file line number Diff line number Diff line change
Expand Up @@ -8909,6 +8909,49 @@ the hold.
Rule W7: this was the campaign's first hold, so every later wildct row runs release 0.1.89. Holds
used: 1 of at most 12. The next row in the registered order is H18w.

### Wildct amendment R7: H25w's CPU Fast Draft (2026-11-07)

Recorded 2026-11-07, after the H52w and H18w holds and before any H25w hold. Mechanics only. No
row's hypothesis, prediction, arm, model, statistic, threshold, cap, disk need, hold count or
order changes. No kernel module was loaded, and nothing on wildct changed.

**What was found.** The pre-registration's H25w line says that the agent manifest "keeps its
default CPU Fast Draft in both arms, as H25 run 4 did", and it stages the draft
(`OpenVINO/Qwen3-0.6B-int4-ov` @ `f864c610`) for that. Its command sets no draft variable. When
the line was written, the launcher drafted by default for every CPU target that serves an
OpenVINO Qwen3 model (#116, merged 2026-10-18). The model optimizer (#163) merged at
2026-10-21T05:59Z, three minutes before the pre-registration (#181). It narrowed that default:
the launcher now drafts on the CPU by default only where an H34-family row is SUPPORTED for the
host's CPU class (`agent_draft_measured` and `agent_ledger_rule H34` in `scripts/k3d-gpu`). No
H34-family row names the i7-13700 (H34w is PENDING). So on wildct, release 0.1.89's
`agent manifest` writes no draft, and the registered command would serve both arms without one.

This was read on 2026-11-07 from main's launcher at `28b1e70b`, which equals 0.1.89's apart from
the version line. The read used `K3D_GPU_SYSROOT`, with a `/proc/cpuinfo` that names the
i7-13700, `K3D_GPU_AGENT_BACKEND=ovms` and `K3D_GPU_AGENT_DEVICE=CPU`, as `run.sh` sets them:

- With no draft variable, the manifest has no `--draft_source_model` or `--draft_device` line,
and the launcher prints "agent: no CPU Fast Draft: no EXPERIMENTS.md H34-family row measured
13th Gen Intel(R) Core(TM) i7-13700 (K3D_GPU_AGENT_DRAFT=CPU forces it)".
- With `K3D_GPU_AGENT_DRAFT=CPU`, it writes `--draft_source_model=OpenVINO/Qwen3-0.6B-int4-ov`
and `--draft_device=CPU`. The manifest is then byte-identical to the default one rendered for
the i9-13900T, the host of H25 run 4.

H25 run 4 ran at `bcca7be`, which carries #116, and H25x served both arms with the same draft
(`bench/xhost/results/2026-10-20/H25x/hold/out/on.yaml`).

**Runner change.** `row.sh h25w` adds `K3D_GPU_AGENT_DRAFT=CPU` to H25w's environment, so the
command serves what the H25w line registers. It reaches both arms alike: `run.sh` renders one
manifest and derives OFF from it by the one `--enable_prefix_caching=false` line, which it still
checks. The draft is read from `<target>/OpenVINO-Qwen3-0.6B-int4-ov`, where the setup staged it
(`fetch_models.py --draft`), so nothing is downloaded in the pod. This adds no arm and gates
nothing. H34w is not affected: `h34.sh` writes its own graphs (A, the target alone; B, with the
draft). Test: `tests/wildct_hold_test.sh` (row.sh's H25w environment), in CI.

**Resume procedure.** Before H25w, step 3b ships main at this amendment's merge commit. Its
`scripts/k3d-gpu` must still equal 0.1.89's apart from the version line, or `hold.sh` refuses the
hold (exit 7), and its `share/` must equal 0.1.89's. Rule W7 keeps release 0.1.89.

### H5 amendment A1: stage-1 harness and definitions (2026-10-19)

Written 2026-10-19T22:30-07:00, before any H5 measurement. It fixes what the
Expand Down
7 changes: 5 additions & 2 deletions bench/wildct/row.sh
Original file line number Diff line number Diff line change
Expand Up @@ -8,7 +8,8 @@
# h52w H52w's procedure: bench/wildct/h52w.sh OUT (it sets its own cluster, opt-out
# and W3, and runs the installed release's default node image; no IMAGE)
# h18w H18's bench/sandbox_round.sh, runs 1-3
# h25w H25's bench/shared-prefix/run.sh
# h25w H25's bench/shared-prefix/run.sh, with the CPU Fast Draft the H25w line registers
# in both arms (K3D_GPU_AGENT_DRAFT=CPU, wildct amendment R7)
# h34w H34's bench/hetero-spec/h34.sh
# h28w-prep bench/zero-copy-models/probe.sh prep (typical's model images only, r0-r6)
# h28w bench/zero-copy-models/probe.sh run (its plain k3d cluster create gets W3 and the
Expand Down Expand Up @@ -60,8 +61,10 @@ case ${row} in
ARGV=(bash "${REPO}/bench/sandbox_round.sh") RUNS=3 ;;
h25w)
need_image
# K3D_GPU_AGENT_DRAFT=CPU: the launcher drafts by default only on a CPU class an H34-family
# row measured, and none names the i7-13700 (wildct amendment R7).
E=(IMAGE="${IMAGE}" MODELS="${W}/models" OUT="${OUT}" LEVELS=typical CLUSTER=k3dgpu-w-h25 SEED_BASE=2560 HOLD=1440
LOAD_CPUS=20-23 K3D_GPU_VENDORS="${OPTOUT}" K3D_GPU_K3S_ARGS="${W3}")
LOAD_CPUS=20-23 K3D_GPU_VENDORS="${OPTOUT}" K3D_GPU_K3S_ARGS="${W3}" K3D_GPU_AGENT_DRAFT=CPU)
ARGV=(bash "${REPO}/bench/shared-prefix/run.sh") ;;
h34w)
need_image
Expand Down
3 changes: 2 additions & 1 deletion tests/wildct_hold_test.sh
Original file line number Diff line number Diff line change
Expand Up @@ -594,8 +594,9 @@ row_envs_match_the_registration() {
local o=K3D_GPU_VENDORS=auto,-nvidia,-intel w3="K3D_GPU_K3S_ARGS=${W3_LEDGER}"
has smoke -- K3D_GPU_CLUSTER=k3dgpu-w-smoke "${o}" "${w3}" "K3D_GPU_IMAGE=${IMG}" || return 1
has h18w -- K3D_GPU_CLUSTER=k3dgpu-w-sbx "IMAGE=${IMG}" "${o}" "${w3}" || return 1
# H25w: the CPU Fast Draft its line registers, in both arms (wildct amendment R7).
has h25w -- CLUSTER=k3dgpu-w-h25 LEVELS=typical SEED_BASE=2560 HOLD=1440 LOAD_CPUS=20-23 "${o}" "${w3}" \
"MODELS=${HOME}/k3dgpu-bench-wildct/models" || return 1
"MODELS=${HOME}/k3dgpu-bench-wildct/models" K3D_GPU_AGENT_DRAFT=CPU || return 1
has h34w -- H34_CLUSTER=k3dgpu-w-h34 H34_VENDORS=auto,-nvidia,-intel H34_LEVELS=typical LOAD_CPUS=20-23 \
"K3D_GPU_IMAGE=${IMG}" "H22_MODELS=${HOME}/k3dgpu-bench-wildct/models" "${w3}" || return 1
has h28w-prep -- LEVELS=typical "IMAGE=${IMG}" || return 1
Expand Down
Loading