From ac6c67b64ed94bbb8a8ba194d8d17f799a17f17d Mon Sep 17 00:00:00 2001 From: Andrew Mello Date: Sat, 7 Nov 2026 03:27:42 -0800 Subject: [PATCH 1/5] test(eat): the xpu refusal must not present k3d-gpu's vLLM on Arc as working eat-up.sh stops on compute_platform=xpu and sends the user to K3D_GPU_AGENT_BACKEND=vllm k3d-gpu agent up. No release up to 0.1.93 serves that path (#339 fixes the two causes), and with the fix it has not yet served end to end on an Arc B70. The check now also asks that the refusal say so, and that it no longer reads "for vLLM on an Intel GPU run". It fails on main: 171 passed, 1 failed. --- tests/eat_test.sh | 8 +++++++- 1 file changed, 7 insertions(+), 1 deletion(-) diff --git a/tests/eat_test.sh b/tests/eat_test.sh index 595f4855..9abba605 100755 --- a/tests/eat_test.sh +++ b/tests/eat_test.sh @@ -700,8 +700,14 @@ cfg="${WORK}/xpu-reason.cfg" cp "${EAT}/agentic-config.cfg" "${cfg}" echo 'compute_platform=xpu' >> "${cfg}" out=$(EAT_CONFIG="${cfg}" EAT_KUBECONFIG=/nonexistent "${EAT}/eat-up.sh" 2>&1) || true -if grep -qF 'K3D_GPU_AGENT_BACKEND=vllm' <<< "${out}"; then ok "eat-up: the xpu refusal points at vLLM on an Intel GPU through k3d-gpu agent up" +if grep -qF 'K3D_GPU_AGENT_BACKEND=vllm' <<< "${out}"; then ok "eat-up: the xpu refusal names k3d-gpu's vLLM on an Intel GPU" else fail "eat-up: the xpu refusal names no alternative: ${out}"; fi +# Releases through 0.1.93 never served that path (#339), and with the fix it has not yet +# served end to end on an Arc B70: the refusal says so instead of sending users there. +if grep -qF 'has not yet served end to end on Arc' <<< "${out}" \ + && ! grep -qF 'for vLLM on an Intel GPU run' <<< "${out}"; then + ok "eat-up: the xpu refusal says k3d-gpu's vLLM has not yet served end to end on Arc" +else fail "eat-up: the xpu refusal presents k3d-gpu's vLLM on an Intel GPU as working: ${out}"; fi # 5. The GenAI Gateway path installs pinned charts and images. EAT's # deploy-genai-gateway.yml syncs helm_charts_base_genai{,_trace} to the node, runs From 0f383560614319e6c8778795de8c345055ec7ac9 Mon Sep 17 00:00:00 2001 From: Andrew Mello Date: Sat, 7 Nov 2026 03:27:53 -0800 Subject: [PATCH 2/5] docs: the vLLM agent has not served in any release up to 0.1.93 No release since the vLLM agent shipped (0.1.21, 2026-10-16) up to 0.1.93 serves with vLLM. It was seen on hardware with the vllm backend on Arc; llm-scaler and the Gaudi image are built on vLLM 0.26 and 0.30, whose upstream source has the same two checks. The hardware canary for #326 found the causes on an Arc Pro B70: the Service named vllm gives the pod VLLM_PORT=tcp://..., on which vLLM exits at engine start, and the warm-up reaches vLLM with no Content-Type, which it answers with 400. The pinned v0.32.0 image exits on the first of these on the same host. #339 fixes both and is in no release yet. Several released texts presented the path as working. They now say what is true and link one status note in the README: - README: the Arc and Gaudi sections state the two causes, the fix, that the end-state proof on an Arc B70 is pending, and that llm-scaler and the Gaudi path have not run on hardware. The vllm table row and the Gaudi device row no longer read as serving. - docs/intel.md: the roadmap lists the vLLM paths under Planned, not Shipped; the engines row says no release up to 0.1.93 serves with vLLM. - docs/platforms.md and docs/intel/upstream-eat.md: vLLM XPU is no longer listed as already covered. - docs/quickstart.md and docs/first-run.md: quickstart and agent up do not serve on a Gaudi in those releases. - docs/fit.md: the fit's vLLM pick carries the same status. - integrations/eat/eat-up.sh and its README: the compute_platform=xpu refusal names K3D_GPU_AGENT_BACKEND=vllm and says it has not yet served end to end on Arc, instead of sending the user there. - canary/README.md: the runtime check runs the image under docker run, not in the launcher's pod. A new section says what that leaves out (the pod's Service variables, the launcher's warm-up request, and any serving of the XPU and Gaudi images, which are surface-only on hosted runners), and that both vLLM agent faults were of that kind. Checks: eat_test 172 passed, 0 failed (171/1 before this commit); detect_test 1252/0 (relative doc links resolve); claims_check and ledger_drift_check clean; platforms_doc_test 195/0; canary_spec_test 110/0; mkdocs build --strict clean; shellcheck 0.11.0 at warning severity clean. --- README.md | 32 +++++++++++++++++++++++++++----- canary/README.md | 18 +++++++++++++++++- docs/first-run.md | 7 ++++--- docs/fit.md | 6 ++++-- docs/intel.md | 25 ++++++++++++++++++------- docs/intel/upstream-eat.md | 2 +- docs/platforms.md | 10 +++++++--- docs/quickstart.md | 2 ++ integrations/eat/README.md | 7 +++++-- integrations/eat/eat-up.sh | 2 +- 10 files changed, 86 insertions(+), 25 deletions(-) diff --git a/README.md b/README.md index 2b8622a1..81c148e3 100644 --- a/README.md +++ b/README.md @@ -838,7 +838,7 @@ The device comes from what the cluster advertises, not from the host alone: | Cluster has | Device | Pod gets | |-------------|--------|----------| -| allocatable `habana.ai/gaudi` | `HPU` | one Gaudi, served by vLLM (see [vLLM on Gaudi](#vllm-on-gaudi)) | +| allocatable `habana.ai/gaudi` | `HPU` | one Gaudi, for vLLM, which no release up to 0.1.93 serves (see [vLLM on Gaudi](#vllm-on-gaudi)) | | allocatable `gpu.intel.com/xe`, else `gpu.intel.com/i915`, on a node labelled `gpu.intel.com/xmx=true`, skipping a driver labelled `k3d-gpu.io/xmx.=false` | `GPU` | one GPU, a `gpu.intel.com/xmx: "true"` node selector; the host render group is added as a supplemental group | | anything else, including an Intel GPU without XMX (or with `xmx` unknown) | `CPU` | no device; the P-cores on a hybrid node; on a node labelled `...cpu-cpuid.AMXTILE=true` it is pinned there | | `npu.intel.com/accel`, only with `K3D_GPU_AGENT_DEVICE=NPU` | `NPU` | the NPU and its accel node's group | @@ -977,7 +977,7 @@ OpenAI API at `/v1` from `svc/vllm`: | Backend | Image (pinned by digest) | Notes | |---------|--------------------------|-------| -| `vllm` | `vllm/vllm-openai-xpu:v0.32.0` | upstream vLLM for Intel GPUs (Data Center GPU and Arc, per [vLLM's XPU guide](https://github.com/vllm-project/vllm/blob/v0.32.0/docs/getting_started/installation/gpu.xpu.inc.md)); v0.32.0 not yet run here on an Arc B-series card | +| `vllm` | `vllm/vllm-openai-xpu:v0.32.0` | upstream vLLM for Intel GPUs (Data Center GPU and Arc, per [vLLM's XPU guide](https://github.com/vllm-project/vllm/blob/v0.32.0/docs/getting_started/installation/gpu.xpu.inc.md)); not yet served here on an Arc B-series card (status below) | | `llm-scaler` | `intel/llm-scaler-vllm:1.5` (vLLM 0.26.0-b2) | Intel's build for Arc B60/B70 | ```bash @@ -1000,8 +1000,27 @@ other backend first so its GPU is free; `agent down` and `agent status` cover bo The chat UI below and other clients that use `svc/ovms` at `/v3` need `http://vllm.k3d-gpu-agent.svc:8000/v1` instead. -Not yet run on Arc hardware here: serving end to end and tensor parallelism -above 1 (`CCL_ZE_IPC_EXCHANGE=pidfd`; oneCCL calls `prctl(PR_SET_PTRACER, getppid())`, +**Status: no release up to 0.1.93 serves with vLLM.** In every release since these +backends and the Gaudi path below shipped (0.1.21, 2026-10-16), `agent up` with vLLM +does not serve, and neither does `agent up --model` when the fit picks vLLM. The +hardware canary for the `v0.33.0` bump ([#326](https://github.com/88plug/k3d-gpu/pull/326)) +found two causes on an Arc Pro B70 on 2026-11-07; the pinned `v0.32.0` image exits there +on the first one too: + +- The Service is named `vllm`, so Kubernetes gives the pod + `VLLM_PORT=tcp://:8000`. vLLM reads `VLLM_PORT` as its own port and exits + at engine start ("appears to be a URI"), and the pod crash-loops. +- The warm-up posts through the API server's service proxy with no `Content-Type`, and + vLLM answers such a request with 400. + +llm-scaler and the Gaudi image are built on vLLM 0.26 and 0.30, whose upstream source +has both checks. [#339](https://github.com/88plug/k3d-gpu/pull/339) fixes both: the pod +spec sets `enableServiceLinks: false`, and the warm-up runs curl inside the pod with +`Content-Type: application/json`. No release up to 0.1.93 includes it. With the fix, +the end-state proof on an Arc B70 (pod Ready, the warm-up answered, a correct chat +answer, through the launcher alone) is pending, and llm-scaler and the Gaudi path have +not run on hardware. Tensor parallelism above 1 has not run on Arc hardware either +(`CCL_ZE_IPC_EXCHANGE=pidfd`; oneCCL calls `prctl(PR_SET_PTRACER, getppid())`, so sibling workers should need no `CAP_SYS_PTRACE` under yama `ptrace_scope=1`). | Variable | Default | Description | @@ -1039,7 +1058,10 @@ for tensor parallelism and Qwen models, `SYS_NICE`, an 8 GiB memory-backed bounds the HPU warm-up, which runs on every start and takes minutes; `K3D_GPU_AGENT_TIMEOUT` (1800 s) bounds it. -Not yet run on Gaudi hardware here: serving end to end. The device plugin runs +Not yet run on Gaudi hardware here: serving end to end. No release up to 0.1.93 serves +it: the Gaudi pod shares the Arc pod spec and warm-up, and with them the +[two faults above](#vllm-backends-for-discrete-arc-battlemage) that +[#339](https://github.com/88plug/k3d-gpu/pull/339) fixes. The device plugin runs without Intel's `habana-container-runtime` hook (see `share/gaudi-device-plugin.yml`), so whether vLLM opens the HPU with only the plugin's device nodes and environment is unverified. vllm-gaudi validates `Qwen3-0.6B`, not `Qwen3-4B`, and tensor diff --git a/canary/README.md b/canary/README.md index bf00314f..40be9be1 100644 --- a/canary/README.md +++ b/canary/README.md @@ -13,10 +13,26 @@ secrets. | Kind | Engines | What it proves | |---|---|---| -| runtime | `vllm-cpu`, `ovms-cpu`, `llama-cpu`; `eat-vllm-cpu` (informational) | The pinned image serves a pinned Qwen3-0.6B the way the launcher runs it. It returns one completion (HTTP 200, `finish_reason` stop or length, tokens > 0, the text matches `/Paris/`) and one stream (at least 2 chunks, then `[DONE]`), and the server reports the version its tag names. | +| runtime | `vllm-cpu`, `ovms-cpu`, `llama-cpu`; `eat-vllm-cpu` (informational) | The pinned image serves a pinned Qwen3-0.6B under `docker run`, with the arguments and environment `engines.toml` gives it (the launcher's, or the fit's for vLLM on the CPU), not in the launcher's pod. It returns one completion (HTTP 200, `finish_reason` stop or length, tokens > 0, the text matches `/Paris/`) and one stream (at least 2 chunks, then `[DONE]`), and the server reports the version its tag names. | | surface | `vllm-xpu`, `llm-scaler`, `gaudi-vllm`, `llama-sycl`, `llama-vulkan`, `llama-full`, `dlstreamer`, `sglang-xeon`, `npu-umd`, `ovms` (and the runtime engines first) | Every flag and env name the launcher and `console/model_opt.py` emit (`canary/flags.sh`) exists in the image, and the constants derived from it still match: `console/plan.py`'s vLLM versions, and the NPU driver and firmware date. No device is needed. | | node | `node` | The root Dockerfile builds. `k3d cluster create --image` it with the k3d `packaging/install.sh` pins and the arguments the launcher passes on a host without NVIDIA (no load balancer, no traefik, the node labelled `nvidia.com/gpu.present=false`). Within 5 minutes the node is Ready, every baked manifest is in place, every DaemonSet they define exists, and every pod in every namespace (kube-system, NFD, the device plugins) is Running and ready or Succeeded; then the pods hold for 20 s with no container restarting. A plugin that needs hardware gets no pod on a hosted runner (its node selector), so a pod still Pending at the timeout, such as one whose image does not pull, fails. | +## What it does not cover + +The runtime check runs the image, not the launcher's pod. Kubernetes adds nothing to its +environment (no Service link variables such as `_PORT=tcp://...`), and +`probe.py` sends its own requests with `Content-Type: application/json`, not the +launcher's warm-up. A fault in the pod around the engine, or in how the launcher asks +it, therefore passes. The two faults that kept the launcher's vLLM agent from serving in +every release up to 0.1.93 were of that kind: its Service, named `vllm`, gave the pod +`VLLM_PORT=tcp://...`, on which vLLM exits, and its warm-up reached vLLM with no +`Content-Type`, which vLLM answers with 400 +([#339](https://github.com/88plug/k3d-gpu/pull/339)). The images that agent serves +(`vllm-xpu`, `llm-scaler`, `gaudi-vllm`) are surface engines only: a hosted runner has +no Intel GPU or Gaudi, so the canary never serves them. `tests/detect_test.sh` checks +the rendered pod spec and the warm-up command against stubs; nothing in CI serves +through them. + ## Files | File | Role | diff --git a/docs/first-run.md b/docs/first-run.md index 3fb78fca..1ca5be29 100644 --- a/docs/first-run.md +++ b/docs/first-run.md @@ -160,9 +160,10 @@ Predicted: 20 to 40 min. What differs: **the LLM runs on the CPU, not the NVIDIA GPU.** The cluster advertises `nvidia.com/gpu` for your own pods, but `agent up` and quickstart -serve on Gaudi, Intel XMX GPUs or the CPU only. To serve an LLM on the NVIDIA -GPU, see [GPUStack](integrations/gpustack.md) Recipe A; it is not verified on a -k3d-gpu cluster. +serve on Intel XMX GPUs or the CPU only (their Gaudi path, vLLM, does not serve in +any release up to 0.1.93: [status](../README.md#vllm-on-gaudi)). To serve an LLM on +the NVIDIA GPU, see [GPUStack](integrations/gpustack.md) Recipe A; it is not verified +on a k3d-gpu cluster. Predicted: 15 to 30 min. The single measured run (H9, 303 s) was on a host of this class, with the LLM on 16 P-cores. diff --git a/docs/fit.md b/docs/fit.md index e9def9cf..7fdf2df7 100644 --- a/docs/fit.md +++ b/docs/fit.md @@ -199,8 +199,10 @@ and `K3D_GPU_AGENT_GPUS` are inputs the fit checks, not values that replace it. or, with it unset, an OpenVINO IR): an OVMS pod holds one GPU, and `fit --gpus 2` with OVMS is refused as "OVMS serves one GPU". With `K3D_GPU_AGENT_BACKEND` unset, the engine the fit picks serves the model: OVMS, vLLM or -llama-server. With `--model`, the model optimizer first picks the variant for the default -engine (OVMS: Gemma 4's OpenVINO IR, for example). When that engine has nothing to serve, +llama-server (vLLM does not serve in any release up to 0.1.93: +[status](../README.md#vllm-backends-for-discrete-arc-battlemage)). With `--model`, the +model optimizer first picks the variant for the default engine (OVMS: Gemma 4's OpenVINO +IR, for example). When that engine has nothing to serve, as with Qwen3.8-Flash-Next, the id is served as given, by the engine its fit picks. When the fit then picks another engine, the optimizer's choices are made again for that engine, pinned to the model the fit sized. A backend that is set and cannot serve the fit is diff --git a/docs/intel.md b/docs/intel.md index 33f68d29..5869dddf 100644 --- a/docs/intel.md +++ b/docs/intel.md @@ -235,7 +235,7 @@ and installs through EAS. | OS | Ubuntu 22.04 / 24.04, x86_64 | Any Linux with Docker that runs k3d; `host-setup` prints pinned driver steps for Ubuntu 24.04/26.04, Fedora, Arch and Manjaro | | Cluster setup | Kubespray onto the host, or brownfield on an existing cluster; the `ei-*-eligible` labels come only from the Kubespray path | One `k3d-gpu up`; `K3D_GPU_NODE_LABELS` sets EAT's labels, `K3D_GPU_BALLOONS=external` hands the NRI policy to EAT [ledger:H14] | | EAT 2.0 install | EAS `es_auto_installer.sh install agent-toolkit`, onto Kubespray or an existing cluster (`existing_kubernetes`) | `k3d-gpu eat up` (`eas up --agent-toolkit`): the existing-cluster mode at the GA tags (EAS `v1.2.0`, EAT `release-2.0.0`), with the k3d settings and patches chosen from the cluster ([intel/eas.md](intel/eas.md)). F5 checks it on the Xeon 6741P. F3 (EAS `main`) is REFUTED ([run](../EXPERIMENTS.md#f3-run-i9-13900t-2026-11-01)): it installed, checked and tore down, and the secret scan found the Keycloak admin password on a command line | -| Engines | vLLM CPU (`vllm/vllm-openai-cpu:v0.24.0`, bf16), 9 model profiles in the single-node table (7 chat, 1 embedding, 1 reranker). An OVMS Helm chart ships but no default path deploys it: `openvino/model_server:2025.4`, plus an unpinned `latest-py` tag for its model-export containers | `agent up` picks per host: vLLM on a Gaudi, OVMS 2026.4.1 on an XMX GPU, else OVMS on the P-cores, or, on a CPU class where an engine-choice row is SUPPORTED (the Xeon 6741P today), llama.cpp with the model's int4 GGUF (for Gemma 4 26B-A4B, the one model measured, Google's QAT Q4_0 GGUF and QAT MTP drafter) [ledger:H59]; vLLM XPU (`v0.32.0`) or llm-scaler on Arc by flag. Its images are pinned by digest in `images.lock` | +| Engines | vLLM CPU (`vllm/vllm-openai-cpu:v0.24.0`, bf16), 9 model profiles in the single-node table (7 chat, 1 embedding, 1 reranker). An OVMS Helm chart ships but no default path deploys it: `openvino/model_server:2025.4`, plus an unpinned `latest-py` tag for its model-export containers | `agent up` picks per host: vLLM on a Gaudi, OVMS 2026.4.1 on an XMX GPU, else OVMS on the P-cores, or, on a CPU class where an engine-choice row is SUPPORTED (the Xeon 6741P today), llama.cpp with the model's int4 GGUF (for Gemma 4 26B-A4B, the one model measured, Google's QAT Q4_0 GGUF and QAT MTP drafter) [ledger:H59]; vLLM XPU (`v0.32.0`) or llm-scaler on Arc by flag. No release up to 0.1.93 serves with vLLM, on a Gaudi or on Arc ([status](../README.md#vllm-backends-for-discrete-arc-battlemage)). Its images are pinned by digest in `images.lock` | | Accelerators | None requested: no `gpu.intel.com`, `habana.ai` or `/dev/dri` on the default path | Intel GPU, NPU, Gaudi, DSA, IAA and QAT plugins started per detected device, and `up` checks that each is schedulable (on the Xeon 6741P it warned when QAT advertised none [ledger:H52b]) | | CPU placement | NRI balloons sized for Xeon NUMA; on a hybrid CPU EAT's reservation needs the patch in [`contrib/eat/`](../contrib/eat) | Balloons baked for hybrid P/E and multi-NUMA hosts; vLLM on P-cores and agent services on E-cores through `contrib/eat/` values | @@ -511,12 +511,14 @@ has not run on that host yet. Shipped since this roadmap was first written: -- **Agent endpoint.** `k3d-gpu agent up`: an OpenAI-compatible server that picks vLLM on - a Gaudi, OVMS on an XMX Intel GPU, else OVMS on the P-cores, from what the node - advertises; vLLM XPU or llm-scaler on Arc by flag ([README](../README.md)). On a CPU - class where an engine-choice row is SUPPORTED, llama.cpp with the model's int4 GGUF takes - the CPU instead: the Xeon 6741P class since 2026-11-01, measured on Gemma 4 26B-A4B with - the QAT GGUF and MTP drafter only [ledger:H59]. +- **Agent endpoint.** `k3d-gpu agent up`: an OpenAI-compatible server that picks OVMS on + an XMX Intel GPU, else OVMS on the P-cores, from what the node advertises + ([README](../README.md)). On a CPU class where an engine-choice row is SUPPORTED, + llama.cpp with the model's int4 GGUF takes the CPU instead: the Xeon 6741P class since + 2026-11-01, measured on Gemma 4 26B-A4B with the QAT GGUF and MTP drafter only + [ledger:H59]. Its vLLM paths (vLLM on a Gaudi by default, vLLM XPU or llm-scaler on Arc + by flag) are in the launcher but do not serve in any release up to 0.1.93; see the + vLLM agent item under Planned below. - **EAS and EAT 2.0.** `k3d-gpu eas up`, `status`, `check` and `down` (and `k3d-gpu eat`, the agent-toolkit layer) drive Intel's EAS installer in its bring-your-own-cluster mode, at the GA tags of 2026-11-06, with every k3d setting and patch chosen from the cluster @@ -562,6 +564,15 @@ Shipped since this roadmap was first written: Planned, not shipped. Each item lands with its own test and, where it touches hardware, a recorded run. +- **vLLM agent.** In releases 0.1.21 through 0.1.93, `agent up` with vLLM does not serve, + on Arc or on a Gaudi. The hardware canary for the `v0.33.0` bump found why on an Arc Pro + B70 on 2026-11-07. The agent's Service is named `vllm`, so Kubernetes gives the pod + `VLLM_PORT=tcp://...`; vLLM reads that as its own port and exits. The warm-up request + carries no `Content-Type`, and vLLM answers it with 400 + ([README](../README.md#vllm-backends-for-discrete-arc-battlemage)). + [#339](https://github.com/88plug/k3d-gpu/pull/339) fixes both; no release up to 0.1.93 + includes it. The end-state proof on an Arc B70 is pending. llm-scaler and the Gaudi + path have not run on hardware. - **Hardware coverage.** Move the `xe`, NPU, Gaudi, DSA, IAA, QAT and AMX rows to verified on real Core Ultra, Arc, Gaudi 3 and Xeon hosts. The campaign on a Xeon 6741P with 4x Arc Pro B70 closed on 2026-11-01 ([above](#eat-by-default-vs-with-k3d-gpu)). Its diff --git a/docs/intel/upstream-eat.md b/docs/intel/upstream-eat.md index 3eb5a985..e711bbf8 100644 --- a/docs/intel/upstream-eat.md +++ b/docs/intel/upstream-eat.md @@ -75,7 +75,7 @@ cluster, read from source only: | #21 adds | Already covered by k3d-gpu | Expected on a k3d-gpu cluster | |---|---|---| -| Arc B60/B70 XPU path (`compute_platform=xpu`, `vllm-openai-xpu:v0.28.0`, TP4 `gemma-4-31B-it`) | B70 detection, `gpu.intel.com/xe` asserted by `up`, vLLM XPU `v0.32.0` by digest through `agent up` | Host Level Zero and OpenCL libraries mounted by `hostPath` resolve inside the k3d node container, not on the host | +| Arc B60/B70 XPU path (`compute_platform=xpu`, `vllm-openai-xpu:v0.28.0`, TP4 `gemma-4-31B-it`) | B70 detection, `gpu.intel.com/xe` asserted by `up`, vLLM XPU `v0.32.0` by digest through `agent up`, which no release up to 0.1.93 serves ([status](../../README.md#vllm-backends-for-discrete-arc-battlemage)) | Host Level Zero and OpenCL libraries mounted by `hostPath` resolve inside the k3d node container, not on the host | | Intel GPU plugin from intel-device-plugins `main` (`intel_gpu_plugin_ref`, unpinned) | Plugin `0.37.1` baked into the node image | Two GPU plugin DaemonSets on one node | | NUMA-aware NRI balloons, one `BalloonsPolicy` per node | Baked balloons for hybrid P/E and multi-NUMA hosts; `K3D_GPU_BALLOONS=external` [ledger:H14] | Two policies if `deploy_nri_balloon_policy=yes`; `eat-up.sh` sets `off` | | Kata sandboxes (`kata-fc`, `kata-qemu`) | `doctor --perf` reports `/dev/kvm` on the host | Kata needs `/dev/kvm` inside the k3d node, which `up` does not pass through | diff --git a/docs/platforms.md b/docs/platforms.md index 9d94c161..6ab61d07 100644 --- a/docs/platforms.md +++ b/docs/platforms.md @@ -394,12 +394,16 @@ B70 runs land: `/usr/lib/x86_64-linux-gnu/intel-opencl`). - What k3d-gpu already covers: B70 detection and the Intel GPU plugin `0.37.1`, baked and pinned; `gpu.intel.com/xe` advertised and asserted by `up`; the label through - `K3D_GPU_NODE_LABELS`; vLLM XPU through `K3D_GPU_AGENT_BACKEND=vllm` (`v0.32.0`, by digest). + `K3D_GPU_NODE_LABELS`. Not covered yet: vLLM XPU through `K3D_GPU_AGENT_BACKEND=vllm` + (`v0.32.0`, by digest) is in the launcher, but no release up to 0.1.93 serves with it + ([status](../README.md#vllm-backends-for-discrete-arc-battlemage)). The fix is + [#339](https://github.com/88plug/k3d-gpu/pull/339), and the end-state proof on an Arc + B70 is pending. - Expected collision: a second GPU plugin. `deploy-intel-gpu-plugin.yml` applies `deployments/gpu_plugin/base/intel-gpu-plugin.yaml` from intel-device-plugins `main` (`intel_gpu_plugin_ref` defaults to `main`, unpinned) beside the baked DaemonSet. - `eat-up.sh` therefore stops on `compute_platform=xpu` and points at - `K3D_GPU_AGENT_BACKEND=vllm`. + `eat-up.sh` therefore stops on `compute_platform=xpu`. Its message names + `K3D_GPU_AGENT_BACKEND=vllm` and says that path has not yet served end to end on Arc. - Expected collision: a second balloons policy. The reworked `nri_cpu_balloons` role renders one NUMA-sized `BalloonsPolicy` per node (`node.`). #21 still installs it only on `deploy_nri_balloon_policy == "yes"` (`fresh-install.sh` line 247), so diff --git a/docs/quickstart.md b/docs/quickstart.md index 16869603..9afa196b 100644 --- a/docs/quickstart.md +++ b/docs/quickstart.md @@ -67,6 +67,8 @@ You see five steps, each with a progress bar: used for this LLM, and neither is an iGPU without XMX (UHD, Iris Xe, N-series). On a Gaudi the server is vLLM instead, and the chat UI step needs `--no-ui` (the UI talks to OpenVINO Model Server only). + No release up to 0.1.93 serves with vLLM, so on a Gaudi this step does not + finish ([status](../README.md#vllm-on-gaudi)). 4. **Chat UI**: Open WebUI, with an admin account created for you. 5. **Port-forward**: the UI on `127.0.0.1` only, not on your network. diff --git a/integrations/eat/README.md b/integrations/eat/README.md index c3d42fac..0ad43d53 100644 --- a/integrations/eat/README.md +++ b/integrations/eat/README.md @@ -301,8 +301,11 @@ except where an item says otherwise, and item 11 is new there. In stops before touching the cluster, reading each key as `read_config_file` does, on `compute_platform=xpu` (fresh-install.sh:219 applies intel-device-plugins' `gpu_plugin` from `main`, unpinned, beside the plugin - k3d-gpu bakes; vLLM on an Intel GPU is `K3D_GPU_AGENT_BACKEND=vllm k3d-gpu - agent up`), `deploy_kata_containers=on` (Kata needs `/dev/kvm`, which a k3d + k3d-gpu bakes; the message names k3d-gpu's vLLM on an Intel GPU, + `K3D_GPU_AGENT_BACKEND=vllm k3d-gpu agent up`, and says it has not yet served end + to end on Arc: no release up to 0.1.93 serves with it, see + [the status](../../README.md#vllm-backends-for-discrete-arc-battlemage)), + `deploy_kata_containers=on` (Kata needs `/dev/kvm`, which a k3d node lacks) and `rack_scale_mode=on` (it builds its own cluster from `rack-profile.yaml`: netplan, BGP, the NVIDIA GPU operator). None of these has run here; each is read from source. diff --git a/integrations/eat/eat-up.sh b/integrations/eat/eat-up.sh index 4e5ed978..c026b047 100755 --- a/integrations/eat/eat-up.sh +++ b/integrations/eat/eat-up.sh @@ -82,7 +82,7 @@ case "$(cfg_value "${EAT_CONFIG}" deploy_nri_balloon_policy)" in *) err "${EAT_CONFIG}: deploy_nri_balloon_policy must be off: ${EAT_TAG} deploys its own NRI balloons policy when it is on or absent, beside k3d-gpu's, which must remain the only one, and in model mode it moves vLLM to a balloon type k3d-gpu's policy does not define" ;; esac case "$(cfg_value "${EAT_CONFIG}" compute_platform)" in - x|xpu) err "compute_platform=xpu: ${EAT_TAG} then applies intel-device-plugins' gpu_plugin from its main branch, unpinned, beside the GPU plugin k3d-gpu bakes; for vLLM on an Intel GPU run K3D_GPU_AGENT_BACKEND=vllm k3d-gpu agent up" ;; + x|xpu) err "compute_platform=xpu: ${EAT_TAG} then applies intel-device-plugins' gpu_plugin from its main branch, unpinned, beside the GPU plugin k3d-gpu bakes. k3d-gpu's own vLLM on an Intel GPU (K3D_GPU_AGENT_BACKEND=vllm k3d-gpu agent up) has not yet served end to end on Arc: see README.md, vLLM backends for discrete Arc" ;; esac case "$(cfg_value "${EAT_CONFIG}" deploy_kata_containers)" in on|yes) err "deploy_kata_containers=on: Kata needs /dev/kvm on the node, and a k3d node is a container without it; set deploy_kata_containers=off" ;; From 17db162d34176410c5c0f2b8a98600cd59ea9f5c Mon Sep 17 00:00:00 2001 From: Andrew Mello Date: Sat, 7 Nov 2026 22:40:04 -0800 Subject: [PATCH 3/5] test(eat): the xpu refusal sends users to k3d-gpu's vLLM from release 0.1.94 Release 0.1.94 carries #339 and is the first whose vLLM agent serves. On 2026-11-08 it served end to end on an Arc Pro B70, unpatched. The xpu refusal in eat-up.sh should therefore send users to K3D_GPU_AGENT_BACKEND=vllm k3d-gpu agent up again, name 0.1.94 as the first release that serves, and no longer say the path has not yet served. Before the eat-up.sh change: 171 passed, 1 failed. --- tests/eat_test.sh | 14 ++++++++------ 1 file changed, 8 insertions(+), 6 deletions(-) diff --git a/tests/eat_test.sh b/tests/eat_test.sh index 9abba605..113702e6 100755 --- a/tests/eat_test.sh +++ b/tests/eat_test.sh @@ -702,12 +702,14 @@ echo 'compute_platform=xpu' >> "${cfg}" out=$(EAT_CONFIG="${cfg}" EAT_KUBECONFIG=/nonexistent "${EAT}/eat-up.sh" 2>&1) || true if grep -qF 'K3D_GPU_AGENT_BACKEND=vllm' <<< "${out}"; then ok "eat-up: the xpu refusal names k3d-gpu's vLLM on an Intel GPU" else fail "eat-up: the xpu refusal names no alternative: ${out}"; fi -# Releases through 0.1.93 never served that path (#339), and with the fix it has not yet -# served end to end on an Arc B70: the refusal says so instead of sending users there. -if grep -qF 'has not yet served end to end on Arc' <<< "${out}" \ - && ! grep -qF 'for vLLM on an Intel GPU run' <<< "${out}"; then - ok "eat-up: the xpu refusal says k3d-gpu's vLLM has not yet served end to end on Arc" -else fail "eat-up: the xpu refusal presents k3d-gpu's vLLM on an Intel GPU as working: ${out}"; fi +# Releases through 0.1.93 never served that path. Release 0.1.94 (#339) is the first that +# does, and it served end to end on an Arc Pro B70: the refusal sends users there and +# names that release, since an older launcher does not serve. +if grep -qF 'K3D_GPU_AGENT_BACKEND=vllm k3d-gpu agent up' <<< "${out}" \ + && grep -qF 'k3d-gpu 0.1.94 or later' <<< "${out}" \ + && ! grep -qF 'has not yet served' <<< "${out}"; then + ok "eat-up: the xpu refusal sends users to k3d-gpu's vLLM on an Intel GPU, from release 0.1.94" +else fail "eat-up: the xpu refusal does not send users to k3d-gpu's vLLM from release 0.1.94: ${out}"; fi # 5. The GenAI Gateway path installs pinned charts and images. EAT's # deploy-genai-gateway.yml syncs helm_charts_base_genai{,_trace} to the node, runs From 3489fbe3dc91b471e9096bebeced4f35c8e1740c Mon Sep 17 00:00:00 2001 From: Andrew Mello Date: Sat, 7 Nov 2026 22:44:05 -0800 Subject: [PATCH 4/5] bench(vllm-agent): the unpatched end-state runs on an Arc Pro B70 (2026-11-08) The record of the hold on crescent (Xeon 6741P, 4x Arc Pro B70, xe, kernel 7.0.0-31, Ubuntu 26.04.1) that ran K3D_GPU_AGENT_BACKEND=vllm k3d-gpu agent up through the launcher alone, with no kubectl patch and no warm-up sent by hand: - release 0.1.94, installed by packaging/install.sh from main at e5f55b29 (k3d-gpu_0.1.94-1_all.deb, sha256 60a1dc5b...), with vllm/vllm-openai-xpu:v0.32.0@sha256:20509fab...; - #357's head 0985b179 from a clean checkout (launcher 0.1.95), with vllm/vllm-openai-xpu:v0.33.0@sha256:be3a9aea.... Both passed. The pod was Ready on one card with enableServiceLinks false and no VLLM_PORT, the launcher's warm-up was answered, "What is 17 times 23?" gave 391 through the port-forward agent up prints, and the hermes parser returned get_weather with {"city": "Paris"}. Teardown left 0 GiB on all four cards. The record holds each run's files, the hold's log, the install and removal of the release, and the scripts as they ran. pins/classes lists bench/vllm-agent/** as historical, since the files hold image digests. --- bench/vllm-agent/results/2026-11-08/README.md | 77 ++++ bench/vllm-agent/results/2026-11-08/hold.log | 80 +++++ .../install-0.1.94/after-uninstall.txt | 9 + .../2026-11-08/install-0.1.94/after.txt | 16 + .../2026-11-08/install-0.1.94/before.txt | 9 + .../2026-11-08/install-0.1.94/install.log | 63 ++++ .../install-0.1.94/install.sh.sha256 | 1 + .../2026-11-08/install-0.1.94/uninstall.log | 13 + .../2026-11-08/main-0.1.94/agent-down.txt | 4 + .../2026-11-08/main-0.1.94/agent-up.clean.txt | 14 + .../2026-11-08/main-0.1.94/agent-up.txt | 14 + .../2026-11-08/main-0.1.94/allocatable.json | 1 + .../2026-11-08/main-0.1.94/chat-req.json | 11 + .../results/2026-11-08/main-0.1.94/chat.json | 1 + .../2026-11-08/main-0.1.94/deploy-vllm.yaml | 156 ++++++++ .../2026-11-08/main-0.1.94/describe-pods.txt | 110 ++++++ .../results/2026-11-08/main-0.1.94/down.txt | 7 + .../results/2026-11-08/main-0.1.94/driver.log | 25 ++ .../results/2026-11-08/main-0.1.94/events.txt | 147 ++++++++ .../main-0.1.94/k3d-containers-after.txt | 0 .../2026-11-08/main-0.1.94/models.json | 1 + .../results/2026-11-08/main-0.1.94/pf.log | 5 + .../results/2026-11-08/main-0.1.94/pods.txt | 8 + .../2026-11-08/main-0.1.94/tool-req.json | 33 ++ .../results/2026-11-08/main-0.1.94/tool.json | 1 + .../results/2026-11-08/main-0.1.94/tools.txt | 6 + .../results/2026-11-08/main-0.1.94/up.txt | 93 +++++ .../2026-11-08/main-0.1.94/vllm-previous.log | 1 + .../results/2026-11-08/main-0.1.94/vllm.log | 334 +++++++++++++++++ .../2026-11-08/pr357-0985b179/agent-down.txt | 4 + .../pr357-0985b179/agent-up.clean.txt | 14 + .../2026-11-08/pr357-0985b179/agent-up.txt | 14 + .../pr357-0985b179/allocatable.json | 1 + .../2026-11-08/pr357-0985b179/chat-req.json | 11 + .../2026-11-08/pr357-0985b179/chat.json | 1 + .../pr357-0985b179/deploy-vllm.yaml | 156 ++++++++ .../pr357-0985b179/describe-pods.txt | 110 ++++++ .../2026-11-08/pr357-0985b179/down.txt | 7 + .../2026-11-08/pr357-0985b179/driver.log | 25 ++ .../2026-11-08/pr357-0985b179/events.txt | 146 ++++++++ .../pr357-0985b179/k3d-containers-after.txt | 0 .../2026-11-08/pr357-0985b179/models.json | 1 + .../results/2026-11-08/pr357-0985b179/pf.log | 5 + .../2026-11-08/pr357-0985b179/pods.txt | 8 + .../2026-11-08/pr357-0985b179/tool-req.json | 33 ++ .../2026-11-08/pr357-0985b179/tool.json | 1 + .../2026-11-08/pr357-0985b179/tools.txt | 6 + .../results/2026-11-08/pr357-0985b179/up.txt | 97 +++++ .../pr357-0985b179/vllm-previous.log | 1 + .../2026-11-08/pr357-0985b179/vllm.log | 335 ++++++++++++++++++ bench/vllm-agent/results/2026-11-08/queue | 2 + .../results/2026-11-08/scripts/checkout.sh | 13 + .../results/2026-11-08/scripts/hold.sh | 42 +++ .../2026-11-08/scripts/install-release.sh | 14 + .../results/2026-11-08/scripts/restore.sh | 44 +++ .../results/2026-11-08/scripts/run.sh | 97 +++++ .../2026-11-08/scripts/takeover-stop.sh | 39 ++ .../2026-11-08/scripts/uninstall-release.sh | 13 + .../results/2026-11-08/takeover-restore.txt | 23 ++ pins/classes | 1 + 60 files changed, 2504 insertions(+) create mode 100644 bench/vllm-agent/results/2026-11-08/README.md create mode 100644 bench/vllm-agent/results/2026-11-08/hold.log create mode 100644 bench/vllm-agent/results/2026-11-08/install-0.1.94/after-uninstall.txt create mode 100644 bench/vllm-agent/results/2026-11-08/install-0.1.94/after.txt create mode 100644 bench/vllm-agent/results/2026-11-08/install-0.1.94/before.txt create mode 100644 bench/vllm-agent/results/2026-11-08/install-0.1.94/install.log create mode 100644 bench/vllm-agent/results/2026-11-08/install-0.1.94/install.sh.sha256 create mode 100644 bench/vllm-agent/results/2026-11-08/install-0.1.94/uninstall.log create mode 100644 bench/vllm-agent/results/2026-11-08/main-0.1.94/agent-down.txt create mode 100644 bench/vllm-agent/results/2026-11-08/main-0.1.94/agent-up.clean.txt create mode 100644 bench/vllm-agent/results/2026-11-08/main-0.1.94/agent-up.txt create mode 100644 bench/vllm-agent/results/2026-11-08/main-0.1.94/allocatable.json create mode 100644 bench/vllm-agent/results/2026-11-08/main-0.1.94/chat-req.json create mode 100644 bench/vllm-agent/results/2026-11-08/main-0.1.94/chat.json create mode 100644 bench/vllm-agent/results/2026-11-08/main-0.1.94/deploy-vllm.yaml create mode 100644 bench/vllm-agent/results/2026-11-08/main-0.1.94/describe-pods.txt create mode 100644 bench/vllm-agent/results/2026-11-08/main-0.1.94/down.txt create mode 100644 bench/vllm-agent/results/2026-11-08/main-0.1.94/driver.log create mode 100644 bench/vllm-agent/results/2026-11-08/main-0.1.94/events.txt create mode 100644 bench/vllm-agent/results/2026-11-08/main-0.1.94/k3d-containers-after.txt create mode 100644 bench/vllm-agent/results/2026-11-08/main-0.1.94/models.json create mode 100644 bench/vllm-agent/results/2026-11-08/main-0.1.94/pf.log create mode 100644 bench/vllm-agent/results/2026-11-08/main-0.1.94/pods.txt create mode 100644 bench/vllm-agent/results/2026-11-08/main-0.1.94/tool-req.json create mode 100644 bench/vllm-agent/results/2026-11-08/main-0.1.94/tool.json create mode 100644 bench/vllm-agent/results/2026-11-08/main-0.1.94/tools.txt create mode 100644 bench/vllm-agent/results/2026-11-08/main-0.1.94/up.txt create mode 100644 bench/vllm-agent/results/2026-11-08/main-0.1.94/vllm-previous.log create mode 100644 bench/vllm-agent/results/2026-11-08/main-0.1.94/vllm.log create mode 100644 bench/vllm-agent/results/2026-11-08/pr357-0985b179/agent-down.txt create mode 100644 bench/vllm-agent/results/2026-11-08/pr357-0985b179/agent-up.clean.txt create mode 100644 bench/vllm-agent/results/2026-11-08/pr357-0985b179/agent-up.txt create mode 100644 bench/vllm-agent/results/2026-11-08/pr357-0985b179/allocatable.json create mode 100644 bench/vllm-agent/results/2026-11-08/pr357-0985b179/chat-req.json create mode 100644 bench/vllm-agent/results/2026-11-08/pr357-0985b179/chat.json create mode 100644 bench/vllm-agent/results/2026-11-08/pr357-0985b179/deploy-vllm.yaml create mode 100644 bench/vllm-agent/results/2026-11-08/pr357-0985b179/describe-pods.txt create mode 100644 bench/vllm-agent/results/2026-11-08/pr357-0985b179/down.txt create mode 100644 bench/vllm-agent/results/2026-11-08/pr357-0985b179/driver.log create mode 100644 bench/vllm-agent/results/2026-11-08/pr357-0985b179/events.txt create mode 100644 bench/vllm-agent/results/2026-11-08/pr357-0985b179/k3d-containers-after.txt create mode 100644 bench/vllm-agent/results/2026-11-08/pr357-0985b179/models.json create mode 100644 bench/vllm-agent/results/2026-11-08/pr357-0985b179/pf.log create mode 100644 bench/vllm-agent/results/2026-11-08/pr357-0985b179/pods.txt create mode 100644 bench/vllm-agent/results/2026-11-08/pr357-0985b179/tool-req.json create mode 100644 bench/vllm-agent/results/2026-11-08/pr357-0985b179/tool.json create mode 100644 bench/vllm-agent/results/2026-11-08/pr357-0985b179/tools.txt create mode 100644 bench/vllm-agent/results/2026-11-08/pr357-0985b179/up.txt create mode 100644 bench/vllm-agent/results/2026-11-08/pr357-0985b179/vllm-previous.log create mode 100644 bench/vllm-agent/results/2026-11-08/pr357-0985b179/vllm.log create mode 100644 bench/vllm-agent/results/2026-11-08/queue create mode 100644 bench/vllm-agent/results/2026-11-08/scripts/checkout.sh create mode 100644 bench/vllm-agent/results/2026-11-08/scripts/hold.sh create mode 100644 bench/vllm-agent/results/2026-11-08/scripts/install-release.sh create mode 100644 bench/vllm-agent/results/2026-11-08/scripts/restore.sh create mode 100644 bench/vllm-agent/results/2026-11-08/scripts/run.sh create mode 100644 bench/vllm-agent/results/2026-11-08/scripts/takeover-stop.sh create mode 100644 bench/vllm-agent/results/2026-11-08/scripts/uninstall-release.sh create mode 100644 bench/vllm-agent/results/2026-11-08/takeover-restore.txt diff --git a/bench/vllm-agent/results/2026-11-08/README.md b/bench/vllm-agent/results/2026-11-08/README.md new file mode 100644 index 00000000..4b8c62b5 --- /dev/null +++ b/bench/vllm-agent/results/2026-11-08/README.md @@ -0,0 +1,77 @@ +# The vLLM agent on an Arc Pro B70, unpatched (2026-11-08) + +The end state of `K3D_GPU_AGENT_BACKEND=vllm k3d-gpu agent up` on one Arc Pro B70, through +the launcher alone: no `kubectl patch`, no warm-up sent by hand, the default image and +model. Two launchers ran in one hold: + +- `main-0.1.94`: release 0.1.94, the first release with + [#339](https://github.com/88plug/k3d-gpu/pull/339) (no Service links in the pod, and + the warm-up sent from inside the pod as JSON). +- `pr357-0985b179`: the head of [#357](https://github.com/88plug/k3d-gpu/pull/357), which + moves the image to `v0.33.0`. That head was regenerated onto main after #339 merged. + +Both passed. Releases 0.1.21 through 0.1.93 do not serve with vLLM: their Service gives +the pod `VLLM_PORT=tcp://...`, on which vLLM exits, and their warm-up reaches vLLM with no +`Content-Type`, which it answers with 400. + +This record covers vLLM XPU on Arc only. The Gaudi path and llm-scaler have not run on +hardware, and neither has tensor parallelism above 1. + +## Host + +crescent: Xeon 6741P, 4x Arc Pro B70 on the `xe` driver, kernel `7.0.0-31-generic`, Ubuntu +26.04.1 LTS. The hold took `~/k3dgpu-bench.lock`. It freed the cards by stopping +`qwen38-a-hr`, `qwen38-b-hr` and `aiperf010` with `docker stop -t 120`, after a snapshot, +a check that neither server had a request in the last 10 minutes, and each server's +baseline answer ("Paris"). Afterwards it started them again, and both servers gave the same +answer. All four cards read 0 GiB before each run and after each teardown. + +## Launchers and versions + +| | `main-0.1.94` | `pr357-0985b179` | +|---|---|---| +| Launcher | `/usr/bin/k3d-gpu` 0.1.94 from `k3d-gpu_0.1.94-1_all.deb` (sha256 `60a1dc5bbc5e6f2a439c34c9ec19dc2c916dcd35c5bec9d46752ee2ebcfd8dec`, the `aur-v0.1.94` release asset) | `scripts/k3d-gpu` 0.1.95 from a clean checkout of `0985b17902e507b13d8460970fbe2a15051ac5c9` (sha256 `e2d97e1960a04516d83ddc1c21c4142a406a2be6797ca1f102bd5feb1b495438`) | +| How installed | `packaging/install.sh` from main at `e5f55b29` (sha256 `62c30da91025f5af25557e5e49da8c496591d98f3fb0f9a393c673b65edbbc55`), which pins 0.1.94; it also put k3d v5.9.0 and kubectl v1.37.2 in `~/.local/bin` | `scripts/checkout.sh` (`git fetch --depth 1` of the commit, no changes) | +| Node image | `cryptoandcoffee/k3d-gpu:0.1.94` (`sha256:8630e14556e70111a097dcf22d15a7484bc94fa19a2d4b04b311ec60778d438e`) | `cryptoandcoffee/k3d-gpu:0.1.95` (`sha256:65499cbe202dca63c28029e03c4eb5139f5de31f5f369d915494f06e6f4dd3fa`) | +| Agent image | `vllm/vllm-openai-xpu:v0.32.0@sha256:20509fabe50dda62c09c0786d1a6effa299d3dbdddcc49331b93e65144652a8c`, vLLM 0.32.0 | `vllm/vllm-openai-xpu:v0.33.0@sha256:be3a9aea234be64ffc2816ac561c597e0f0b1f44a61a7f4912780e4d77450086`, vLLM 0.33.0 | +| Model | `Qwen/Qwen3-4B` at `1cfa9a7208912126459214e8b04321603b3df60c`, float16, `hermes` tool parser, `qwen3` reasoning parser | the same | + +The release package and the k3d and kubectl binaries the installer added were removed after +the hold (`install-0.1.94/after-uninstall.txt`). + +## Results + +| | `main-0.1.94` | `pr357-0985b179` | +|---|---|---| +| `k3d-gpu up` | rc 0 in 48 s, allocatable `gpu.intel.com/xe: 4` | rc 0 in 49 s, the same | +| `agent up` | rc 0 in 255 s | rc 0 in 258 s | +| Pod | Ready, 0 restarts, limits `gpu.intel.com/xe: 1`, `enableServiceLinks: false`, no `VLLM_PORT` in its environment, one render node (`renderD129`) | the same, `renderD128` | +| VRAM while serving | 27.50 GiB on 0000:48:00.0, 0 on the other three cards | 27.30 GiB on 0000:12:00.0, 0 on the other three | +| KV cache (vLLM's log) | 137,472 tokens | 131,776 tokens | +| The launcher's warm-up line | `warm-up reply (2s): I am a large language model developed by Alibaba Cloud.` | the same | +| "What is 17 times 23?" | `391` (ends `\boxed{391}`), finish `stop`, 1,426 completion tokens (1,283 reasoning), 22.7 s | the same | +| Tool call | `get_weather` with arguments `{"city": "Paris"}`, finish `tool_calls`, 1.6 s | the same | +| Teardown | `agent down` and `down` rc 0, 0 GiB on all four cards, no k3d containers left | the same | +| `run.sh` verdict | PASS | PASS | + +Both requests went through the endpoint `agent up` prints +(`kubectl -n k3d-gpu-agent port-forward svc/vllm 8000:8000`). The chat request had no +system prompt, thinking left on, temperature 0 and `max_tokens` 4096. The tool request +offered one function, `get_weather(city)`, with "What is the weather in Paris right now?" +and `tool_choice: auto`. `run.sh` passes a run only when the pod is Ready with one +`gpu.intel.com/xe`, the launcher printed its warm-up reply, the chat answer contains 391, +and the tool call is `get_weather` with `city` Paris. + +## Files + +| Path | What | +|---|---| +| `hold.log` | The hold, in order: the lock, the services stopped, each run's driver lines, the restart and the baseline answers | +| `takeover-restore.txt` | The stop and restart logs of the three services, and the names stopped | +| `queue` | The runs the hold took, as `