Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
11 changes: 8 additions & 3 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -11,7 +11,8 @@ A Docker-based [rancher/k3s](https://hub.docker.com/r/rancher/k3s) node image on
> i9-13900T host. On a second client host, an i7-13700 whose owner opts its GPUs
> out, a plain `up` found the hybrid P/E cores and the ISA extensions the launcher
> labels and nothing the host lacks [ledger:H52w], and `k3d-gpu sandbox up` kept
> agent sandboxes on its E-cores behind default-deny egress [ledger:H18w].
> agent sandboxes on its E-cores behind default-deny egress [ledger:H18w]. There,
> OVMS's shipped prefix caching gave an added agent its first chunk 14.3x sooner [ledger:H25w].
> On a Xeon 6741P with 4x Arc Pro B70, a plain `up` found the
> B70s (`xe`), AMX and AVX-512 [ledger:H52]. In a second run there, `xe` advertised
> 2 of the 4 B70s: a serving workload held about half the VRAM of each card, and
Expand Down Expand Up @@ -1449,7 +1450,11 @@ check` and the teardown passed, but enterprise-inference's Keycloak step put the
admin password on a process command line, where the run's secret scan found it. F5 ran it
at the GA tags on the Xeon 6741P and is SUPPORTED
([EXPERIMENTS.md](EXPERIMENTS.md#f5-run-xeon-6741p-2026-11-07)): `eat up` with `failed=0` and
no hand edit, `eat check`, a secret scan of 0 and `eat down` all passed. The commands are `k3d-gpu eat up` (`--dry-run`,
no hand edit, `eat check`, a secret scan of 0 and `eat down` all passed. H62r ran Intel's own
documented install of the same tags in a KVM guest on that Xeon: with EAS's documented
`nri_reserved_cpu_list` it finished, and inside EAT's banking flow it delivered as many tasks
within the turn budget as k3d-gpu's CPU stack, 21 of 24 each
([EXPERIMENTS.md](EXPERIMENTS.md#h62r-run-xeon-6741p-2026-11-07)). The commands are `k3d-gpu eat up` (`--dry-run`,
`--explain`; `--legacy` runs EAT 1.x), `eat status`, `eat check` and `eat down`, or `k3d-gpu
eas` with the layer named. See [docs/intel/eas.md](docs/intel/eas.md), and
[docs/intel.md](docs/intel.md#upstream-findings) for the upstream findings and issue drafts.
Expand Down Expand Up @@ -1581,7 +1586,7 @@ What works on k3d, and what does not:
| Piece | On k3d-gpu |
|-------|------------|
| Controller, Sandbox, SandboxTemplate, SandboxClaim, SandboxWarmPool | Yes. Plain manifests, no webhook or cert-manager. |
| NetworkPolicy | Enforced by k3s's embedded kube-router controller. The template allows ingress on 8888 only from pods labelled `k3d-gpu.io/sandbox-client=true` (and upstream's `sandbox-router`). Egress is limited to cluster DNS and public addresses. RFC1918, CGNAT `100.64.0.0/10` (tailscale) and link-local are blocked, so a sandbox cannot reach pods, services, the k3d docker network, tailnet peers or the host by those addresses. A blocked connect fails at once with `Connection refused` (kube-router rejects it), not a timeout [ledger:H18]. The host's public IPs stay reachable, so bind inference servers to a private address or firewall them. Ingress is gated on a pod label that any namespace can set: fine for a single-user dev cluster, not for a shared one. |
| NetworkPolicy | Enforced by k3s's embedded kube-router controller. The template allows ingress on 8888 only from pods labelled `k3d-gpu.io/sandbox-client=true` (and upstream's `sandbox-router`). Egress is limited to cluster DNS and public addresses. RFC1918, CGNAT `100.64.0.0/10` (tailscale) and link-local are blocked, so a sandbox cannot reach pods, services, the k3d docker network, tailnet peers or the host by those addresses. A blocked connect fails at once with `Connection refused` (kube-router rejects it), not a timeout [ledger:H18], also on a second hybrid host [ledger:H18w]. The host's public IPs stay reachable, so bind inference servers to a private address or firewall them. Ingress is gated on a pod label that any namespace can set: fine for a single-user dev cluster, not for a shared one. |
| gVisor / Kata (`runtimeClassName`) | Not available. The node image ships neither, and Kata needs `/dev/kvm` in the node container. Sandboxes run under runc, hardened instead: non-root, all capabilities dropped, no privilege escalation, `RuntimeDefault` seccomp, no service-account token. That is container isolation, not a VM or user-space kernel boundary. |
| sandbox-router, sandboxd | Not installed. Upstream publishes them only as staging `latest-main` images, which cannot be pinned to a release. Clients reach a sandbox by pod IP (from a `k3d-gpu.io/sandbox-client=true` pod) or `kubectl port-forward`, which containerd opens inside the pod's network namespace, so the policy does not block it. |

Expand Down
22 changes: 13 additions & 9 deletions docs/intel.md
Original file line number Diff line number Diff line change
Expand Up @@ -108,13 +108,14 @@ the plugins, and asserts the result, with no manual step.
`nvidia=2, intel=1, hybrid=1`. That run used commit `109154d`. The hybrid
`test` there only checked that each pod printed its `Cpus_allowed_list`; the
stricter check that the list sits inside the P-core or E-core list landed
after it (`fb5c17d`) and has not yet run on hardware. The second is a Xeon 6741P
after it (`fb5c17d`); its first hardware run was on the third host, below. The second is a Xeon 6741P
(Granite Rapids, 48 cores, AMX, AVX-512, 3 DSA, 3 IAA, 3 QAT 4xxx) with 4x Arc Pro
B70 on `xe`. A plain `up` ran there on 2026-10-20 [ledger:H52], and `up` after
`k3d-gpu host-setup --apply` on 2026-10-29 [ledger:H52b]. The third is wildct, an
i7-13700 (P-cores 0-15, E-cores 16-23) with a UHD 770 iGPU and 2 NVIDIA GPUs, all three
opted out by the box owner (`K3D_GPU_VENDORS=auto,-nvidia,-intel`), so only its CPU rows
have run. A plain `up` of release 0.1.89 ran there on 2026-11-07 [ledger:H52w].
have run. A plain `up` of release 0.1.89 ran there on 2026-11-07, and the strict `test`
passed: its P-core pod ran on CPU 0 and its E-core pod on CPU 17 [ledger:H52w].
- **Verified by sim**: `tests/detect_test.sh` builds a fake sysfs/devfs tree,
sets `K3D_GPU_SYSROOT`, and runs `detect`, `doctor` or `test` against it, with
`docker`, `k3d` and `kubectl` stubbed. 1028 cases pass (2026-10-20). Sim covers detection,
Expand Down Expand Up @@ -215,7 +216,7 @@ That split, orchestration and serving on fast cores with agent tools on many sma
is the placement k3d-gpu sets with one pod label, not isolation: package power, LLC and
DRAM bandwidth stay shared, so E-core load still slowed CPU-served prefill on the
P-cores [ledger:H3]. Sandboxes on E-cores behind default-deny egress is SUPPORTED on the
i9-13900T [ledger:H18]. No Xeon run backs it yet.
i9-13900T [ledger:H18] and on wildct's i7-13700 [ledger:H18w]. No Xeon run backs it yet.

### EAT by default vs with k3d-gpu

Expand All @@ -234,7 +235,7 @@ and installs through EAS.
| Hosts | Xeon. Minimum 48 cores and 32 GB RAM, recommended 96+ cores and 64 GB; Docker mode asks for 96+ cores and 128 GB | Any Intel host it detects: client Core and Core Ultra, Xeon, Arc, Gaudi (detection and config fixture-tested per family, [platforms.md](platforms.md)). EAT ran end to end on one i9-13900T with no AMX and no AVX-512, n=1 [ledger:H1b] |
| OS | Ubuntu 22.04 / 24.04, x86_64 | Any Linux with Docker that runs k3d; `host-setup` prints pinned driver steps for Ubuntu 24.04/26.04, Fedora, Arch and Manjaro |
| Cluster setup | Kubespray onto the host, or brownfield on an existing cluster; the `ei-*-eligible` labels come only from the Kubespray path | One `k3d-gpu up`; `K3D_GPU_NODE_LABELS` sets EAT's labels, `K3D_GPU_BALLOONS=external` hands the NRI policy to EAT [ledger:H14] |
| EAT 2.0 install | EAS `es_auto_installer.sh install agent-toolkit`, onto Kubespray or an existing cluster (`existing_kubernetes`) | `k3d-gpu eat up` (`eas up --agent-toolkit`): the existing-cluster mode at the GA tags (EAS `v1.2.0`, EAT `release-2.0.0`), with the k3d settings and patches chosen from the cluster ([intel/eas.md](intel/eas.md)). F5 checks it on the Xeon 6741P. F3 (EAS `main`) is REFUTED ([run](../EXPERIMENTS.md#f3-run-i9-13900t-2026-11-01)): it installed, checked and tore down, and the secret scan found the Keycloak admin password on a command line |
| EAT 2.0 install | EAS `es_auto_installer.sh install agent-toolkit`, onto Kubespray or an existing cluster (`existing_kubernetes`) | `k3d-gpu eat up` (`eas up --agent-toolkit`): the existing-cluster mode at the GA tags (EAS `v1.2.0`, EAT `release-2.0.0`), with the k3d settings and patches chosen from the cluster ([intel/eas.md](intel/eas.md)). F5 ran it once on the Xeon 6741P and is SUPPORTED ([run](../EXPERIMENTS.md#f5-run-xeon-6741p-2026-11-07)). F3 (EAS `main`) is REFUTED ([run](../EXPERIMENTS.md#f3-run-i9-13900t-2026-11-01)): it installed, checked and tore down, and the secret scan found the Keycloak admin password on a command line |
| Engines | vLLM CPU (`vllm/vllm-openai-cpu:v0.24.0`, bf16), 9 model profiles in the single-node table (7 chat, 1 embedding, 1 reranker). An OVMS Helm chart ships but no default path deploys it: `openvino/model_server:2025.4`, plus an unpinned `latest-py` tag for its model-export containers | `agent up` picks per host: vLLM on a Gaudi, OVMS 2026.4.1 on an XMX GPU, else OVMS on the P-cores, or, on a CPU class where an engine-choice row is SUPPORTED (the Xeon 6741P today), llama.cpp with the model's int4 GGUF (for Gemma 4 26B-A4B, the one model measured, Google's QAT Q4_0 GGUF and QAT MTP drafter) [ledger:H59]; vLLM XPU (`v0.32.0`) or llm-scaler on Arc by flag. Its images are pinned by digest in `images.lock` |
| Accelerators | None requested: no `gpu.intel.com`, `habana.ai` or `/dev/dri` on the default path | Intel GPU, NPU, Gaudi, DSA, IAA and QAT plugins started per detected device, and `up` checks that each is schedulable (on the Xeon 6741P it warned when QAT advertised none [ledger:H52b]) |
| CPU placement | NRI balloons sized for Xeon NUMA; on a hybrid CPU EAT's reservation needs the patch in [`contrib/eat/`](../contrib/eat) | Balloons baked for hybrid P/E and multi-NUMA hosts; vLLM on P-cores and agent services on E-cores through `contrib/eat/` values |
Expand Down Expand Up @@ -268,9 +269,11 @@ stated as the ledger graded it; only the SUPPORTED rows back a claim.
| H51 | Graded agents per box: neither CPU stack served one graded agent within the 10 s SLO ([run](../EXPERIMENTS.md#h51-run-xeon-6741p-2026-10-29)) | REFUTED |

Detection (H52 REFUTED, H52b REFUTED) is in the [capability matrix](#capability-matrix).
H54d and H54q are in the table below. After the campaign, H62r repeats H62 at Intel's GA
tags of 2026-11-06 (EAS v1.2.0 installing EAT `release-2.0.0` with enterprise-inference
v1.1.0); it is pre-registered and not yet run
H54d and H54q are in the table below. After the campaign, H62r, in the table above, repeated
H62 at Intel's GA tags of 2026-11-06 (EAS v1.2.0 installing EAT `release-2.0.0` with
enterprise-inference v1.1.0). It ran on 2026-11-07 with EAS's documented
`nri_reserved_cpu_list` set to the guest's sibling vCPUs, because in a KVM guest of that shape
EAS's topology probe reads one thread per core
([pre-registration](../EXPERIMENTS.md#h62r-pre-registration-h62-at-the-ga-tags-eas-v120-with-eat-20-on-the-xeon-6741p-2026-11-06)).

H54 rows on that host: Gemma 4 26B-A4B (QAT weights) on the Xeon CPU, and split with a B70
Expand Down Expand Up @@ -299,8 +302,9 @@ in its registered order. Each row is stated as the ledger graded it.
| [ledger:H25w] | H25 replicated: with OVMS's shipped prefix caching on, an added agent's first chunk arrived 14.3x sooner than with it off (CI 13.38-14.89), under matched `typical` load ([run](../EXPERIMENTS.md#h25w-run-wildct-i7-13700-2026-11-07)) | SUPPORTED |
| [ledger:H34w] | H34 replicated: OVMS's CPU Fast Draft against the 4B target alone under matched `typical` load. Neither the pre-registered gain nor its exclusion was resolved, so the i7-13700 class gets no CPU draft by default ([run](../EXPERIMENTS.md#h34w-run-wildct-i7-13700-2026-11-07)) | INCONCLUSIVE |

H28w, H57wa, H57wm, H57wb and the engine-choice row H59w are pre-registered there and not
yet run.
H28w has run there; its result is in review
([#344](https://github.com/88plug/k3d-gpu/pull/344)) and not yet in the ledger. H57wa, H57wm,
H57wb and the engine-choice row H59w are pre-registered there and not yet run.

Facts from `0ecf67a` that shape a k3d run:

Expand Down
2 changes: 1 addition & 1 deletion docs/intel/eas.md
Original file line number Diff line number Diff line change
Expand Up @@ -220,7 +220,7 @@ Open EAS PRs that change a decision when they merge (`overlay.py --watch`, check

| EAS PR | Effect on the overlay |
|---|---|
| [#50](https://github.com/intel/enterprise-ai-solutions/pull/50) and [#52](https://github.com/intel/enterprise-ai-solutions/pull/52) (OpenShift) | 0006 no longer applies: the selector moves into `_eg_k8s_provider`, still `control-plane: ""`. That is a STOP naming #50. Once a k3s vars file lands, 0006 becomes drop-fixed |
| [#50](https://github.com/intel/enterprise-ai-solutions/pull/50) and [#52](https://github.com/intel/enterprise-ai-solutions/pull/52) (OpenShift) | 0006 no longer applies: the selector moves into `_eg_k8s_provider`, still `control-plane: ""`. That is a STOP naming #50. Once a k3s vars file lands, 0006 becomes drop-fixed. #50 merged into EAS `main` on 2026-11-02 and #52 is open (read 2026-11-07); neither is in `v1.2.0`, so only a move of the `main` profile past #50 meets the STOP |
| [#61](https://github.com/intel/enterprise-ai-solutions/pull/61) (Headlamp SSO), merged 2026-10-30 | 0002 fails on its `global_config.yaml` hunk context: a STOP naming #61, so 0002 needs regenerating. 0004 (skipped on one node) and 0005 (docs, never applied) fail there too. 0006 still applies. EAS `main` is `0f7bc2b` since (read 2026-11-01); the pin stays `97f6157`, where the decisions above hold, and moving it past #61 needs 0002 regenerated first (read from #61's head, not re-checked on the merge) |

## Pins
Expand Down
Loading
Loading