Repository navigation
feat(kata): add selectable Kata workspaces - #166
Conversation
Add selectable container workspaces alongside the default VM backend, using shared BOM/provider/tool-gate configuration and the stock OpenShell Kubernetes driver. Unify RHDH, static GitOps and Make composition, enrollment, lifecycle, credential isolation, owner browser access and retained audit handling. Include offline contracts, build workflows, installation and architecture guides, and the bounded OpenAI/data-science qualification matrix. Live checks covered container lifecycle and fresh-workspace replay plus VM inference, chat, audited tools, model update/restart persistence and RHDH deletion. Developed with assistance from OpenAI Codex.
…l deployment paths
Expose Kata in Make, GitOps and Developer Hub. Bind each workspace to a reviewed RuntimeClass, validate sandbox placement, and reuse the container lifecycle for enrollment, browser chat and restart.
Pin built-in NemoClaw references to the verified multi-arch image index so Kata accepts enabled sandbox images.
Regenerate the RHDH and GitOps catalogs from the pinned profile images so both entry points use the same immutable references.
Report firmware for context while requiring KVM on selected workers. Red Hat docs do not require UEFI for plain Kata.
Retain the shared Rust admission extension for VM and Kubernetes driver configuration, the deployment-mode Make settings, and the PR image publication checks.
Document the OperatorHub and KataConfig setup used for the qualification cluster. Keep platform resources outside SAW charts.
Keep the configured observer and grant the RHDH generator and portal\nprovisioner read access to the workspace status ConfigMaps. Limit the\nrole to get on the three named ConfigMaps.
Pin the cluster-local interceptor image built from 1cdc9d4 for the live Kata test.
Use the selected OpenShell gateway version in the runtime plan. Keep the enrollment default aligned with the gateway image pin.
04d6bcc to
adeea57
Compare
Return fixed reasons for approval, identity, handler, scheduling, label, and placement failures. Align RHDH and enrollment status output.
Read the owner-scoped public CA ConfigMap through the Kubernetes API from enrollment and controller pods. Limit both RBAC grants to that ConfigMap. Remove cross-namespace CA volume mounts.
The identity chart writes the public CA ConfigMap in the registration control namespace. Read it there and scope both ServiceAccounts to that named ConfigMap.
Check approved live admission hashes before portal credentials or Make\napplies a workspace. Require complete hashes for first registrations\nand check generated policies during enrollment.\n\nTest the shared preflight, portal, Make, and projection paths.
Check the approved Kata runtime and existing admission boundaries before ApplicationSet emits a workspace Application. Grant only the required read access. Keep first-registration checks in enrollment.
OpenShell 0.1.6 adds OPENSHELL_GATEWAY_TLS_SERVER_NAME to the supervisor Pod when the gateway endpoint uses HTTPS. Allow this single stock variable in the strict runtime admission rule.
The OpenShell gateway adds OPENSHELL_GATEWAY_NAME to each supervisor. Allow that stock variable in the supervisor admission contract.
The stock shared namespace driver creates and reads per-sandbox NetworkPolicies. Give the gateway those verbs in its namespace. Keep the grant in the chart so GitOps can apply it.
Reject the pinned OpenClaw image when an OpenShift namespace UID cannot traverse its workspace directory. Require a digest-pinned profile override before RHDH, GitOps, or Make provisions a workspace. Record the documented image build and the live canary failure.
Record the successful RHDH Kata canary and the remaining checks. Explain that an image override changes the validation policy digest and that the portal generator must reload its site Secret.
There was a problem hiding this comment.
Recommendation: request changes for F1 and F2. Functional CLI checks passed on both actual Kata workloads, but full qualification is incomplete. Testing ended at the user's request. This report reviews PR #166 as the sole merge candidate, including the shared container implementation it delivers. PR #153 is historical context and is not a prerequisite merge.
Reviewed head: 5760c1910b6844c3412471c9560ece747a0cddb1. Target snapshot: 561dc781a6c6ab36a86dcffe5b597954cd26908d (an ancestor of the head). Review date: 2026-11-17. Deployment: cluster-dm6p7, OpenShift 4.22.22, amd64. The source is unchanged; site configuration is isolated on codex/pr166-review-dm6p7 in the operator's fork.
Final refresh confirmed the same head/base, PR still open, eleven successful CI checks and four skipped checks. Site-only HEAD is e0a5923e4dbeafc5e4525a9a3563e11e33987873; the candidate checkout is clean. See final PR metadata.
Findings and qualification blockers
F1 — P2: documented provider inputs are rejected before the policy is derived
The first-install guide explicitly says not to author providerPolicy; saw-users/templates/applications.yaml derives it from the governed catalog and operator-owned providerAllowHosts. However, the shared configuration_facts() requires site.providerPolicy.allowHosts to exist before that derivation. Both the Make preflight and RHDH preflight call it on the raw site. A site following the guide is therefore rejected with ProviderEgressRejected, even with the approved NVIDIA host explicitly configured.
Locations: shared preflight, lines 236–241, Make caller, RHDH caller, and installation instruction.
Reproduced against the fresh cluster by removing only providerPolicy from the otherwise complete owner configuration and invoking the candidate's read-only Make preflight. It fails with SAW preflight failed: ProviderEgressRejected before registration.
Derive the catalog policy before checking it, or make the preflight consume the documented operator inputs while preserving the eventual generated-policy checks. Add a first-install test that omits the generated field. Existing fixture-based tests contain a manually populated providerPolicy and miss this case. For continued qualification, this run explicitly retains the matching NVIDIA policy in private site inputs; this is a documented setup workaround, not a fix to the candidate.
F2 — P2: the documented second Make stage fails against its own Helm-owned policy
After a successful foundation install, enabling the inspected approval and rerunning make saw-create fails before Helm upgrade. The helper performs a server-side apply dry run over resources that the first stage installed through Helm. Kubernetes rejects the unchanged projector credential admission policy with conflict with "helm" … .spec.matchConstraints; its API-defaulted atomic match constraints are owned by Helm. The helper reduces this to oc operation failed, hiding the actionable cause. This blocks the documented two-stage Make workflow and subsequent read-only preflight against that foundation.
Location: server-side dry run before Helm upgrade, line 355. Minimal reproduction: export the already-installed saw-projector-a343914741091c751a27-credentials policy from helm get manifest workspace-pr166a -n saw-workspaces-pr166, then submit it with oc apply --server-side --dry-run=server -f <policy>. Even the unchanged manifest conflicts; selecting field manager helm also conflicts with its existing Update ownership.
Accept operator provider hosts before GitOps derives policy. Dry-run Helm-owned resources without field manager conflicts, while rejecting resources outside the release. Add regression coverage.
Summary
Add a separate
deployment.mode: katafor APPENG-6912. RHDH, GitOps, and Make use the shared Kubernetes workspace lifecycle developed in PR #153. OpenShell worker Pods use an owner-approved KataRuntimeClass. The gateway and supervisors use the default runtime.PR #166 is the merge candidate for both the shared container lifecycle and Kata changes. It contains the shared changes once. PR #153 provides development history; it does not need to merge first.
Cluster preparation
SucceededCSV.KataConfigYAML for that pool. Wait for the node installation and required reboots.kataRuntimeClass handler, worker selector, overhead, and UID. Approve its placement digest in the owner site Secret.container-notebook-imagetarget. Add its digest-pinned profile override and scoped ImageStream pull access to the site Secret. Approve the new profile and validation policy digests.docs/kata-workspaces.md. The SAW YAML binds Kata only to worker Pods and checks Sandbox and Pod admission. Keep the gateway and supervisors on the default runtime.Gate and failure diagnosis
make saw-preflightperforms read-only cluster checks. RHDH checks approved profile and platform inputs before it stores credentials or registers an owner. GitOps checks the owner and Kata platform before projecting an Application. Enrollment checks generated policy resources before it publishes the immutable runtime binding. A first registration has no generated policy to compare until Argo creates it.The first live runs stopped at policy approval drift, gateway policy RBAC, and a stock notebook image that could not traverse
/sandboxunder the OpenShift namespace UID. Those causes are fixed or gated in this branch. The notebook image override also changes a literal image reference in the validation admission policy. The live policy's only approval difference was that image reference. The reviewed validation digest was updated. The portal generator reads the site Secret through an environment variable, so it had to restart before the new digest reached the owner Argo Application. The corrected enrollment Job then succeeded in the same registration.Review fixes on 2026-11-18
providerAllowHostsinput before GitOps derivesproviderPolicy. If a policy is already present, its hosts must match. Make, RHDH, and the GitOps projector use the same check.--force-conflictsin the server dry run. A resource outside this release fails before apply. Field ownership errors get a safe, specific message.75f6b88.Kata canary on 2026-11-17
48f96ee5-ec90-48ff-ac3e-4623eeadd0a9and PipelineRunsaw-create-7xpcfsucceeded. RHDH reportedWorkspace ready.runtimeClassName: kata. Their Pod UIDs mapped to different QEMU PIDs on the worker node.Remaining live coverage
Tool allow and deny rules, network denial, secret isolation, audit, persistence, restart, stop/start, deletion, GitOps and Make creation, a container regression, and two active owners remain unverified. These checks remain before full qualification.