[FEATURE] Mount tool images into a Harness's filesystem

📝 Feature Summary

Let a Harness mount digest-pinned OCI images read-only next to its runtime image, with their bin/ on PATH, using Substrate's image volumes.

❓ Problem Statement / Motivation

Agents need CLIs their harness image does not ship (go, node, kubectl, helm). Today the options are:

an image built FROM the harness image, which must be rebuilt for every kagent release because the harness checks its pinned CLI version, and once feat: default built-in Harness images from release charts #3151 lands the release owns those images;
skills or plugins, which are capped at 100 MiB, never reach PATH, and land in /data and so in every snapshot.

#1822 asked for this decoupling, and the reply there was "I think just PATHs?". Substrate now supports it: agent-substrate/substrate#923 added image volumes as "how tooling gets into images built by third parties without rebuilding them", and kagent already uses one for the sandbox guest.

💡 Proposed Solution

apiVersion: api.kagent.dev/v1alpha3
kind: Harness
metadata:
  name: claude-go
spec:
  claude: {}
  workload:
    image: registry.example.com/claude-harness@sha256:...
    toolImages:
      - name: go
        oci: registry.example.com/tools/go@sha256:...

The controller adds an image volume per entry to the ActorTemplate, mounted read-only at /opt/kagent/tools/<name>. This works the same for every runtime: kagent, claude, codex and BYO.
It sets KAGENT_TOOL_PATHS instead of PATH, because a template PATH replaces the image's. Each runtime appends it to its own PATH at start, so harness binaries always win. For the kagent runtime this reaches the skills bash tool in both the Python and Go ADK, which inherit the process environment. For BYO it becomes part of the documented contract.
The Harness is the platform-managed runtime configuration, so operators decide which tools are offered with the RBAC they already use for Harnesses, and agents keep choosing a Harness as today.
References are digest-pinned like every other image in the API, and the controller resolves no tags: keeping a digest current is the job of whatever writes the Harness (GitOps, Renovate). A change is a new revision; running Sessions keep theirs.
Documented contract for a tool image: files under one prefix, binaries static or self-contained (the built-in harness images are musl).

🔄 Alternatives Considered

A field on the AgentTemplate: agents could combine tools freely instead of one Harness per combination, but any agent author could then mount any image, and operators would need an admission policy to restrict it. A later step could let a Harness list allowed tool images and an AgentTemplate pick among them.
A Sandbox: it has its own filesystem, so the agent's CLI cannot run go test on its own workspace without syncing files.
A sidecar (feat: add extraContainers to SharedDeploymentSpec for sidecar support #1724, feat(substrate): render BYO extraContainers in the SandboxAgent ActorTemplate #2691, feat(substrate): render BYO extraContainers in the SandboxAgent ActorTemplate #2799): it runs in a separate mount namespace.

🎯 Affected Service(s)

Controller, the runtimes' start-up (PATH), BYO contract docs (#2884).

📚 Additional Context

Open to maintainers' preference on the field's name, the mount path and the variable's name.

🙋 Are you willing to contribute?

I am willing to submit a PR for this feature

feat(harness): mount tool images read-only into the runtime

I opened #3192 with the Harness field, but I'm happy to change the shape. Another option: a separate resource for tool images that an Agent references by name, the way an Agent references a ModelConfig.

apiVersion: api.kagent.dev/v1alpha3
kind: ToolImage
metadata:
  name: go
spec:
  oci: registry.example.com/tools/go@sha256:...
---
apiVersion: api.kagent.dev/v1alpha3
kind: Agent
spec:
  harnessRef: claude
  toolImages: [go, node]

Agents combine tools freely, with no Harness per combination (claude-go, claude-go-node, codex-go...).
Operators still decide what is offered, through RBAC on the new kind, so no admission policy is needed to stop an agent from mounting an arbitrary image.
The reference stays digest-pinned on the resource, and whatever writes it (GitOps, Renovate) keeps it current. The controller resolves no tags.
The resource could later declare which runtimes it fits, so the controller refuses a mismatch.

The Harness field is the smaller change. Which do you prefer?

Can't we re-use skills for this potentially?

I thought about it, but I don't think skills work for toolchains.

Skills are limited to 100 MiB and 10k files per artifact. The go binary is small, but it needs the compiler tools and the standard library source to build anything. That is around 190 MB, so it doesn't fit, even with a distroless base. Skills are also pulled and extracted by the runtime on every start. An image volume is pulled once and cached by the node.

The purpose is also different. A skill needs a SKILL.md because the model reads it. A tool image just adds binaries to PATH, and the model doesn't need to know it exists.

A small static binary could fit in a skill, but it would still need the PATH wiring from #3192. The image reference already uses the same digest-pinned oci format as skills, so pinning works the same way.

Both shapes in #3192 let operators keep toolchains platform-only with plain RBAC, on the Harness or on a ToolImage kind. In both, updating a digest once applies to every agent that uses it. The real difference is combining tools. With the Harness field, each combination needs its own Harness. With a separate kind, an agent just lists the toolchains it needs. I'm fine with either, so it depends on whether you want a new kind.

FWIW I was actually thinking about switching the OCI pulls and eventually S3 as well to use volumes and then just copy the data over

I think that's a good idea, and it would definitely improve skills.

For tools, I'd keep them separate from skills in the API, because a skill is something the model reads, while a tool image only puts binaries on PATH plus those are two semantically different concepts.

I see two good shapes, both on the AgentTemplate, since needing Go comes from what the agent does, not from its runtime:

A toolImages field next to skills, with the same oci source. It matches how skills work today and needs no new kind.
A ToolImage kind that templates reference by name. Each image is defined once, and operators decide which tools exist through RBAC on the CR.

Either one avoids a Harness per combination. IMO the kind is the better model semantically, but the field is the smaller change, and we could add a reference to it later.

What would you prefer? And can tools go first, or should they wait for skills and plugins to move to volumes?

IMHO this could stay inside skills with additional BIN PATH for extra tools

Copy skills
Copy tools

We want to enhance existing Kagent tools image to support that.

Thanks for the feedback.

I'm not sure I fully understand it yet, so let me say what I'm trying to get to, and you can tell me where your idea and mine differ.

We need one or more toolchain images per team (e.g. go, golangci-lint, pre-commit, gh), shared by several Claude and Codex agents, with their binaries on PATH. They can't be derived from the harness image, since that would need a rebuild per kagent release.

The copy is the part I'm unsure about. Skills are copied into the harness's skills directory because the model reads them from there. A toolchain is only executed, never read, so I don't see what the copy buys. And the Go toolchain alone is ~190 MB on disk. Copied, it lands in every agent that uses it, on every start, and in every snapshot, for bytes that are identical across all of them. That's why #3192 mounts it in place from the image volume with bin/ on PATH. I may be missing a reason the copy matters, so if there is one I'd be glad to know it :)

On where it lives in the spec, I'm open. I'd just like to understand the thinking behind keeping it under skills, so I build the right thing.

A few questions to get there:

Is a tool package selected per agent next to its skills, or pulled in by a skill that needs it? And is its bin path fixed, like bin/, or declared by the package?
"Existing kagent tools image": is that kagent-dev/tools, mounted into the agent so its binaries run in the workspace? And could a team point at its own image instead?
feat(harness): mount tool images read-only into the runtime #3192 already projects the image as a volume, so if skills move to volumes they could share that code. Does it make sense for tool images to go first, or do you want to land the skills change before?

Navigation Menu

Repository navigation

[FEATURE] Mount tool images into a Harness's filesystem #3190

📝 Feature Summary

❓ Problem Statement / Motivation

💡 Proposed Solution

🔄 Alternatives Considered

🎯 Affected Service(s)

📚 Additional Context

🙋 Are you willing to contribute?

Metadata

Assignees

Labels

Type

Projects

Milestone

Relationships

Development

[FEATURE] Mount tool images into a Harness's filesystem #3190

Description

📝 Feature Summary

❓ Problem Statement / Motivation

💡 Proposed Solution

🔄 Alternatives Considered

🎯 Affected Service(s)

📚 Additional Context

🙋 Are you willing to contribute?

Activity

QuentinBisson commented on Oct 14, 2026

EItanya commented on Oct 14, 2026

QuentinBisson commented on Oct 14, 2026

QuentinBisson commented on Oct 14, 2026

EItanya commented on Oct 14, 2026

QuentinBisson commented on Oct 14, 2026

dimetron commented on Oct 14, 2026

QuentinBisson commented on Oct 14, 2026

Metadata

Metadata

Assignees

Labels

Type

Projects

Milestone

Relationships

Development

Issue actions