Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
31 changes: 31 additions & 0 deletions cmd/gvisor-start-probe/ids_linux.go
Original file line number Diff line number Diff line change
@@ -0,0 +1,31 @@
//go:build linux

package main

import (
"errors"
"fmt"
"strings"
)

// containerIDs is an exact allowlist for controlled trials, not a runtime
// inventory. Never accept prefixes or derive this list from receiver output.
type containerIDs map[string]struct{}

func (ids *containerIDs) Set(id string) error {
if id == "" || len(id) > 256 || strings.ContainsAny(id, " \t\r\n") {
return errors.New("container-id must be a nonempty exact runtime ID of at most 256 bytes without whitespace")
}
if *ids == nil {
*ids = make(containerIDs)
}
(*ids)[id] = struct{}{}
return nil
}

func (ids *containerIDs) String() string { return fmt.Sprintf("%d exact runtime IDs", len(*ids)) }

func (ids containerIDs) resolves(id string) bool {
_, ok := ids[id]
return ok
}
46 changes: 46 additions & 0 deletions cmd/gvisor-start-probe/ids_linux_test.go
Original file line number Diff line number Diff line change
@@ -0,0 +1,46 @@
//go:build linux

package main

import (
"flag"
"strings"
"testing"
)

func TestContainerIDsExactMatching(t *testing.T) {
var ids containerIDs
flags := flag.NewFlagSet("trial", flag.ContinueOnError)
flags.Var(&ids, "container-id", "runtime identity")
if err := flags.Parse([]string{"--container-id", "sandbox-a", "--container-id", "sandbox-b", "--container-id", "sandbox-a"}); err != nil {
t.Fatal(err)
}
if len(ids) != 2 {
t.Fatalf("got %d identities, want 2", len(ids))
}
for _, id := range []string{"sandbox-a", "sandbox-b"} {
if !ids.resolves(id) {
t.Fatal("known ID rejected")
}
}
for _, id := range []string{"", "sandbox", "sandbox-a-suffix", "unknown"} {
if ids.resolves(id) {
t.Fatal("unverified ID accepted")
}
}
}

func TestContainerIDsRejectInvalidValues(t *testing.T) {
var ids containerIDs
for _, id := range []string{"", "with space", "with\nnewline", strings.Repeat("a", 257)} {
if err := ids.Set(id); err == nil {
t.Fatal("invalid ID accepted")
}
}
if len(ids) != 0 {
t.Fatal("invalid IDs changed allowlist")
}
if err := ids.Set(strings.Repeat("a", 256)); err != nil {
t.Fatal(err)
}
}
9 changes: 4 additions & 5 deletions cmd/gvisor-start-probe/main_linux.go
Original file line number Diff line number Diff line change
Expand Up @@ -22,9 +22,10 @@ func main() {

func run() (exitCode int) {
socket := flag.String("socket", "/run/kubescape/gvisor-events.sock", "private Unix socket for the SecCheck remote sink")
containerID := flag.String("container-id", "", "exact container ID obtained independently from the local runtime before start")
var ids containerIDs
flag.Var(&ids, "container-id", "exact container ID obtained independently from the local runtime before start; repeat for multiple containers")
flag.Parse()
if *containerID == "" {
if len(ids) == 0 {
fmt.Fprintln(os.Stderr, "container-id is required")
return 2
}
Expand All @@ -46,9 +47,7 @@ func run() (exitCode int) {
outputErrors := make(chan error, 1)
receiver := &gvisor.Receiver{
SocketPath: *socket,
Resolve: func(id string) bool {
return id == *containerID
},
Resolve: ids.resolves,
OnStart: func(ctx context.Context, start gvisor.Start) {
if err := output.Encode(ctx, start); err != nil {
if errors.Is(err, context.Canceled) || errors.Is(err, context.DeadlineExceeded) ||
Expand Down
123 changes: 123 additions & 0 deletions docs/gvisor-start-probe.md
Original file line number Diff line number Diff line change
Expand Up @@ -94,3 +94,126 @@ This document records the procedure, not results. Live `runsc` and node-agent
host eBPF observations must be added after the Linux trial; the existing
[gVisor proof of concept](https://github.com/yellow-forrest/gvisor-visibility-poc)
is useful prior work but is not validation of this receiver.

## Repeated and concurrent starts

The probe accepts repeated `--container-id` flags. Each value is matched in
full, never as a prefix. The original single-ID invocation still works.
Obtain all IDs from the local runtime before starting the selected containers.
Reuse the receiver-owned `/run/kubescape` directory (mode `0700`) and the
trace configuration from the controlled trial above. The sink endpoint, probe
and readiness check all use `/run/kubescape/gvisor-events.sock`; `TRIAL_DIR`
stores evidence only. If you choose a different socket, replace that literal
with the same absolute path in all three places. JSON does not expand shell
variables, so the trace configuration must contain the actual pathname.

Create a private evidence directory before redirecting output:

```sh
umask 077
TRIAL_DIR="$(mktemp -d)"
./gvisor-start-probe \
--socket /run/kubescape/gvisor-events.sock \
--container-id "$CONTAINER_A" \
--container-id "$CONTAINER_B" \
> "$TRIAL_DIR/starts.jsonl" 2> "$TRIAL_DIR/probe.stderr"
```

Run the following readiness check in a second terminal while the probe runs.
Socket existence alone is not readiness: the receiver binds the path
before calling `listen`. Before starting either container, require a successful
connection and version-1 handshake with a bounded retry, for example:

```sh
python3 - /run/kubescape/gvisor-events.sock <<'PYREADY'
import socket
import sys
import time

deadline = time.monotonic() + 5
while time.monotonic() < deadline:
with socket.socket(socket.AF_UNIX, socket.SOCK_SEQPACKET) as connection:
connection.settimeout(0.2)
try:
connection.connect(sys.argv[1])
connection.sendall(b"\x08\x01") # Version-1 protobuf handshake.
if connection.recv(32) == b"\x08\x01":
break
except OSError:
pass
time.sleep(0.05)
else:
raise SystemExit("receiver readiness handshake timed out")
PYREADY
```

The check closes its connection without sending a start and produces no trial
event. Abort the trial if the check fails. It only establishes that the receiver
answered; keep checking that the probe is running while the workload starts.
Stop it with SIGTERM after the workload and wait for a zero exit status before
checking artifacts. A nonzero exit invalidates that run.

For twenty repeated starts, either collect each independently created ID in
advance and repeat the flag for every ID, or run separate probe invocations
and combine only the sanitized start JSONL files. Keep a runtime-derived
expected-count manifest for exactly those invocations. For the concurrent
case use one receiver with two IDs, overlapping sandbox lifetimes, and runtime
records that demonstrate the overlap. Two sequential starts are not evidence
of concurrency. The ID flag remains a manual trial allowlist; it does not
implement the production runtime inventory join.

## Check retained evidence

Use Python 3.9 or newer. Create `expected.json` from independently recorded
runtime starts, not from the probe output. Its keys must be exact runtime IDs
and its values the expected number of starts. For example, replace these
placeholders with the two real IDs:

```json
{"exact-runtime-id-a": 1, "exact-runtime-id-b": 1}
```

Use a harmless marker matching `[A-Za-z0-9_-]{16,}`, such as
`GVS_TRIAL_CANARY_2026`, in the root and child container argv and cwd fixtures.
Only ASCII letters, digits, underscores and hyphens are accepted, so ordinary
JSON string encoding leaves the marker literal. Quotes, backslashes, whitespace
and non-ASCII markers are rejected before reading evidence. Do not use real
credentials. After the probe has stopped, check each run:

```sh
python3 scripts/check-gvisor-start-trial.py \
--expected "$TRIAL_DIR/expected.json" \
--events "$TRIAL_DIR/starts.jsonl" \
--canary GVS_TRIAL_CANARY_2026 \
--artifact "$TRIAL_DIR/probe.stderr"
```

Repeat `--artifact` for additional retained logs or exports. The checker
rejects missing or excess start counts, unknown identities, unexpected event
fields, invalid
source/timestamps, blank JSONL records, nonzero sender-reported drop counts,
and the canary in the specified artifacts. Each input file is limited to
16 MiB. It prints neither
input records nor parser error payloads. It does not inspect unspecified
files, prove that the canary was actually used, verify the manifest's origin,
measure local queue drops, or prove runtime identity from a JSON file. JSON
record counts cannot prove timing, concurrency, or per-connection isolation.
If a manifest expects two starts for one ID, two identical records satisfy that
count; the checker does not authenticate events or distinguish replayed copies
from distinct starts. Independent runtime evidence is still required.
A passing check is an artifact consistency check, not a lifecycle integration
gate or proof that application code executed.

Fill in [the trial record](gvisor-start-trial-record.md) with version details,
runtime ground truth, privacy checks, host eBPF observations, and limitations.
Keep the synthetic fixture separate from sanitized evidence. Debug/strace
output can itself include argv or cwd; inspect it privately and sanitize it
before retaining or sharing. The checker can detect a known marker, not every
possible secret or encoding. A failed privacy check means the artifacts must
not be published.

The checker can be tested without `runsc`:

```sh
python3 -m unittest discover -s scripts -p test_check_gvisor_start_trial.py
```
71 changes: 71 additions & 0 deletions docs/gvisor-start-trial-record.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,71 @@
# gVisor start trial record

**Status: NOT RUN.** This is a template for the live experiment from
[design #23](https://github.com/kubescape/designs-and-proposals/pull/23).
Neither the merged receiver nor the offline checker establishes live coverage.
Copy this record for a real trial and replace pending entries with measured
results and sanitized evidence references. Do not mark a gate passed from a
synthetic sender test or an offline checker result alone.

## Environment and ownership

| Item | Recorded value |
|---|---|
| Trial date and operator | Pending |
| Linux distribution, kernel and architecture | Pending |
| Exact runsc version/commit and binary checksum | Pending |
| containerd and CRI versions | Pending |
| node-agent version/commit and configuration | Pending |
| Probe version/commit | Pending |
| Runtime root and namespace | Pending |
| Sandbox/root/child ID relationship from runtime inventory | Pending |
| Selected trace metadata and initialization configuration | Pending |
| Receiver owner, socket permissions and runtime access | Pending |
| Owner of Default trace session, conflict behavior | Pending |

Record actual commands against the pinned runtime and any deviations from
[the procedure](gvisor-start-probe.md). Configure tracing before startup and
never overwrite another monitor's session with `--force`. Trace configuration
is host setup, not something this branch installs.

## Trial gates

| Scenario | Expected evidence | Result |
|---|---|---|
| Ordinary container baseline | Runtime IDs/times and host process/network observations | Pending |
| One gVisor container | Independently recorded runtime ID/times, trace start and host comparison | Pending |
| Twenty repeated starts | Expected vs observed per-ID counts, 20/20 starts without unexplained misses | Pending |
| Two concurrent sandboxes | Overlapping runtime lifetimes, distinct IDs/connections, no identity crossover | Pending |
| Root and child privacy fixtures | Canary used in both argv and cwd; absent from retained/forwarded output and errors | Pending |
| Receiver absent and slow | Startup impact, retry delay, sender/local loss, bounded resources | Pending |
| Receiver restart | Trace-session ownership, recovery behavior and any missed early starts | Pending |
| Normal exit and forced termination | Runtime state vs trace transport; no stop inferred from disconnect alone | Pending |
| Controlled outbound, refused connection and DNS | Compare runtime/trace/host evidence and name only the observed behavior | Pending |

## Sanitized evidence and interpretation

- Runtime-derived expected manifest and provenance: pending.
- Probe exit status, checker invocation and result: pending.
- Sanitized observation files, including stderr and privacy review: pending.
- Versioned workload and harmless canary fixture: pending.
- Expected/observed starts and timing comparison: pending.
- Sender-reported loss and separately measured local loss: pending.
- Host eBPF comparison, including what could not be observed: pending.
- Startup latency and CPU/memory measurements with method: pending.
- Failures, missing events and unexplained differences: pending.

The start hook precedes application execution. Receiver timestamps are
observation times, not runtime start timestamps. Correlating exact IDs does
not establish actor/session identity or Kubernetes attribution. The current
probe uses a manually supplied allowlist, not node-agent's runtime inventory.

## Decision before integration

State which gates passed, failed or remain untested, with supporting evidence.
Unexplained missed starts, identity ambiguity or privacy failures block the
verified-start integration. Source-health gaps must be recorded and addressed
before claiming reliable collection. Network and stop integration require
separate source validation. Managed GKE remains unverified unless its runtime
configuration and socket access are independently demonstrated.

**Current decision: pending live Linux trial.**
Loading
Loading