diff --git a/_docs/32-namespaces-and-cgroups.md b/_docs/32-namespaces-and-cgroups.md new file mode 100644 index 0000000..4e42a8f --- /dev/null +++ b/_docs/32-namespaces-and-cgroups.md @@ -0,0 +1,742 @@ +--- +title: "Namespaces and cgroups" +order: 32 +part: "Containers and Virtualization" +description: "The eight Linux namespaces and cgroup v2's unified hierarchy as the two primitives every container runtime assembles from — unshare vs setns vs clone3, the pid-1 reparenting consequence, memory.max enforced by a real OOM kill, and the genuine ENOMEM Go hits calling unshare(CLONE_NEWPID) directly from a threaded runtime." +duration: "55 minutes" +--- + +Every `pmon` version so far has supervised, signaled, and identified a +child, but the child has always lived in the same world as its parent: same +process-id space, same hostname, same view of every device and byte of +RAM on the box. Real container runtimes give a process a *different* +world — its own pid 1, its own hostname, and a hard ceiling on memory and +CPU — without a hypervisor anywhere in sight. The one new idea this +chapter teaches is that "container" is not a kernel object at all: it is +two independent primitives, **namespaces** (what a process can *see*) and +**cgroups** (what a process is *allowed to use*), wired together by +whatever userspace tool assembles them. `pmon containerize` is that tool, +built by hand, with nothing between the syscalls and the terminal. + +The code is in `examples/32-namespaces-and-cgroups/`. This is a **VM +example** — namespace and cgroup writes need real root and a real kernel, +so `./demo.sh run` deploys to `systems-target` when `TARGET` is +set; the `README.md` there covers the CLI and the paired positive/negative +verification in full. + +{% include excalidraw.html + file="32-namespace-overview" + alt="Two columns, host and pmon containerize's PID-1 child, with eight namespace-type rows: PID, mount, UTS, network, time, IPC, user, cgroup. The first four rows are amber boxes with an arrow marked 'differs' between real /proc/<pid>/ns/* inode numbers — pid 4026531836 vs 4026532530, mount 4026531832 vs 4026532528, UTS 4026531838 vs 4026532529, network 4026531833 vs 4026532531 — read on systems-target. The last four rows are grey dashed boxes marked 'shared with host' with identical inode numbers on both sides — time 4026531834, IPC 4026531839, user 4026531837, cgroup 4026531835 — because pmon containerize only requests CLONE_NEWNS|CLONE_NEWUTS|CLONE_NEWNET|CLONE_NEWPID and deliberately leaves the other four alone." + caption="Figure 32.1 — the eight Linux namespaces, host vs pmon's PID-1 child, every inode number read live from /proc/<pid>/ns/* on systems-target" %} + +> **Tools used** — `ssh`/`scp` and the Python runner +> (`scripts/test-all-examples.py`) on the host; `readlink`, `cat`, `pgrep`, +> `nsenter`, `dmesg`, `strace` (systems-target VM). Everything here ships +> with Fedora or is preinstalled in the lab VM by cloud-init. + +## Eight namespaces, one job each + +A Linux namespace does not hide anything from the kernel — the kernel +always has the complete, global view. It hides things from the *process*: +each namespace type wraps one kind of global kernel state so a process +inside it sees only its own slice. + +- **PID** — a private process-id tree. The first process created inside a + new PID namespace is *always* pid 1 of that namespace, no matter what + its "real" (host-visible) pid is. +- **Mount** — a private copy of the mount table. Mounting or unmounting + inside the namespace does not touch the host's view, once propagation is + detached (below). +- **UTS** — the hostname and NIS domain name. One `sethostname(2)` call, + fully isolated. +- **Network** — its own loopback, interfaces, routes, and port space; a + bare network namespace starts with only `lo`, down. +- **IPC** — its own System V IPC objects and POSIX message queues. +- **User** — its own uid/gid mapping, letting an unprivileged host user + become "root" inside the namespace over a mapped range — the one + namespace `pmon containerize` does *not* request, discussed below. +- **Cgroup** — virtualizes which cgroup a process believes is its root, + so `/proc/self/cgroup` reads `/` from inside even when the real path is + nested five levels deep on the host. +- **Time** (Linux 5.6+) — lets a namespace offset `CLOCK_MONOTONIC` and + `CLOCK_BOOTTIME`, the one facility built specifically for checkpoint/ + restore rather than isolation. + +`pmon containerize` requests exactly four: `CLONE_NEWNS | CLONE_NEWUTS | +CLONE_NEWNET | CLONE_NEWPID`. Figure 32.1 is the receipt: every inode +number in it came from `readlink /proc//ns/*` on `systems-target` +during this chapter's own run, and the four requested types read a +*different* inode on the child than on the host's pid 1, while the four +left alone — time, IPC, user, cgroup — read the *same* inode on both +sides. A namespace you didn't ask for is not a weaker namespace; it is no +namespace at all, just the host's, shared. + +## unshare, setns, and clone3 — three ways into a namespace + +The three syscalls answer three different questions: + +- **`unshare(2)`** — "take *me*, right now, out of my current namespaces + and into brand-new ones." For most types this applies to the calling + process/thread immediately. `CLONE_NEWPID` is the deliberate exception: + unshare(2) can never move an already-running process into a new PID + namespace (a process's own pid can't change mid-life), so it only + arranges that the *next* child the caller creates becomes pid 1 of a + fresh one. That single asymmetry is why every implementation in this + chapter unshares first and forks second, not the other way around. +- **`setns(2)`** — "join a namespace that already exists," given an open + file descriptor on one of the `/proc//ns/*` links. This is what + `nsenter` does under the hood: open the target's namespace fds, `setns` + into each, then fork and exec the requested command. Like `unshare`, + `setns(CLONE_NEWPID)` only takes effect for children the caller creates + *after* the call — the caller's own pid never moves. +- **`clone3(2)`** — create the child and select its namespaces in the + *same* syscall, via `struct clone_args.flags`. This is the one true + atomic option: there is no window where the new task exists but hasn't + yet joined its namespaces. `pmon`'s C++ build calls it directly (below); + Go's fix reaches the same atomicity indirectly, through the ordinary + `clone(2)` flags word the runtime already issues for every process it + spawns. + +## The pid-1 consequence + +Becoming pid 1 of a new namespace is not a label, it is a job with two +real obligations the kernel enforces. First, **reparenting**: inside a +PID namespace, an orphaned process is reparented to namespace-local pid 1, +not to the host's real init — so pid 1 must be prepared to `wait()` for +processes it never directly spawned, exactly the way real init is. +Second, **reaping**: default signal dispositions are altered for pid 1 — +an unhandled `SIGTERM`/`SIGKILL` that would kill any other process is +ignored by pid 1 unless it installs its own handler. `pmon containerize` +sidesteps both obligations by design: its pid-1 child immediately +`execve`s CMD, so CMD — not pmon's own code — inherits the pid-1 role for +whatever it does next. That is a real, stated simplification, not an +oversight: a supervisor that expects to run *multiple* processes inside +one PID namespace (the way `podman run` with more than one process, or a +process manager container, would) needs an init that actually reaps — +which is exactly what chapters 11 and 12 already built, just not wired to +this namespace yet. + +## Cgroup v2: the unified hierarchy + +Cgroup v1 let different controllers (cpu, memory, …) organize processes +into *different* tree shapes — a process could be in one cpu group and an +unrelated memory group. Cgroup v2 unifies this: **one tree**, and every +controller you enable sees the same groups. `systems-target` mounts it at +the conventional single point: + +```bash +[vm]$ mount | grep cgroup +cgroup2 on /sys/fs/cgroup type cgroup2 (rw,nosuid,nodev,noexec,relatime,seclabel,nsdelegate,memory_recursiveprot,memory_hugetlb_accounting) +[vm]$ cat /sys/fs/cgroup/cgroup.controllers +cpuset cpu io memory hugetlb pids rdma misc dmem +[vm]$ cat /sys/fs/cgroup/cgroup.subtree_control +cpu memory pids +``` + +Three files matter for `containerize`. **`memory.max`** is a hard byte +ceiling — cross it and the kernel's OOM killer runs *inside this cgroup +only*, picking a victim from the cgroup's own processes regardless of +system-wide memory pressure. **`cpu.max`** is written as `"$QUOTA +$PERIOD"` microseconds — `"max 100000"` (pmon's default) means unthrottled +inside a 100 ms accounting period; a real ceiling would read e.g. +`"50000 100000"` for half a core. A sibling knob `containerize` doesn't +set, worth knowing: **`cpu.weight`** (default 100, range 1–10000) doesn't +cap anything — it only decides who wins when several cgroups *compete* +for the same CPU, the cgroup analogue of `nice`. `memory.max` and +`cpu.max` answer "how much," `cpu.weight` answers "whose turn," and a real +container runtime writes all three. + +The fourth file is the pressure signal: **PSI** (Pressure Stall +Information), `memory.pressure`, is not a counter of *how much* memory a +cgroup used — it is how much time tasks in this cgroup spent *stalled* +waiting for memory, per-cgroup, independent of pressure anywhere else on +the box: + +```bash +[vm]$ cat /sys/fs/cgroup/pmon-memcheck/memory.pressure +some avg10=0.00 avg60=0.00 avg300=0.00 total=0 +full avg10=0.00 avg60=0.00 avg300=0.00 total=0 +``` + +`pmon containerize` reads the `some avg10=` field after CMD exits and +prints it as `pmon: cgroup mem.pressure some=` — a real number, not a +placeholder, confirming the kernel is actually accounting this cgroup, not +just enforcing a limit blindly. + +One subtlety governs the whole design: a *non-root* cgroup cannot both +hold processes directly and delegate a controller to its children (the +"no internal process" constraint) — but the **root** cgroup is explicitly +exempt. That is why `setup_cgroup()` can write `+memory +cpu` into +`/sys/fs/cgroup/cgroup.subtree_control` without first relocating `pmon` +itself into a leaf: the root is the one place in the tree allowed to hold +processes and delegate at the same time. + +## What a container runtime actually assembles + +Strip away the marketing and a container is five primitives, wired +together by one userspace program: **namespaces** (this chapter, minus +user — `containerize` never requests `CLONE_NEWUSER`, so its "root" inside +the container is genuinely uid 0 on the host, which is why the demo needs +real root rather than rootless mapped uids), **cgroups** (this chapter), +a **rootfs** (a different filesystem view via `pivot_root`/bind mounts — +chapter 34 builds this), **capabilities** (chapter 14's five sets, +dropped to a minimal set before `exec`), and a **seccomp/Landlock** +filter (chapter 33, next). `pmon containerize` deliberately stops after +the first two, on the same host filesystem CMD would otherwise see — the +point is to make namespaces and cgroups *visible* as syscalls you could +write yourself, before chapter 34 shows what podman assembles from all +five at once inside a real multi-stage Containerfile. + +{% include excalidraw.html + file="32-cgroup-hierarchy" + alt="The cgroup v2 unified hierarchy from root down to a leaf. The root band shows cgroup.controllers listing cpuset, cpu, io, memory, hugetlb, pids, rdma, misc, dmem, and cgroup.subtree_control enabling plus memory plus cpu, delegated to children. An arrow labeled enables leads down into the pmon-oomdemo leaf band, showing memory.max as 67108864 (64 MiB), memory.swap.max as 0 (no swap headroom), cpu.max as max 100000 (unthrottled), cgroup.procs holding pmon's own pid written before fork, and memory.pressure PSI reading some avg10=0.00. A dark box below states the hog process (sh, real anon-rss 64384kB) writes past the 64 MiB ceiling. An arrow labeled 'OOM killer fires' leads into a kernel band showing the real dmesg lines from this session: oom-kill constraint=CONSTRAINT_MEMCG oom_memcg=/pmon-oomdemo task=sh pid=27013, and 'Memory cgroup out of memory: Killed process 27013 (sh) anon-rss:64384kB', with pmon's own report alongside it: signal=9 (KILL), exit 137." + caption="Figure 32.2 — the cgroup v2 unified hierarchy, root to leaf, ending in the real dmesg OOM-kill line this session produced" %} + +## How the code works + +All three languages share one shape: build the cgroup, unshare/clone into +fresh namespaces, become pid 1, exec CMD, then the parent waits and +reports the cgroup-side evidence. `setup_cgroup`/`setupCgroup` (identical +logic across C++, Go, and Rust — read `cgroup.subtree_control`, add +`+memory`/`+cpu` as whole tokens only if missing so `"cpu"` never +false-matches inside `"cpuset"`, `mkdir` the leaf, write `memory.max` then +`cpu.max` then a best-effort `memory.swap.max=0`, then write pmon's own +pid into `cgroup.procs`) runs **before** any namespace call in every +language, on purpose: cgroup membership is inherited across `fork`/ +`clone`/`exec`, so writing pmon's pid first means every later child — the +pid-1 task, and CMD after it execs — is under the limits from its very +first instruction, never after. + +{% include codetabs.html langs="C++|Go|Rust" %} + +```cpp + // Mount, UTS and network namespaces take effect on THIS process right now. + // The PID namespace is deferred: the next clone3(2) child becomes PID 1 of + // a fresh one; we (the caller) stay in our original PID namespace so we + // can waitpid(2) the child the ordinary way. + if (auto r = checked(::unshare(CLONE_NEWNS | CLONE_NEWUTS | CLONE_NEWNET | CLONE_NEWPID), + "unshare"); + !r) { + std::println(stderr, "containerize: {}: {}", r.error().what, std::strerror(r.error().err)); + return 1; + } + + // Detach mount propagation recursively so nothing we do in this namespace + // (or CMD does) leaks a mount event back to the host's mount table. + if (auto r = checked(::mount("none", "/", nullptr, MS_REC | MS_PRIVATE, nullptr), + "mount MS_PRIVATE"); + !r) { + std::println(stderr, "containerize: {}: {}", r.error().what, std::strerror(r.error().err)); + return 1; + } + + // sethostname(2) now lands in the fresh UTS namespace, not the host's. + if (auto r = checked(::sethostname(hostname.c_str(), hostname.size()), "sethostname"); !r) { + std::println(stderr, "containerize: {}: {}", r.error().what, std::strerror(r.error().err)); + return 1; + } + + // clone3(2): flags=0 (the namespace work is already done above), stack=0 + // (kernel copy-on-write duplicates our stack exactly like fork(2)), + // exit_signal=SIGCHLD (so waitpid(2) below works the ordinary way). + clone_args_v0 cl{}; + cl.exit_signal = SIGCHLD; + long rc = ::syscall(SYS_clone3, &cl, sizeof(cl)); + if (rc < 0) { + std::println(stderr, "containerize: clone3: {}", std::strerror(errno)); + return 1; + } + + if (rc == 0) { + // --- child: PID 1 of the new PID namespace --- + if (::getpid() == 1) { + std::println("pmon: child sees pid 1"); + } + std::fflush(stdout); + + char buf[256]{}; + if (::gethostname(buf, sizeof(buf) - 1) == 0) { + std::println("pmon: hostname={}", buf); + } + std::fflush(stdout); + + std::vector argv(cmd.begin(), cmd.end()); + argv.push_back(nullptr); + ::execvp(argv[0], argv.data()); + std::println(stderr, "pmon: exec {}: {}", argv[0], std::strerror(errno)); + std::fflush(stderr); + _exit(127); + } + + // --- parent: waits, then reports the cgroup-side evidence --- + int status = 0; + if (::waitpid(static_cast(rc), &status, 0) < 0) { + std::println(stderr, "containerize: waitpid: {}", std::strerror(errno)); + return 1; + } + + auto psi = read_file(cgroup_path + "/memory.pressure"); + if (psi) { + std::println("pmon: cgroup mem.pressure some={}", parse_psi_some_avg10(*psi)); + } + + int exit_code = 0; + if (WIFEXITED(status)) { + int code = WEXITSTATUS(status); + std::println("pmon: child exited status={}", code); + exit_code = code; + } else if (WIFSIGNALED(status)) { + int sig = WTERMSIG(status); + std::println("pmon: child killed signal={} ({})", sig, signal_name(sig)); + exit_code = 128 + sig; + } +``` + +```go + self, err := os.Executable() + if err != nil { + fmt.Fprintf(os.Stderr, "containerize: os.Executable: %v\n", err) + return 1 + } + // The hostname has to travel to the re-exec'd child: it, not this + // process, is the one that ends up inside the fresh UTS namespace (see + // the package comment). + initArgs := append([]string{reexecMarker, hostname, "--"}, cmd...) + c := exec.Command(self, initArgs...) + c.Stdin, c.Stdout, c.Stderr = os.Stdin, os.Stdout, os.Stderr + // Cloneflags asks the runtime's own fork/exec trampoline to create the + // child already inside fresh mount/uts/net/pid namespaces, atomically, + // in the SAME clone(2) that creates the process — no separate unshare(2) + // on our side, and so none of the "one child per fresh PID namespace" + // restriction that broke the direct-unshare attempt. + c.SysProcAttr = &syscall.SysProcAttr{ + Cloneflags: syscall.CLONE_NEWNS | syscall.CLONE_NEWUTS | syscall.CLONE_NEWNET | syscall.CLONE_NEWPID, + } + if err := c.Start(); err != nil { + fmt.Fprintf(os.Stderr, "containerize: exec %s: %v\n", cmd[0], err) + return 1 + } + + waitErr := c.Wait() + + if psi, err := os.ReadFile(cgroupPath + "/memory.pressure"); err == nil { + fmt.Printf("pmon: cgroup mem.pressure some=%s\n", parsePSISomeAvg10(string(psi))) + } + + exitCode := 0 + switch { + case waitErr == nil: + fmt.Println("pmon: child exited status=0") + default: + var ee *exec.ExitError + if errors.As(waitErr, &ee) { + ws := ee.Sys().(syscall.WaitStatus) + if ws.Signaled() { + sig := int(ws.Signal()) + fmt.Printf("pmon: child killed signal=%d (%s)\n", sig, signalName(sig)) + exitCode = 128 + sig + } else { + exitCode = ws.ExitStatus() + fmt.Printf("pmon: child exited status=%d\n", exitCode) + } + } else { + fmt.Fprintf(os.Stderr, "containerize: wait: %v\n", waitErr) + exitCode = 1 + } + } + + _ = os.Remove(cgroupPath) // best-effort; a leftover cgroup is harmless + + return exitCode +``` + +```rust + // Mount, UTS and network namespaces take effect on THIS process right + // now. The PID namespace is deferred: the next fork(2) child becomes + // PID 1 of a fresh one; we (the caller) stay in our original PID + // namespace so we can waitpid(2) the child the ordinary way. + if let Err(e) = unshare( + CloneFlags::CLONE_NEWNS + | CloneFlags::CLONE_NEWUTS + | CloneFlags::CLONE_NEWNET + | CloneFlags::CLONE_NEWPID, + ) { + eprintln!("containerize: unshare: {e}"); + return 1; + } + + // Detach mount propagation recursively so nothing we (or CMD) do in + // this namespace leaks a mount event back to the host's mount table. + if let Err(e) = mount( + Some("none"), + "/", + None::<&str>, + MsFlags::MS_REC | MsFlags::MS_PRIVATE, + None::<&str>, + ) { + eprintln!("containerize: mount MS_PRIVATE: {e}"); + return 1; + } + + // sethostname(2) now lands in the fresh UTS namespace, not the host's. + if let Err(e) = sethostname(&hostname) { + eprintln!("containerize: sethostname: {e}"); + return 1; + } + + // SAFETY: the child performs only async-signal-safe operations + // (getpid, gethostname, writes to already-open stdio fds, execvp) + // before it either execs or _exits — it never returns up through this + // stack frame into arbitrary Rust code, satisfying fork(2)'s contract. + match unsafe { fork() } { + Ok(ForkResult::Child) => { + if getpid().as_raw() == 1 { + println!("pmon: child sees pid 1"); + } + if let Ok(hn) = gethostname() { + println!("pmon: hostname={}", hn.to_string_lossy()); + } + let _ = std::io::stdout().flush(); + + let prog = CString::new(cmd[0].as_str()).unwrap_or_default(); + let cargs: Vec = cmd + .iter() + .map(|s| CString::new(s.as_str()).unwrap_or_default()) + .collect(); + // execvp never returns on success. + let err = execvp(&prog, &cargs).unwrap_err(); + eprintln!("pmon: exec {}: {err}", cmd[0]); + std::process::exit(127); + } + Ok(ForkResult::Parent { child }) => { + let status = waitpid(child, None); + + if let Ok(psi) = fs::read_to_string(format!("{cgroup_path}/memory.pressure")) { + println!( + "pmon: cgroup mem.pressure some={}", + parse_psi_some_avg10(&psi) + ); + } + + let exit_code = match status { + Ok(WaitStatus::Exited(_, code)) => { + println!("pmon: child exited status={code}"); + code + } + Ok(WaitStatus::Signaled(_, sig, _)) => { + let n = sig as i32; + println!("pmon: child killed signal={n} ({})", signal_name(n)); + 128 + n + } + Ok(_) => 0, + Err(e) => { + eprintln!("containerize: waitpid: {e}"); + 1 + } + }; + + let _ = fs::remove_dir(&cgroup_path); // best-effort cleanup + + exit_code + } + Err(e) => { + eprintln!("containerize: fork: {e}"); + 1 + } + } +``` + +C++ and Rust get there through the same shape — `unshare` first, taking +mount/UTS/network effect immediately on the caller; the mount-private +remount so nothing propagates back to the host; `sethostname`; then a +fork-shaped child creation (`clone3(flags=0)` in C++, `nix::fork` in +Rust — both give "no code runs between clone and exec except what we +wrote," fork(2)'s classic contract) whose child becomes pid 1 of the +deferred PID namespace and immediately prints its two proof lines before +`execvp`. The parent side is identical in shape across both: `waitpid`, +read `memory.pressure`, print the mem-pressure line, then translate the +wait status into `pmon: child exited status=` or `pmon: child killed +signal= ()` with a hand-rolled signal-name table (not +`strsignal(3)`/`Signal::from_raw`'s locale-or-library-dependent prose, +reused verbatim from every earlier `pmon` chapter) and the `128+n` +convention. Go's excerpt is the odd one out and is the subject of the +Concurrency lens below. + +**Fragile bits, stated plainly**: `containerize` never remounts a fresh +`/proc` inside the new mount namespace, so tools that read `/proc` from +inside — including `ps` run via `nsenter` — still see the host's process +table (the cross-check below demonstrates this directly); a real rootfs +container fixes this with `mount -t proc proc /proc` after `pivot_root`, +which is chapter 34's job, not this one's. `memory.swap.max` is written +best-effort because not every kernel exposes swap accounting identically. +And `deploy-to-vm.sh` rebuilds the remote command line with a bare `$*`, +which flattens quoting — every `CMD` in this chapter's runs is therefore +space-free tokens (`/usr/bin/echo hi`, `sh /tmp/hog.sh`), with anything +needing real shell logic staged as a script file on the guest first. + +## Errors, three ways + +The exit-code contract is identical across all three binaries, and it +splits by *when* the kernel or pmon rejects something. A **usage** error +(exit 2) is caught before any privileged call — a missing `-- CMD`: + +```bash +[host]$ ./app containerize --hostname x +containerize: missing -- CMD +``` + +A **setup** error (exit 1) is a syscall that failed before CMD ever ran. +The ordinary case is the privilege check — `containerize: must run as +root` when invoked as a normal user, real output from this session, +unprivileged, on the reference host. The instructive case is the one Go's +first implementation actually produced on `systems-target`, verbatim: + +``` +containerize: exec /usr/bin/echo: fork/exec /home/fedora/app: cannot allocate memory +``` + +That is `ENOMEM` from a real `clone(2)` call, captured live — the +Concurrency lens below explains exactly why. Finally, a **child-side** +error is CMD's own fate, reported through the same wait-status +translation in every language: the OOM negative control's `pmon: child +killed signal=9 (KILL)` becomes exit `137` (`128 + 9`), and `pmon` +propagates it unchanged rather than collapsing it to a generic failure — +the caller can tell "CMD chose to fail" from "CMD was killed" from the +exit code alone, no log-scraping required. + +## Concurrency lens + +{% include excalidraw.html + file="32-clone3-flow" + alt="Three lanes. The left lane, C++/Rust, shows unshare taking effect on the caller, then clone3 with flags=0 (fork semantics) creating a child that prints its pid/hostname proof lines and execs CMD — labeled PASS, real run this session. The middle lane, Go's first attempt, shows runtime.LockOSThread pinning the goroutine then a raw unshare call, followed by os/exec.Start issuing an internal vfork/pidfd probe fork, then the real fork for CMD failing with clone returning -1 ENOMEM and the message fork/exec: cannot allocate memory — real, this session. The bottom lane, Go's shipped fix, shows setup_cgroup writing its own pid first same as the other two languages, then exec.Command with SysProcAttr.Cloneflags carrying CLONE_NEWNS|CLONE_NEWUTS|CLONE_NEWNET|CLONE_NEWPID, one clone(2) call creating the child already inside all four namespaces with no prior unshare and no LockOSThread needed, the child re-execing itself as __ns_init__ to mount / MS_PRIVATE, sethostname, print the proof lines, and exec CMD as PID 1 — labeled PASS 13/13 across cpp, go, rust on systems-target, this session. A note explains the ENOMEM is a real, kernel-documented restriction: unshare(CLONE_NEWPID) allows the caller to create exactly one child in the fresh namespace before further forks there return ENOMEM, and os/exec's own vfork/pidfd fast path performs an internal probe fork that silently spends that one shot." + caption="Figure 32.3 — C++/Rust's unshare-then-clone3 shape beside Go's two attempts: a real ENOMEM, then the SysProcAttr.Cloneflags fix that passes" %} + +Namespace changes made by `unshare(2)`/`setns(2)` are per-**thread** +kernel state, not per-process — the same fact chapter 14 hit with +capabilities. A Go program is multithreaded before `main` even runs, and +the goroutine scheduler is free to migrate any goroutine onto any OS +thread between two lines of Go code. The textbook answer is +`runtime.LockOSThread()`: pin the calling goroutine to its current OS +thread so a subsequent raw `unshare()` and the fork/exec that must land on +the *same*, now-unshared thread can't be split across two different ones. +That is exactly what this chapter's Go build tried first — `LockOSThread`, +then `unix.Unshare(CLONE_NEWNS|CLONE_NEWUTS|CLONE_NEWNET|CLONE_NEWPID)`, +then re-exec itself via `os/exec` so the fork would land on the pinned +thread. It built. Its pid/hostname proof lines were real. And it failed +the memory-hog negative control on real hardware, every time, with exit 1 +and `fork/exec /home/fedora/app: cannot allocate memory` — not a typo, not +a flaky VM, a `strace` on `systems-target` shows the actual syscall: + +``` +[vm]$ sudo strace -f -e trace=clone,unshare,execve -o /tmp/go-trace.log \ + /home/fedora/app containerize --hostname pmon-vm-check -- /usr/bin/echo cmd-ran +containerize: exec /usr/bin/echo: fork/exec /home/fedora/app: cannot allocate memory +``` +``` +24652 unshare(CLONE_NEWNS|CLONE_NEWUTS|CLONE_NEWPID|CLONE_NEWNET) = 0 +24652 clone(child_stack=NULL, flags=CLONE_VM|CLONE_PIDFD|CLONE_VFORK, parent_tid=[7]) = 24657 +24657 +++ exited with 0 +++ +24652 clone(child_stack=NULL, flags=CLONE_VM|CLONE_PIDFD|CLONE_VFORK|SIGCHLD, parent_tid=...) = -1 ENOMEM (Cannot allocate memory) +``` + +`unshare` genuinely succeeds. The first `clone` after it — a vfork/pidfd +probe `os/exec` performs internally on first use, unrelated to our +request — also succeeds and exits immediately, and in doing so becomes +the fresh PID namespace's pid 1. That is the real, kernel-documented +restriction: `unshare(CLONE_NEWPID)` lets the caller create exactly *one* +child in the new namespace; every fork after that one returns `ENOMEM`. +LockOSThread solved the problem it was built for — thread affinity — and +was powerless against this one, because this failure has nothing to do +with which thread is running: it is about how many times the *same* +thread group is allowed to fork into a namespace it merely unshared +rather than was born into. + +The fix is the same move chapter 14 made for capabilities: stop calling +the namespace syscall directly from live, threaded Go code, and describe +it declaratively instead. `exec.Cmd.SysProcAttr.Cloneflags` asks the +runtime's own fork/exec trampoline to create the namespaces **and** the +real child in one `clone(2)` call — there is no separate `unshare()` +anywhere, so there is no "used up my one child" state to exhaust, and +`LockOSThread` becomes unnecessary because there is no longer a +namespace-mutating syscall for a goroutine to be scheduled away from. The +general rule, twice proven now across two different kernel primitives: +whenever Go needs to change something that is per-thread and +irreversible, push the whole operation into the runtime's fork/exec +trampoline via `SysProcAttr` rather than performing it by hand on a +pinned goroutine — the trampoline is the one place in a Go binary +guaranteed to run alone, between one `clone(2)` and the `execve` right +after it. + +## Build, run, observe + +```bash +[host]$ LIBVIRT_DEFAULT_URI=qemu:///system \ + python3 scripts/test-all-examples.py --only 32-namespaces-and-cgroups --mode vm +``` + +On this host with the lab up, the runner built all three languages and +verified each against `systems-target` (Fedora 44, kernel +6.19.10-300.fc44.x86_64): + +``` +example cpp go rust +32-namespaces-and-cgroups PASS PASS PASS + +3 passed, 0 failed, 0 skipped +``` + +Each language's verify run passed all 13 assertions +(`PASS 13 / FAIL 0`). By hand, the positive run — a trivial CMD proves +both the PID and UTS namespaces at once: + +```bash +[host]$ export LIBVIRT_DEFAULT_URI=qemu:///system TARGET=systems-target SUDO=1 +[host]$ ./demo.sh cpp run containerize --hostname pmon-vm-check -- /usr/bin/echo cmd-ran +→ running on systems-target (Ctrl-C to stop): +pmon: child sees pid 1 +pmon: hostname=pmon-vm-check +cmd-ran +pmon: cgroup mem.pressure some=0.00 +pmon: child exited status=0 +``` + +And the negative control — a real memory hog (`head -c 200000000 +/dev/zero | tr '\0' x`, held in a shell variable so it can't be swapped or +paged away, staged as a guest script since `deploy-to-vm.sh` would mangle +its embedded pipe) under `--mem-max 67108864` (64 MiB, far below the +~190 MiB it tries to hold): + +```bash +[host]$ ./demo.sh cpp run containerize --hostname pmon-oomdemo --mem-max 67108864 --cgroup pmon-oomdemo -- sh /tmp/hog.sh +pmon: child sees pid 1 +pmon: hostname=pmon-oomdemo +pmon: cgroup mem.pressure some=0.00 +pmon: child killed signal=9 (KILL) +[host]$ echo $? +137 +``` + +The hog never reaches its own trailing `echo should-not-print` in any of +the three languages — proof the cgroup terminated it mid-allocation, +not merely slowed it down. Go, Rust, and C++ all produce byte-identical +output and the same exit 137; that identity is exactly what +`verify.lua`'s 13 assertions per language check. + +## Cross-check on systems-target + +Three independent tools confirm the same two facts Figure 32.1 claims — +that the namespaces are real, and that the missing `/proc` remount has a +real, visible consequence. First, the namespace identities themselves, +read directly while a long-lived `containerize` child is alive: + +```bash +[vm]$ sudo /home/fedora/app containerize --hostname pmon-box --cgroup pmon-crosscheck -- sleep 60 & +[vm]$ CHILD=$(pgrep -f '^sleep 60$') +[vm]$ readlink /proc/1/ns/pid /proc/1/ns/mnt /proc/1/ns/uts /proc/1/ns/net +pid:[4026531836] +mnt:[4026531832] +uts:[4026531838] +net:[4026531833] +[vm]$ sudo readlink /proc/$CHILD/ns/pid /proc/$CHILD/ns/mnt /proc/$CHILD/ns/uts /proc/$CHILD/ns/net +pid:[4026532530] +mnt:[4026532528] +uts:[4026532529] +net:[4026532531] +``` + +Every one of the four requested namespaces reads a different inode than +the host's pid 1. The three namespaces `containerize` never requests read +*identical* inodes on both sides — `time:[4026531834]`, +`ipc:[4026531839]`, `user:[4026531837]`, `cgroup:[4026531835]`, on host +and child alike — confirming those are genuinely shared, not silently +isolated by some other mechanism. + +Second, `nsenter` — the `setns(2)`-based tool — demonstrates the missing +`/proc` remount directly instead of asserting it: + +```bash +[vm]$ sudo nsenter -t $CHILD -p -m -u -n ps -ef | head -3 +UID PID PPID C STIME TTY TIME CMD +root 1 0 0 14:42 ? 00:00:06 /usr/lib/systemd/systemd --switched-root... +root 2 0 0 14:42 ? 00:00:00 [kthreadd] +``` + +`nsenter` genuinely `setns`'d into the child's pid/mount/uts/net +namespaces before forking `ps` — but `ps` reads `/proc`, and the mount +namespace it entered never got a *fresh* procfs mounted after `unshare`; +it still resolves to the host's original procfs instance, so `ps` prints +the host's entire process table instead of a scoped one. This is not a +bug in `containerize`; it is precisely why chapter 34's Containerfile +build includes a `pivot_root` and a fresh `/proc` mount — a namespace +without a matching filesystem view is only half a container. + +Third, the kernel's own OOM log, read independently of anything `pmon` +printed: + +```bash +[vm]$ sudo dmesg | grep -A1 "oom-kill:constraint=CONSTRAINT_MEMCG" +oom-kill:constraint=CONSTRAINT_MEMCG,nodemask=(null),cpuset=/,mems_allowed=0,oom_memcg=/pmon-oomdemo,task_memcg=/pmon-oomdemo,task=sh,pid=27013,uid=0 +Memory cgroup out of memory: Killed process 27013 (sh) total-vm:71332kB, anon-rss:64384kB, file-rss:3488kB, shmem-rss:0kB, UID:0 pgtables:176kB oom_score_adj:0 +``` + +`CONSTRAINT_MEMCG` and `oom_memcg=/pmon-oomdemo` are the kernel's own +words for "this cgroup's `memory.max`, not global memory pressure, is why +this process died" — the same verdict `pmon`'s `signal=9 (KILL)` line +reported, confirmed at the source that actually made the decision. + +## What you learned + +- Namespaces hide global kernel state from a *process*, not from the + kernel; `unshare(2)` moves the caller into new ones now (except + `CLONE_NEWPID`, which only ever affects the *next* child), `setns(2)` + joins one that already exists via an open `/proc//ns/*` fd, and + `clone3(2)` creates the child and its namespaces atomically in one + syscall — the only option with no in-between state. +- A cgroup v2 leaf's `memory.max`/`cpu.max` only prove anything paired + with a negative control: a real hog under a 64 MiB ceiling must be + OOM-killed (exit 137, `dmesg`'s `oom_memcg=` naming the cgroup) — a + cgroup that let it finish would be decorative, not enforcing — and PSI + (`memory.pressure`) is the per-cgroup stall signal, not a system-wide one. +- Namespace changes are per-thread state, and a Go program is threaded + before `main` runs: `unshare(CLONE_NEWPID)` also caps the caller to + exactly one child in the fresh namespace, a restriction `os/exec`'s own + internal probe fork silently spends, producing a real `ENOMEM` this + chapter reproduced live — the fix is `SysProcAttr.Cloneflags`, pushing + the whole operation into the runtime's own fork/exec trampoline, the + same move chapter 14 made for capabilities. +- A container is namespaces + cgroups + rootfs + capabilities + seccomp, + assembled by userspace, not one kernel object — and a namespace without + its matching filesystem view is observably incomplete: `nsenter`'s `ps + -ef` proved it by showing the host's whole process table through a + correctly-`setns`'d pid namespace that never got a fresh `/proc`. + +Next, **seccomp and Landlock**: filtering which syscalls a process may +even attempt, and restricting which files it may reach, as the two +primitives chapter 32 left out of "what a container runtime assembles." + +--- + +

verified — every +transcript and number above was produced this session against the +systems-target guest (Fedora 44, kernel +6.19.10-300.fc44.x86_64): the runner printed +32-namespaces-and-cgroups PASS PASS PASS (3 passed, 0 +failed, 0 skipped) in vm mode, each language's verify.lua +run at 13/13; the positive run emitted pmon: child sees pid 1 +/ pmon: hostname=pmon-vm-check (exit 0) and the memory-hog +negative control emitted pmon: child killed signal=9 (KILL) +(exit 137), byte-identical across cpp/go/rust; readlink +/proc/<pid>/ns/* showed the host's pid 1 and the containerized +child's PID/mount/UTS/network namespaces at different inodes and their +time/IPC/user/cgroup namespaces at identical inodes; dmesg +showed the real CONSTRAINT_MEMCG/oom_memcg=/pmon-oomdemo +kill of pid 27013; nsenter -p -m -u -n ps -ef was run live and +showed the host's full process table, confirming the missing +/proc remount. The Go ENOMEM failure quoted in +the Concurrency lens is this session's own strace output +against the pre-fix build, kept as evidence of a real, reproduced defect +before the Cloneflags fix that now ships and passes. A +one-line verify.lua bug this chapter also surfaced and fixed +— an unescaped - in a Lua pattern silently failing the +hostname assertion — is recorded in the example's own history, not +restated here as a finding about the chapter's subject matter.

diff --git a/_docs/33-seccomp-and-landlock.md b/_docs/33-seccomp-and-landlock.md new file mode 100644 index 0000000..e73a7d9 --- /dev/null +++ b/_docs/33-seccomp-and-landlock.md @@ -0,0 +1,513 @@ +--- +title: "Seccomp and Landlock" +order: 33 +part: "Containers and Virtualization" +description: "fwatch sandboxes itself with two independent, unprivileged kernel layers -- a seccomp-bpf syscall allowlist built from a real strace profile, and a Landlock ruleset that restricts filesystem reads to one directory -- probing the Landlock ABI at runtime instead of assuming it, and proving both with negative controls that watch the exact denial happen." +duration: "55 minutes" +--- + +Chapter 32 gave `fwatch` its own view of the process tree and its own +resource limits through namespaces and cgroup v2 -- isolation the kernel +grants to a group of processes. This chapter hands the same idea to a +*single* process, no grouping required: `watch --sandbox DIR` now installs +a syscall allowlist and a filesystem allowlist against itself, using two +mechanisms any unprivileged process can invoke directly. **seccomp-bpf** +decides, syscall by syscall, whether the calling process is allowed to make +this call at all. **Landlock** decides, path by path, which parts of the +filesystem that process may still read. Neither needs root, a namespace, or +a VM -- which is also why every number in this chapter was reproduced on +the plain host and confirmed unchanged on the `systems-target` lab guest. + +The one new idea worth sitting with: seccomp-bpf is **classic BPF**, the +same in-kernel bytecode VM whose modern descendant, eBPF, this book has so +far only used as an *observation* tool (`bpftrace`, `bcc-tools`, `bpftool` +in Part 8). A seccomp filter is the mirror image -- you are not attaching a +program that watches a syscall and reports it, you are attaching a program +that *decides* the syscall's outcome, on the hot path, before the kernel +does the work. Same VM, same verifier discipline, opposite job. + +The code is in `examples/33-seccomp-and-landlock/`. The run script there +builds/sets up and runs it; its `README.md` covers the CLI, the exit-code +contract, and the paired positive/negative verification in full. + +{% include excalidraw.html + file="33-seccomp-filter-flow" + alt="Two stacked bands. The top userspace band shows fwatch issuing a syscall such as epoll_wait, read, or socket, beside a note that this is classic BPF -- you write the filter, Part 8's bpftrace only observes. The bottom kernel band shows the filter running on every syscall entry: a BPF_LD_ABS load of the syscall number and architecture, a compiled jump table of 26 entries for cpp and rust or 39 for go, a match arrow to SECCOMP_RET_ALLOW that returns control to the caller, and a no-match arrow down to the filter's default action. Two default-action boxes sit side by side: SCMP_ACT_ERRNO(EPERM), the one fwatch actually installs, where the process keeps running and sees EPERM on socket(2); and a dashed, unused SCMP_ACT_KILL_PROCESS box, where a separate probe program's getpid() call ends the whole process with SIGSYS. Footer notes report the real guest transcript: socket(AF_INET,...) = -1 EPERM under fwatch's filter, and the ACT_KILL_PROCESS probe's getpid() ending in killed by SIGSYS (core dumped)." + caption="Figure 33.1 — a syscall's path through fwatch's seccomp-bpf filter: one cBPF program, run again on every entry, deciding ALLOW versus its default action" %} + +> **Tools used** — `strace` (host and `systems-target` VM, to build and +> confirm the syscall profile), `cmake`+`ninja`/`go build`/`cargo` (the +> Python runner's build phase, host, all three toolchains), `ssh`/`scp` +> (via `scripts/test-all-examples.py --mode vm` and its staging step onto +> `systems-target`), a small ad hoc `gcc`-built C program (host and guest, +> to observe `SCMP_ACT_KILL_PROCESS`), `grep`/`/proc//status` (both). +> Everything here ships with Fedora's toolchain or is preinstalled on the +> lab guest per `references/vm-lab.md`; nothing needs root. + +## seccomp-bpf: the filter you write + +`seccomp(2)` installs a cBPF program that the kernel runs on *every* +syscall the process makes from that point on. The program reads two fields +out of `struct seccomp_data` -- the architecture and the syscall number -- +and returns one of a small set of verdicts: `SECCOMP_RET_ALLOW` (run it), +`SECCOMP_RET_ERRNO` (fail it with a chosen errno, without ever entering the +real syscall handler), or `SECCOMP_RET_KILL_PROCESS` (end the process with +`SIGSYS`, no return). Once a filter is loaded it can only ever be +*tightened* -- filters stack, and a later one can narrow what an earlier +one allowed but never widen it. + +`fwatch`'s allowlist did not come from guessing. The comment in the C++ +source says it directly: the syscall set is "empirically confirmed with +`strace -f` across a full watch session." That is the practical way to +build a seccomp profile -- run the real workload under `strace -f -c`, +read off the syscalls it actually issues, and allowlist exactly those. +Reproduced on the host against the unsandboxed watch loop: + +```bash +[host]$ strace -f -c ./cpp/build/release/app watch /tmp/lsp33-profile --timeout-ms 2000 +fwatch: watching /tmp/lsp33-profile +fwatch: exiting (timeout) +% time seconds usecs/call calls errors syscall +------ ----------- ----------- --------- --------- ------------------ + 37.74 0.000080 8 10 close + 13.21 0.000028 4 7 fstat + 12.74 0.000027 9 3 write + 11.79 0.000025 4 6 read + 9.43 0.000020 20 1 epoll_wait + 3.77 0.000008 2 3 epoll_ctl + 3.30 0.000007 7 1 inotify_add_watch + 2.36 0.000005 5 1 inotify_init1 + 1.42 0.000003 3 1 timerfd_create + 1.42 0.000003 3 1 signalfd4 + 1.42 0.000003 3 1 epoll_create1 + 0.94 0.000002 2 1 timerfd_settime + 0.47 0.000001 1 1 rt_sigprocmask + 0.00 0.000000 0 22 mmap + 0.00 0.000000 0 7 mprotect + 0.00 0.000000 0 1 munmap + 0.00 0.000000 0 3 brk + ... (dynamic-linker startup: + execve, openat, pread64, + access, arch_prctl, …) +------ ----------- ----------- --------- --------- ------------------ +100.00 0.000212 2 87 1 total +``` + +That is exactly the source of `kWatchSyscalls` in `examples/33-seccomp-and-landlock/cpp/src/main.cpp` +(`read`/`write`/`close`, the inotify/timerfd/signalfd trio, epoll's three +calls, the allocator's `mmap`/`munmap`/`brk`, and `fstat`/`rt_sigprocmask`). +The rows elided above (`execve`, `openat`, `pread64`, `access`, +`arch_prctl`, `set_tid_address`, `set_robust_list`, `prlimit64`, `rseq`) are +the dynamic linker resolving `libc`/`libstdc++` *before* `main` runs -- +real syscalls this same process makes, but not ones the watch loop itself +issues, so they are outside what the filter needs to allow once it loads +after startup. + +`install_seccomp` (same file) calls `seccomp_init(SCMP_ACT_ERRNO(deny_errno))` +-- that argument is the filter's **default action**, applied to anything +not explicitly allowed -- then `seccomp_rule_add(ctx, SCMP_ACT_ALLOW, nr, 0)` +once per syscall, and `seccomp_load(ctx)` to install it. `seccomp_load` +requires `no_new_privs` first (set via `prctl`), a kernel guard against a +sandboxed process regaining privilege through a setuid binary it execs. + +## `SCMP_ACT_ERRNO` versus `SCMP_ACT_KILL` + +`fwatch` deliberately chose `SCMP_ACT_ERRNO(EPERM)`, not +`SCMP_ACT_KILL_PROCESS`, as its default action, and the difference is +observable, not cosmetic. With `ACT_ERRNO`, a denied syscall returns `-1` +with `errno` set and the calling code continues to run -- the process can +log the denial, clean up, and exit on its own terms; that is what makes +`probe --forbidden-syscall`'s own `EPERM`-and-continue behavior possible. +With `ACT_KILL_PROCESS`, a denied syscall never returns at all: the kernel +delivers an uncatchable `SIGSYS` and the process dies mid-instruction. To +see the contrast for real rather than assert it, a small standalone probe +(not part of `fwatch`, built only to observe this) installs a filter +allowing only `write`/`exit_group`/`rt_sigreturn` under +`SCMP_ACT_KILL_PROCESS`, then calls the unlisted `getpid()`: + +```bash +[vm]$ strace -f -e trace=getpid /tmp/kill-demo 2>&1 | tail -3 +before getpid +getpid() = 39 ++++ killed by SIGSYS (core dumped) +++ +``` + +`strace` shows the syscall entering the kernel and *never returning* -- +the trailing line is `strace` reporting the process's death, not a normal +return value. `ACT_ERRNO` is the right choice whenever the caller has +useful cleanup to do or a human is watching the diagnostic; `ACT_KILL` is +the right choice when a denied syscall means the process's memory can no +longer be trusted (a common posture for hardened service sandboxes) and +continuing at all is the greater risk. + +## Landlock: rulesets, rules, and an ABI you probe + +Landlock restricts what a process (and everything it execs from here on) +may still do to the filesystem, expressed as three calls with no +convenience wrapper in any of the three languages' standard libraries -- +`landlock_create_ruleset`, `landlock_add_rule`, `landlock_restrict_self`, +reached through a raw `syscall(2)` in C++ and Go and through the `landlock` +crate (which itself wraps the same three numbers) in Rust. A **ruleset** +declares *which access rights this program intends to handle at all* -- +`fwatch` handles only `LANDLOCK_ACCESS_FS_READ_FILE` and `_READ_DIR`, and +leaves `handled_access_net` at zero. A **rule** then grants a subset of +those handled rights to one filesystem object, here a `PATH_BENEATH` rule +binding read rights to a directory fd. `restrict_self` is the point of no +return: it activates the ruleset against the *calling* process and every +descendant, and there is no syscall to lift it -- the only way out is +`execve`-ing a program that never asked for the restriction to begin with, +and even then the new program inherits the same ruleset. + +The ABI matters here in a way it rarely does elsewhere in this book, and +the chapter's own two machines make the point better than any prose: + +```bash +[host]$ ./cpp/build/release/app probe --sandbox /tmp --outside /etc/hostname +fwatch: landlock ABI=9 enforced +fwatch: probe outside /etc/hostname: EACCES (Permission denied) +[vm]$ /tmp/lsp33-cpp/app probe --sandbox /tmp --outside /etc/hostname +fwatch: landlock ABI=7 enforced +fwatch: probe outside /etc/hostname: EACCES (Permission denied) +``` + +Same distribution, same syscall numbers, and yet the host (kernel +`7.1.3-200.fc44.x86_64`) reports ABI 9 while `systems-target` (kernel +`6.19.10-300.fc44.x86_64`) reports ABI 7 -- two real Fedora 44 kernels a +few point releases apart, disagreeing about what Landlock supports, while +agreeing completely on the observable behavior the ruleset enforces. Every +call in this example reaches that number the only real way: calling +`landlock_create_ruleset(NULL, 0, LANDLOCK_CREATE_RULESET_VERSION)` and +reading back the version the *running* kernel returns, never a compile-time +constant. A binary that assumed "ABI 9 because that's what built it" would +misbehave the moment it ran on the guest. + +{% include excalidraw.html + file="33-landlock-ruleset" + alt="Two stacked bands. The top band builds the ruleset: landlock_create_ruleset called first with NULL/0/VERSION to probe the ABI, returning 9 on the host versus 7 on the systems-target guest despite both being Fedora 44 kernels; then landlock_create_ruleset(attr) with handled_access_fs set to READ_FILE and READ_DIR, feeding into landlock_add_rule binding a PATH_BENEATH rule to one directory's O_PATH file descriptor; then prctl(PR_SET_NO_NEW_PRIVS,1) feeding into landlock_restrict_self, marked irrevocable, with a note that Go's build also passes the TSYNC flag here to cover every OS thread already running. The bottom band shows the filesystem view after restrict_self: the sandboxed directory /tmp/lsp33-watch-sandbox with READ_FILE and READ_DIR still allowed and the watch loop working normally, beside /tmp/lsp33-outside-target/secret.txt where open(O_RDONLY) returns EACCES, with an arrow from the ruleset's restrict_self box down to each outcome, solid to the grant and dashed to the denial. A footer note across the bottom states that handled_access_net was left at zero in this ruleset, and that Landlock's TCP bind/connect rights have existed since ABI 4 and UDP rights only since ABI 10 -- so even at ABI 7 or 9 a Landlock-only sandbox does not cover the network by itself." + caption="Figure 33.2 — the Landlock ruleset/rule model, with the ABI probed live rather than assumed and the real grant/deny split from this chapter's probes" %} + +**What Landlock can and cannot do.** Filesystem mediation -- +open/read/write/execute/rename, scoped to a path -- has been solid since +ABI 1 (Linux 5.13). Network mediation is newer and arrived in pieces: +`LANDLOCK_ACCESS_NET_BIND_TCP`/`_CONNECT_TCP` since ABI 4, but UDP +bind/connect rights not until ABI 10; `fwatch` targets neither, so on both +of these machines a Landlock-restricted process can still open arbitrary +sockets outside the tree entirely -- that gap is exactly why `fwatch` +layers seccomp *underneath* it rather than relying on Landlock alone. +IPC-scoping flags (restricting signals and abstract-socket connections +between sandboxed and unsandboxed processes) arrived at ABI 6, so both +machines here have them even though neither example uses them. The +practical rule this chapter teaches: **probe the ABI, then gate each +feature on the version number it actually needs** -- never gate on "did +`landlock_create_ruleset` return non-negative." + +## How the code works + +Both mechanisms share one setup shape across all three languages: probe +the Landlock ABI, build and enforce the ruleset, then install the seccomp +filter, each step logging a status line before the identical epoll watch +loop from chapter 09 runs unmodified. C++ and Rust do this **in-process** +with raw syscalls; Go does something structurally different, and the +concurrency lens below explains why. + +{% include codetabs.html langs="C++|Go|Rust" %} + +```cpp +// Create a ruleset that handles READ_FILE + READ_DIR (available since ABI +// v1), add one path-beneath rule granting exactly those rights under `dir`, +// then restrict_self. On success the calling process — and everything it +// execs from here on — can only read inside `dir`. +[[nodiscard]] std::expected apply_landlock(const std::string& dir) { + constexpr std::uint64_t access = LANDLOCK_ACCESS_FS_READ_FILE | LANDLOCK_ACCESS_FS_READ_DIR; + landlock_ruleset_attr attr{.handled_access_fs = access, .handled_access_net = 0, .scoped = 0}; + + const long rs = sys_landlock_create_ruleset(&attr, sizeof(attr), 0); + if (rs < 0) { + return std::unexpected(last_error()); + } + Fd ruleset{static_cast(rs)}; + + auto dirfd = checked(::open(dir.c_str(), O_PATH | O_DIRECTORY | O_CLOEXEC)); + if (!dirfd) { + return std::unexpected(dirfd.error()); + } + landlock_path_beneath_attr pb{.allowed_access = access, .parent_fd = dirfd->get()}; + if (sys_landlock_add_rule(ruleset.get(), LANDLOCK_RULE_PATH_BENEATH, &pb, 0) < 0) { + return std::unexpected(last_error()); + } + dirfd->reset(); + + // Landlock requires no_new_privs (or CAP_SYS_ADMIN) before restrict_self. + if (::prctl(PR_SET_NO_NEW_PRIVS, 1, 0, 0, 0) < 0) { + return std::unexpected(last_error()); + } + if (sys_landlock_restrict_self(ruleset.get(), 0) < 0) { + return std::unexpected(last_error()); + } + return {}; +} +``` + +```go +// installSeccomp loads an allowlisting filter with SECCOMP_FILTER_FLAG_TSYNC +// so every OS thread already running in this process is covered, not just +// the calling one. Returns the number of syscalls admitted. +func installSeccomp(allowed []uint32, denyErrno uint32) (int, error) { + // seccomp(2) requires no_new_privs (or CAP_SYS_ADMIN), same as Landlock; + // harmless to set again if applyLandlock already did. + if err := unix.Prctl(unix.PR_SET_NO_NEW_PRIVS, 1, 0, 0, 0); err != nil { + return 0, fmt.Errorf("prctl(PR_SET_NO_NEW_PRIVS): %w", err) + } + prog := buildAllowlist(allowed, denyErrno) + fprog := sockFprog{Len: uint16(len(prog)), Filter: &prog[0]} + _, _, errno := unix.Syscall(unix.SYS_SECCOMP, uintptr(unix.SECCOMP_SET_MODE_FILTER), + uintptr(unix.SECCOMP_FILTER_FLAG_TSYNC), uintptr(unsafe.Pointer(&fprog))) + if errno != 0 { + return 0, errno + } + return len(allowed), nil +} +``` + +```rust +/// Installs an allowlist of `allowed` syscalls; everything else returns +/// errno `deny_errno`. Returns the number of syscalls admitted. +fn install_seccomp(allowed: &[i64], deny_errno: i32) -> Result { + let rules = allowed.iter().map(|&nr| (nr, vec![])).collect(); + let filter = SeccompFilter::new( + rules, + SeccompAction::Errno(deny_errno as u32), + SeccompAction::Allow, + TargetArch::x86_64, + ) + .context("seccomp: build filter")?; + let program: BpfProgram = filter.try_into().context("seccomp: compile filter")?; + seccompiler::apply_filter(&program).context("seccomp: install filter")?; + Ok(allowed.len()) +} +``` + +C++'s `apply_landlock` is the clearest read of the three-call shape: build +the `landlock_ruleset_attr` naming the *rights this program is willing to +have restricted at all* (`handled_access_fs`, leaving `handled_access_net` +at zero -- an explicit, auditable "we do not sandbox network here", not an +oversight), open the target directory `O_PATH`-only (no read/write access +needed just to name it as a rule target), bind the rule with +`landlock_add_rule`, and only then call `prctl(PR_SET_NO_NEW_PRIVS)` and +`landlock_restrict_self` -- in that order, because `restrict_self` demands +`no_new_privs` already be set. Rust's `landlock` crate (source in +`rust/src/main.rs`) compresses the same four calls into one builder chain +-- `Ruleset::default().handle_access(...).create()?.add_rule(...)?.restrict_self()?` +-- and adds one thing the raw syscall path does not give you for free: it +checks the returned `RulesetStatus`, so a kernel that silently +best-effort-ignores an unsupported right is caught (`bail!` if +`status.ruleset != RulesetStatus::FullyEnforced`) instead of quietly +running unsandboxed. + +Go's `installSeccomp` is the odd one out for a reason worth reading +closely: `golang.org/x/sys/unix` exposes the raw `seccomp(2)` syscall +number but no BPF assembler, so `buildAllowlist` (same file) hand-emits the +cBPF program byte for byte -- `BPF_LD|BPF_W|BPF_ABS` to load the syscall +number, one `BPF_JMP|BPF_JEQ` comparison per allowed syscall that jumps to +`RET_ALLOW` on a match, and a fall-through `RET_ERRNO`. That is, in +miniature, exactly what `libseccomp` and `seccompiler` compile down to +underneath their friendlier APIs -- Go's version just makes the compiled +form visible because nothing hides it. The other detail to notice is +`SECCOMP_FILTER_FLAG_TSYNC` in the syscall arguments and its Landlock +counterpart in `applyLandlock`: both matter only because of what the Go +runtime does before `main` ever runs, covered next. + +## Errors, three ways + +Every failure path in this example distinguishes three outcomes, and the +distinction is what makes a negative control trustworthy rather than +decorative. A **setup failure** -- Landlock unsupported, `seccomp_load` +rejecting a malformed filter, a permission error opening the sandbox +directory -- is a real bug and exits `1` with the syscall name and +`strerror` text, identically formatted across languages (`SysErr`-style in +C++/Rust, a wrapped `error` in Go). A **confirmed denial** -- the sandbox +did exactly what it claims, `EPERM` on the forbidden syscall or `EACCES` on +the outside path -- exits `20`, `fwatch`'s own "probe passed" convention. +An **unconfirmed denial** -- the probed operation *succeeded* when it +should have been blocked -- exits `21`: a real sandboxing bug, distinct +from both a clean pass and an unrelated error. Reusing the ordinary `0/1/2` +contract for this would erase exactly the distinction the chapter is about: +a script that only checks "exit code is nonzero" cannot tell "the sandbox +worked" from "the sandbox is broken." + +## Concurrency lens + +Both Landlock and seccomp are, by default, **per-thread** kernel state -- +they restrict the calling thread, not automatically every thread in the +process. That is invisible in C++ and Rust here because both builds are +deliberately single-threaded before the sandbox goes on: `main` calls +`apply_landlock` and `install_seccomp` before the epoll loop starts any +concurrency of its own, so "the calling thread" and "the whole process" +are the same thing. + +Go cannot make that assumption, and its `main.go` says so in a comment +worth quoting directly: "the Go runtime schedules goroutines across OS +threads it creates and destroys on its own schedule (GC workers, sysmon, +threads parked in blocking syscalls) -- applying either sandbox to only the +calling thread would leave those other threads unrestricted, and a +goroutine the runtime later migrates onto one of them would silently run +outside the sandbox." The fix is two-part, mirroring the C++/Rust identity +lesson from chapter 14 almost exactly: `runtime.LockOSThread()` pins the +sandboxing goroutine to one OS thread so the thread issuing the Landlock +and seccomp syscalls never changes out from under it mid-sequence, and +both `landlock_restrict_self`'s `TSYNC` flag and `SECCOMP_FILTER_FLAG_TSYNC` +apply the new restriction to *every* thread already running in the process +atomically, not just the caller's. `applyLandlock` tries `TSYNC` first and +falls back to per-thread-only only if the running kernel rejects the flag +-- a second, smaller instance of "probe, don't assume." Confirmed +behaviorally, not just by reading the flag names: a goroutine the runtime +schedules onto a different OS thread *after* the sandbox is installed +still gets `EACCES`/`EPERM` on the restricted operations, because `TSYNC` +already covered that thread before it existed as a goroutine destination. + +## Build, run, observe + +```bash +[host]$ LIBVIRT_DEFAULT_URI=qemu:///system \ + python3 scripts/test-all-examples.py --only 33-seccomp-and-landlock --mode vm +``` + +Against `systems-target` this session (the build phase compiles cpp with +`cmake`+`ninja`, go with `go build`, and rust with `cargo build --release` +before any of the three ever touches the guest): + +``` +building 3 example-lang combinations (jobs=1)... + build 33-seccomp-and-landlock [cpp]: ok + build 33-seccomp-and-landlock [go]: ok + build 33-seccomp-and-landlock [rust]: ok + +verifying... + verify 33-seccomp-and-landlock [cpp]: PASS + verify 33-seccomp-and-landlock [go]: PASS + verify 33-seccomp-and-landlock [rust]: PASS + +example cpp go rust +33-seccomp-and-landlock PASS PASS PASS + +3 passed, 0 failed, 0 skipped (logs in build-logs/) +``` + +By hand, `run-sandbox-checks.sh` staged and run as root on the guest (the +book's convention for every namespace/cgroup/seccomp/Landlock demo, even +though neither mechanism here actually requires root): + +```bash +[vm]$ sudo bash /tmp/lsp33-run-sandbox-checks.sh /tmp/lsp33-cpp/app +=== forbidden-syscall === +fwatch: probe forbidden-syscall: EPERM (Operation not permitted) +fwatch: seccomp filter installed (26 syscalls allowed) +forbidden-syscall-exit=20 +=== outside === +fwatch: landlock ABI=7 enforced +fwatch: probe outside /tmp/lsp33-outside-target/secret.txt: EACCES (Permission denied) +outside-exit=20 +=== watch-sandbox (positive control: watching inside the tree still works) === +fwatch: landlock ABI=7 enforced +fwatch: seccomp filter installed (26 syscalls allowed) +fwatch: watching /tmp/lsp33-watch-sandbox +event: deleted inside.txt +fwatch: exiting (timeout) +watch-sandbox-exit=0 +``` + +The third block is the control that makes the first two mean anything: +sandboxing that broke the watch loop itself would be worthless no matter +how convincingly it denied the two probes. + +## Cross-check on `systems-target` + +`strace` and `/proc` are independent of `fwatch`'s own exit-code claims, +so both are worth reading directly against the same guest run. First, the +denied syscall itself, traced from outside the process: + +```bash +[vm]$ strace -f -e trace=socket,seccomp /tmp/lsp33-cpp/app probe --forbidden-syscall 2>&1 | tail -4 +seccomp(SECCOMP_SET_MODE_FILTER, 0, {len=34, filter=0x26638170}) = 0 +fwatch: seccomp filter installed (26 syscalls allowed) +socket(AF_INET, SOCK_STREAM, IPPROTO_IP) = -1 EPERM (Operation not permitted) +fwatch: probe forbidden-syscall: EPERM (Operation not permitted) +``` + +Second, the kernel's own record of the sandbox state, read from a +long-lived sandboxed process rather than a probe that exits immediately: + +```bash +[vm]$ /tmp/lsp33-cpp/app watch --sandbox /tmp/lsp33-status-check --timeout-ms 3000 & +[vm]$ grep -i seccomp /proc/$!/status +Seccomp: 2 +Seccomp_filters: 1 +``` + +`Seccomp: 2` is `SECCOMP_MODE_FILTER` (mode 1 is the older, filter-less +"strict" mode; 0 is unsandboxed) -- `/proc` confirms the filter is live +independently of anything the process prints about itself. +`ip_unprivileged_port_start`-style sysctl shortcuts do not apply here since +neither mechanism has a global toggle to check against; the closest +equivalent negative control not shown above is simply repeating the +`outside` probe against a path *inside* the sandboxed tree, which the demo +script's third block already exercises implicitly by watching successfully +there. + +## What you learned + +- Seccomp-bpf is classic BPF running the opposite job from the eBPF tools + in Part 8: a program you write decides each syscall's outcome + (`SECCOMP_RET_ALLOW`/`_ERRNO`/`_KILL_PROCESS`) instead of merely + observing it, built from a real `strace -f -c` profile of the workload, + not a guess. +- `SCMP_ACT_ERRNO` lets a denied call return and the caller keep running; + `SCMP_ACT_KILL_PROCESS` ends the process on `SIGSYS` with no return -- + confirmed by watching `strace` show `getpid()` never return in the + kill-mode probe. +- Landlock's ruleset/rule/restrict_self model is layered access control you + build yourself, and the ABI must be probed at runtime + (`landlock_create_ruleset(NULL, 0, VERSION)`) rather than assumed -- this + session's own host and `systems-target` guest returned different + versions (9 vs 7) from the same distribution. +- Both mechanisms are per-thread by default; a single-threaded process + never notices, but Go's always-threaded runtime needs + `runtime.LockOSThread()` plus the `TSYNC` flags to cover threads the + scheduler creates around the sandboxing call -- the same "identity is + per-thread, not per-process" lesson chapter 14 taught for capabilities. + +Next, **`examples/34-our-programs-in-containers`**: the same daemons this +book has built, running inside rootless Podman containers built from +multi-stage UBI 10 Containerfiles -- where seccomp and Landlock profiles +like this chapter's stop being something a program installs on itself and +start being something a container runtime installs *for* it. + +--- + +

verified — every transcript +and number above was produced this session, reproduced on both the plain +host (kernel 7.1.3-200.fc44.x86_64, libseccomp 2.6.0, GCC +16.1.1, Go 1.26.5, Rust/Cargo 1.97.1) and the systems-target +guest (kernel 6.19.10-300.fc44.x86_64): the runner printed +33-seccomp-and-landlock PASS PASS PASS (3 passed, 0 failed, +0 skipped) in vm mode; run-sandbox-checks.sh on the guest +produced forbidden-syscall-exit=20 with the printed +EPERM line, outside-exit=20 with the printed +EACCES line, and watch-sandbox-exit=0 with an +event: deleted inside.txt line, byte-identical in shape across +cpp/go/rust (allowed-syscall counts differ: 26 for cpp/rust, 39 for go); +strace on the guest showed socket(...) = -1 EPERM +under the installed filter and, in a separate ad hoc +SCMP_ACT_KILL_PROCESS probe built for this chapter, +getpid() followed by killed by SIGSYS (core dumped); +/proc/<pid>/status read Seccomp: 2 with +Seccomp_filters: 1 for a live sandboxed process; and the +Landlock ABI probe returned 9 on the host versus 7 on +systems-target, read live via +landlock_create_ruleset(NULL, 0, LANDLOCK_CREATE_RULESET_VERSION) +in every language, never hardcoded. Landlock's network-rights history +(TCP since ABI 4, UDP since ABI 10) and current crate/library pins +(landlock 0.4.5, seccompiler 0.5.0, libseccomp +2.6.1 as the latest upstream release) were confirmed against upstream +sources at authoring time and are worth re-checking before a future +release, as with any dependency pin in this book.

diff --git a/_docs/34-our-programs-in-containers.md b/_docs/34-our-programs-in-containers.md new file mode 100644 index 0000000..5df77cd --- /dev/null +++ b/_docs/34-our-programs-in-containers.md @@ -0,0 +1,536 @@ +--- +title: "Our programs in containers" +order: 34 +part: "Containers and Virtualization" +description: "The same three daemons this book has built all along, now shipped as multi-stage UBI10 Containerfiles and run under rootless podman -- pid 1's signal and reaping duties, the GOMAXPROCS-from-cgroup story that catches C++'s hardware_concurrency() lying about its own container, debugging across the namespace boundary with podman exec/nsenter/gdb, and the seccomp default that quietly denies io_uring." +duration: "60 minutes" +--- + +Chapter 32 gave one of our programs its own namespaces and cgroup directly, +by calling `unshare`/`clone3` itself. Chapter 33 had `fwatch` sandbox itself +with seccomp and Landlock. This chapter hands both jobs to something else: +a container runtime. `app`, a tiny stand-in entrypoint built three times, +runs as `podman`'s idea of a container -- and that changes what "process 1" +means, what "how many CPUs do I have" means, and what a debugger needs to +cross to reach it. None of this is exotic: it is exactly the daemons this +book already writes, deployed the way most of them actually ship. + +The code is in `examples/34-our-programs-in-containers/`. The run script +there builds/sets up and runs it; its `README.md` covers the CLI, the +output contract, and the full verification story. + +{% include excalidraw.html + file="34-image-layering" + alt="Two side-by-side bands. The left, discarded builder stage starts from registry.access.redhat.com/ubi10/ubi:10.2 at 220 MB measured, runs dnf install of gcc-c++, golang, or rustup, and statically links the binary (cpp -static-libstdc++ -static-libgcc, go CGO_ENABLED=0, rust rustup 1.97.1). An amber COPY --from=builder arrow crosses into the right, shipped runtime stage, which starts from registry.access.redhat.com/ubi10-micro:10.2 at 24.3 MB measured, copies only the one binary to /usr/local/bin/app, and sets ENTRYPOINT/CMD. A footer note reports the real measured final image sizes -- cpp and go both 27.2 MB, rust 24.9 MB -- barely above the bare micro base, and explains that static linking is what makes copying one binary sufficient." + caption="Figure 34.1 -- app's multi-stage Containerfile: the 220 MB builder never ships, only the binary crosses into the 24.3 MB runtime base" %} + +> **Tools used** -- `podman` build/run/exec/inspect/unshare/stop (host, +> rootless), `nsenter` (host, both directly and via `podman unshare`), +> `gdb` (host, attached across the container's pid namespace), `ldd`/`file` +> (host, on binaries copied out of the built images), `python3`+`ctypes` +> (host, probing `io_uring_setup` under three seccomp postures). Everything +> here is checked by `scripts/check-host.sh`; podman ships with Fedora and +> needs no root. + +## Multi-stage images: the builder never ships + +A **Containerfile** -- podman's name for the same syntax Dockerfiles use -- +can declare more than one `FROM`. Each `FROM` starts a fresh, unrelated +build stage; a later stage can `COPY --from=` specific files +out of it, and only the *last* stage's filesystem becomes the image podman +actually tags. `app`'s C++ build is the clearest read of the shape: + +```dockerfile +FROM registry.access.redhat.com/ubi10/ubi:10.2 AS builder +RUN dnf -y install gcc-c++ cmake ninja-build libstdc++-static && dnf clean all +WORKDIR /src +COPY CMakeLists.txt CMakePresets.json ./ +COPY src ./src +RUN cmake --preset release \ + -DCMAKE_CXX_FLAGS="-static-libstdc++ -static-libgcc" \ + -DCMAKE_EXE_LINKER_FLAGS="-static-libstdc++ -static-libgcc" \ + && cmake --build --preset release + +FROM registry.access.redhat.com/ubi10-micro:10.2 AS runtime +COPY --from=builder /src/build/release/app /usr/local/bin/app +ENTRYPOINT ["/usr/local/bin/app"] +CMD ["serve"] +``` + +The builder stage is `ubi10/ubi:10.2` -- measured on this host at 220 MB -- +with a full `dnf` toolchain installed into it. The runtime stage is +`ubi10-micro:10.2`, measured at 24.3 MB: no shell, no package manager, only +`glibc` and the handful of libraries UBI's micro variant ships. `COPY +--from=builder` reaches back into the discarded stage for exactly one +file. Building all three languages' images fresh on this host: `lsp34-cpp` +and `lsp34-go` land at 27.2 MB, `lsp34-rust` at 24.9 MB -- a few MB above +the bare micro base, because the whole 220 MB builder is gone from the +final image. + +That gap only closes because of the static-vs-dynamic linking recap +chapter 4 built with `sysprobe`: pulling the three languages' binaries back +out of these images with `podman cp` and running `ldd` on them tells the +same story `ldd` told there, sharpened by one deliberate choice. Go's +binary is `not a dynamic executable` -- `CGO_ENABLED=0` gives a fully +static binary, so the runtime stage needs nothing beyond the file itself. +Rust's binary still links `libgcc_s.so.1` and `libc.so.6` -- both already +present in `ubi10-micro`, so no extra `RUN` is needed. C++'s binary is the +interesting middle case: `ldd` lists `libm.so.6`, `libc.so.6`, and the vDSO, +but *not* `libstdc++.so.6` -- the Containerfile's `-static-libstdc++ +-static-libgcc` flags embed exactly those two libraries into the binary, +which is what lets the runtime stage skip installing them. A plain, +dynamically-linked C++ build (chapter 4's default) would need `libstdc++` +copied or installed into the runtime stage too; this chapter's Containerfile +avoids that by choosing static linking specifically for the two libraries +the micro base does not carry. Rust's builder has its own wrinkle worth +naming: UBI10's `dnf` `rust` package is 1.92.0, older than this book's +pinned `rust-toolchain.toml` channel (1.97.1), so the builder installs the +exact toolchain via `rustup` instead of `dnf install rust`. + +## Rootless podman: one user, one private uid range + +Every container here runs **rootless** -- no `sudo`, no privileged daemon +-- which is possible because of a second Linux mechanism from chapter 32's +namespace tour: the **user namespace**. `podman` maps container uid 0 to +the invoking host user and a further range of container uids to a private, +per-user block of otherwise-unused host uids, declared in `/etc/subuid` and +`/etc/subgid`. On this host: + +```bash +[host]$ grep "^$(whoami):" /etc/subuid /etc/subgid +/etc/subuid:rsedor:524288:65536 +/etc/subgid:rsedor:524288:65536 +``` + +That line grants `rsedor` a private range of 65,536 host uids starting at +524288 -- ids no other account owns. `podman unshare cat /proc/self/uid_map`, +run in podman's own user namespace, shows exactly how the two ranges +combine: + +```bash +[host]$ podman unshare cat /proc/self/uid_map + 0 1000 1 + 1 524288 65536 +``` + +Container uid 0 (root, inside) maps to host uid 1000 (`rsedor`, outside) -- +one id. Container uids 1 through 65536 map to the private subuid block. +The consequence worth sitting with: a process that is root *inside* the +container has, outside it, only the privileges of the ordinary host user +who launched podman -- root in the container cannot write to another +user's files, bind privileged host ports, or load kernel modules, because +outside the namespace it never had uid 0 to begin with. This is the same +"identity is a kernel-side mapping, not a label" lesson chapter 14 taught +for capabilities, applied to the uid itself: the mapping table, not the +number a `ps` inside the container prints, is what the kernel actually +checks. + +## PID 1: signal duties naive skips + +Every container's first process inherits a role most programs never +occupy: it is PID 1 of its own PID namespace. The kernel treats PID 1 +specially -- a signal for which it has installed **no explicit handler** +is simply dropped instead of running that signal's default action. An +ordinary process ignoring `SIGTERM` still dies, because "terminate" is the +default action when no handler exists; PID 1 with no handler does not even +get that. Only `SIGKILL` and `SIGSTOP` still work. `app naive` demonstrates +this by installing nothing: it calls no `sigaction`, `signalfd`, or +`signal.Notify` at all before its heartbeat loop. `app serve` is the fix: +it blocks `SIGTERM`/`SIGINT`/`SIGCHLD`, receives them explicitly, stops +spawning new work, forwards `SIGTERM` to the worker process it supervises, +waits for that worker to exit, and only then exits itself -- so nothing is +left running or unreaped when the container stops. + +{% include excalidraw.html + file="34-pid1-signal-flow" + alt="Two side-by-side bands contrasting serve and naive receiving SIGTERM as a container's PID 1. The serve band: podman stop sends SIGTERM, the signalfd/Notify/Signals wake fires, job_spawner.request_stop() runs, kill(worker, SIGTERM) plus waitpid/Wait() reaps it, a shutting-down line prints, and the process exits 0 -- measured at cpp 192ms, go 60 to 87ms, rust 101ms, all well inside a 3000ms grace budget. The naive band: podman stop sends SIGTERM into a kernel that finds no explicit handler on PID 1, so the signal is simply dropped for cpp and rust -- podman waits out the full grace period and falls back to SIGKILL, measured at cpp 2108ms and rust 2074ms. A muted box notes the Go exception: the Go runtime installs its own fatal-signal handler at startup regardless of user code, so naive still dies in about 60ms -- but with no shutting-down line and no cleanup, in every language." + caption="Figure 34.2 -- serve's graceful pid-1 shutdown sequence beside naive's kernel-masked signal, with this session's real podman stop timings" %} + +Reaping is the other pid-1 duty, and it splits into two real +implementation strategies across the three languages, because Go's and +Rust's runtimes are not safe to `fork()` once other threads or goroutines +exist. C++ uses a raw `fork(2)` for both the worker and each short-lived +job -- no `exec`, just a different function running in the child -- because +`app`'s C++ build is deliberately single-threaded up to that point, so the +forking thread and the whole process are the same thing. Go and Rust +instead **self-reexec**: `serve` resolves its own binary path +(`os.Executable()` / `std::env::current_exe()`) and starts `worker`/`job` +as genuinely separate processes running the same binary with a different +argument, then reaps each one with `Cmd.Wait()` / `Child::wait()` -- +ordinary `wait4(2)` under the hood, one goroutine or thread parked per +child *is* the reap loop, no `SIGCHLD` involved anywhere in those two +builds. + +## Container-aware CPU and memory detection + +The headline trap: three ways of asking "how many CPUs do I have" give two +different answers under the same constraint. `podman run --cpus=2 +--memory=128m` sets the container's cgroup v2 `cpu.max` to `200000 +100000` (200ms of CPU time per 100ms period -- two cores' worth) and +`memory.max` to `134217728` bytes. Reading those files back from inside +each language's own process and reporting them alongside a +parallelism estimate is `app`'s first printed line: + +{% include codetabs.html langs="C++|Go|Rust" %} + +```cpp +void print_container_line() { + const unsigned n = std::thread::hardware_concurrency(); // THE TRAP: host cpus + std::println("container: cpu.max={} effective_parallelism={} mem.max={}", + cpu_max_display(), n, mem_max_display()); +} +``` + +```go +func printContainerLine() { + fmt.Printf("container: cpu.max=%s effective_parallelism=%d mem.max=%s\n", + cpuMaxDisplay(), runtime.GOMAXPROCS(0), memMaxDisplay()) +} +``` + +```rust +fn print_container_line() { + // Cgroup-aware, unlike the C++ version's hardware_concurrency(). + let n = std::thread::available_parallelism().map(|n| n.get()).unwrap_or(0); + println!( + "container: cpu.max={} effective_parallelism={} mem.max={}", + cpu_max_display(), + n, + mem_max_display() + ); +} +``` + +All three share the same `cpu_max_display()`/`mem_max_display()` helpers +(not shown above -- they resolve `/proc/self/cgroup`'s `"0::"` line +to find *this process's own* cgroup, then read `cpu.max`/`memory.max` +under `/sys/fs/cgroup` at that path, falling back to the root cgroup file +if the per-path one is missing). Those helpers agree completely across +languages, because they are just reading the same kernel files. The +`effective_parallelism` field is where they diverge, and the divergence is +real, not theoretical: run all three images with `--cpus=2` on this +16-core host, + +``` +cpp: container: cpu.max=200000/100000 effective_parallelism=16 mem.max=134217728 +go: container: cpu.max=200000/100000 effective_parallelism=2 mem.max=134217728 +rust: container: cpu.max=200000/100000 effective_parallelism=2 mem.max=134217728 +``` + +`std::thread::hardware_concurrency()` is `sysconf(_SC_NPROCESSORS_ONLN)` +under the hood -- it asks how many CPUs the *host* has online and has no +concept of a cgroup, so it reports 16 regardless of `--cpus=2`. As of the +`go 1.26` language version pinned in `go.mod`, `runtime.GOMAXPROCS(0)`'s +*default* (unset by the environment or a prior call) divides the cgroup's +CPU bandwidth quota by its period whenever that is lower than the host's +core count, and re-checks periodically in case an orchestrator changes the +limit live -- confirmed here as `2`, matching the actual constraint +(`GODEBUG=containermaxprocs=0` opts back out, for anyone who needs the old +host-count behavior). Rust's `std::thread::available_parallelism()` on +Linux performs the same quota-over-period division against the calling +thread's affinity mask, landing on the same correct `2`. A C++ thread pool +sized off `hardware_concurrency()` in this container would spawn sixteen +threads to share two cores' worth of CPU time -- oversubscribed, thrashing +the scheduler, and never once reading the number that was actually true. + +## How the code works + +`serve`'s full startup, beyond the resource-detection line above, follows +one shape in all three languages: block/register the shutdown signals, +start the supervised worker, start a periodic job spawner, then block +until a signal arrives. C++ blocks `SIGTERM`/`SIGINT`/`SIGCHLD` with +`sigprocmask(SIG_BLOCK, ...)` and receives them as `signalfd(2)` reads +inside an ordinary `poll(2)` loop -- the same pattern chapter 12 built for +`pmon`, so no code ever runs in real signal-handler context. Go and Rust +skip that machinery entirely: Go's `signal.Notify` delivers onto a +channel, and Rust's `signal_hook::iterator::Signals` delivers as a +blocking iterator on the main thread; both are ordinary, safely-scheduled +code, not async-signal-restricted handlers, because neither language's +runtime lets you install a raw POSIX handler as directly as C does. The +`std::jthread` (C++) and background goroutine/thread (Go/Rust) that spawn +jobs every second all take the same shape: loop, sleep in short slices so +a stop request is noticed quickly, spawn one job, repeat -- and every one +is cooperatively stopped (`stop_token`, an atomic flag, a channel close) +before `serve` waits on the worker, so no job-spawn races the shutdown +sequence. + +The one hardcoded assumption worth naming: `cpu_max_display()` only +handles cgroup v2's single-line `cpu.max` format (`"max"` or +`" "`); a host still running cgroup v1 would need the older +`cpu.cfs_quota_us`/`cpu.cfs_period_us` pair instead, and none of these +builds falls back to that -- they print `"unknown"` from the empty-file +path instead of guessing. + +## Errors, three ways + +Setup failures -- a `fork`/`spawn` that fails, a `signalfd`/`Signals` that +cannot be installed -- exit `1` with the syscall name and its error text, +formatted identically across languages. Usage errors (an unrecognized +subcommand, missing arguments to `job`) exit `2` before anything is +spawned. There is no third "confirmed denial" exit code in this example +the way chapter 33's sandbox probes had one -- `app`'s only externally +visible contract is its stdout lines and its exit code on a clean +shutdown (`0`), and the *absence* of the `shutting down` line from +`naive`'s log is itself the evidence `verify.lua` checks, not a distinct +exit status. The subtler three-way split is behavioral, not code-based: a +`SIGTERM`-forwarded worker in C++ needs one line of insurance no other +language needs, covered next. + +## Concurrency lens + +The one real bug this example produced during development lived in C++'s +raw `fork(2)` path, and it is the same class of mistake chapter 14 warned +about for credentials: **signal masks are per-thread kernel state, and +`fork()` copies only the calling thread into the child.** `serve` blocks +`SIGTERM`/`SIGINT`/`SIGCHLD` with `sigprocmask(SIG_BLOCK, ...)` so it can +receive them via `signalfd`. The worker, forked from that same thread, +inherited the *blocked* mask -- so when `serve` later forwarded `SIGTERM` +to it with `kill(worker_pid, SIGTERM)`, the signal did not run its default +(terminate) action. It just sat **pending**, forever: the worker was +unkillable by anything short of `SIGKILL`, exactly the failure mode +`naive` demonstrates deliberately at the container's pid-1 layer, now +reproduced accidentally one process down. The fix is one line in the +child, immediately after `fork()`: + +```cpp + sigset_t empty; + sigemptyset(&empty); + ::sigprocmask(SIG_SETMASK, &empty, nullptr); +``` + +which mirrors `pmon`'s reset-before-`execvp` from chapter 12 almost +exactly -- the difference is that there is no `exec` here to reset the +mask for free, so the reset has to be written explicitly. Go and Rust +never meet this bug at all, and not by luck: both languages avoid +`fork()`-without-`exec()` entirely, because forking a multithreaded +runtime (goroutine scheduler threads, Rust's allocator and async +executors) risks a forked child whose only surviving thread might be +holding a lock some other, vanished thread owned at the moment of the +fork. Self-reexec sidesteps the whole hazard class by never forking a +live multithreaded process in the first place -- the new process starts +clean, mask and all, from `execve`. + +## Debugging across the namespace boundary + +Three tools reach the same PID 1 from three different starting points. +`podman exec` is the easiest: it is already inside the container runtime's +view, so `podman exec cat /proc/1/status` just works. Reaching the +same process from the **bare host** needs a different approach, because a +rootless container's namespaces are private to podman's own unprivileged +user namespace -- plain `nsenter` targeting the container's host-side pid +fails outright: + +```bash +[host]$ nsenter --target "$HOSTPID" --pid --mount -- readlink /proc/1/exe +nsenter: reassociate to namespaces failed: Operation not permitted +``` + +`podman unshare` is the missing step: it re-executes its argument already +inside podman's rootless user namespace, and from there `nsenter` can join +the container's pid and mount namespaces on top: + +```bash +[host]$ podman unshare nsenter --target "$HOSTPID" --pid --mount -- readlink /proc/1/exe +/usr/local/bin/app +``` + +{% include excalidraw.html + file="34-ns-debugging-map" + alt="A host band above a podman-user-namespace band. podman exec reaches the container PID 1 directly from inside the container runtime. podman unshare nsenter first joins podman's own user namespace, then nsenter --target --pid --mount reaches the same PID 1 from there -- both confirming Name: app and /proc/1/exe resolving to /usr/local/bin/app. A dashed, failing arrow shows plain nsenter without podman unshare returning EPERM: reassociate to namespaces failed. Below, real evidence boxes show the subuid mapping rsedor:524288:65536 and a live zombie scan of /proc/[0-9]*/stat reporting zero." + caption="Figure 34.3 -- podman exec and podman unshare nsenter reach the same container PID 1 from two different starting namespaces; plain nsenter cannot" %} + +A live `gdb` attach is the surprise worth reporting plainly: because +rootless podman's user namespace is *owned by the same host uid* that is +running `gdb`, ordinary ptrace permission checks succeed even with no +`podman unshare` step at all -- + +```bash +[host]$ gdb -batch -p "$HOSTPID" -ex "info proc exe" -ex detach -ex quit +warning: Target and debugger are in different PID namespaces; thread +lists and other data are likely unreliable. Connect to gdbserver inside +the container. +exe = '/usr/local/bin/app' +``` + +`gdb` attaches and correctly resolves the executable, but its own warning +names the limitation precisely: it is reading `/proc//*` from +*outside* the container's pid namespace, so anything that depends on pid +numbers matching what the container sees (`info threads`, breakpoints set +by a pid the container reports) is unreliable from here. A `gdbserver` +started with `podman exec` inside the container, with `gdb`'s `target +remote` reaching it from the host, is the reliable version of this same +cross-boundary attach -- the same shape chapter 28's remote-debugging +topology already covers. + +## Performance and security trade-offs + +Two costs are worth pricing accurately against what a container buys. The +**default seccomp profile** podman ships (`/usr/share/containers/seccomp.json` +on this host) declares `defaultAction: SCMP_ACT_ERRNO` with +`defaultErrnoRet: 38` (`ENOSYS`) for anything not on its allowlist -- and +`io_uring_setup`, `io_uring_enter`, and `io_uring_register` are not on it. +That is directly relevant after chapter 10: a program built to use +`io_uring` for its I/O will silently fail to set up a ring inside a +default-profile container. Probing the same call three ways on this host +makes the boundary observable rather than assumed: + +```bash +[host]$ python3 -c "..." # io_uring_setup(1, NULL) via ctypes.syscall(425, ...) +# bare host: -1 errno=14 (Bad address) -- reaches the kernel +# default podman seccomp profile: -1 errno=38 (Function not implemented) -- blocked before the kernel sees it +# --security-opt seccomp=unconfined: -1 errno=14 (Bad address) -- reaches the kernel again +``` + +`EFAULT` (14, "Bad address") is what the real kernel handler returns for a +null `params` pointer -- proof the call reached it. `ENOSYS` (38) under +the default profile is seccomp's own default-deny return, generated +without the syscall ever running; `seccomp=unconfined` removes the filter +and the exact same `EFAULT` reappears, isolating the profile as the cause. +Namespace and cgroup accounting add their own small, measured tax -- +every `read`/`write` a namespaced process makes still walks the same +cgroup memory accounting chapter 32 introduced, and every `cpu.max` +lookup this chapter's own `app` performs is one extra `/proc`+`/sys` read +at startup, not a per-syscall cost. What a container buys in return is +concrete and already demonstrated above: one image that carries its own +static-linked dependencies, a private pid/mount/user namespace so a +compromised process cannot see or touch the rest of the host's process +tree or filesystem by default, and a resource ceiling the kernel enforces +whether or not the program inside ever asks about it. The next chapter's +`overheadbench` puts hard numbers on the remaining rungs of this same +ladder -- container versus full KVM guest versus bare host. + +## Build, run, observe + +```bash +[host]$ python3 scripts/test-all-examples.py --only 34-our-programs-in-containers --mode local +``` + +Against podman 5.8.4 rootless on this host (Fedora Linux 44, kernel +`7.1.3-200.fc44.x86_64`, 16 cpus), a fresh build/run/teardown of all three +languages: + +``` +building 3 example-lang combinations (jobs=1)... + build 34-our-programs-in-containers [cpp]: ok + build 34-our-programs-in-containers [go]: ok + build 34-our-programs-in-containers [rust]: ok + +verifying... + verify 34-our-programs-in-containers [cpp]: PASS + verify 34-our-programs-in-containers [go]: PASS + verify 34-our-programs-in-containers [rust]: PASS + +example cpp go rust +34-our-programs-in-containers PASS PASS PASS + +3 passed, 0 failed, 0 skipped (logs in build-logs/) +``` + +By hand, one consolidated run against the Go image, constrained the same +way `verify.lua` constrains it: + +```bash +[host]$ podman run -d --name lsp34demo --cpus=2 --memory=128m lsp34-go serve +[host]$ podman logs lsp34demo | head -5 +container: cpu.max=200000/100000 effective_parallelism=2 mem.max=134217728 +app: pid=1 ppid=0 +app: worker started pid=8 +app: job pid=13 seq=1 done +app: reaped pid=13 status=0 +[host]$ podman stop -t 3 lsp34demo # elapsed_ms=87 +[host]$ podman logs lsp34demo | tail -1 +app: shutting down (SIGTERM) +``` + +The `naive` half of the trap, run the same way with a 2s grace instead of +3s, on all three images: `podman stop -t 2` measured `cpp` at 2108ms and +`rust` at 2074ms -- the full grace period, both `SIGKILL`ed -- while `go` +stopped in 60ms because its runtime installs its own fatal-`SIGTERM` +handler regardless of user code. Every one of the three logs, checked with +`grep -c "shutting down"`, reported zero: none of the three ever ran the +graceful path, which is exactly the point `naive` exists to make. + +## Cross-check + +`podman inspect` is independent of anything `app` prints about its own +limits, and it agrees with the container's self-report exactly: + +```bash +[host]$ podman inspect lsp34demo --format \ + 'CPUQuota={{.HostConfig.CpuQuota}} CPUPeriod={{.HostConfig.CpuPeriod}} Memory={{.HostConfig.Memory}}' +CPUQuota=200000 CPUPeriod=100000 Memory=134217728 +``` + +`200000/100000` from `podman inspect` matches `app`'s own +`cpu.max=200000/100000` line, and `134217728` matches `mem.max=134217728` +-- two independent readers of the same cgroup files agreeing. The `SIGTERM` +grace behavior is its own cross-check in miniature: `serve`'s stop time +(87-192ms across languages) sits nowhere near the 3-second grace budget +`podman stop -t 3` allows, while `naive`'s cpp/rust stop time (2074-2108ms) +sits right at the 2-second grace budget `podman stop -t 2` allows before +`SIGKILL` -- the timing itself is the proof that one path ran a graceful +shutdown and the other waited out a timeout. + +## What you learned + +- A multi-stage Containerfile ships only its last stage; pairing a full + `ubi10/ubi:10.2` builder (220 MB measured) with an `ubi10-micro:10.2` + runtime (24.3 MB measured) and copying one static-linked binary across + keeps the final image within a few MB of the bare runtime base. +- A container's PID 1 gets no default signal action for anything it has + not explicitly handled -- `naive`'s absent handler is why `podman stop` + has to wait out the full grace period and fall back to `SIGKILL`, and + why `serve`'s explicit forward-then-reap sequence is not optional + ceremony. +- "How many CPUs do I have" has three real answers under the same + `--cpus=2` constraint: C++'s `hardware_concurrency()` reports the host's + 16, while Go's cgroup-aware `GOMAXPROCS` default and Rust's + `available_parallelism()` both report the true 2 -- confirmed live on + this host, not merely documented. +- `podman exec`, `podman unshare nsenter`, and even a direct `gdb -p` + attach (because rootless podman's user namespace shares the host uid) + all reach the same container PID 1 from different starting points, while + the default seccomp profile silently denies `io_uring` calls that a + chapter-10-style program might depend on. + +Next, **virtualization and KVM**: the same daemons run a rung further out +-- inside a full guest kernel instead of sharing the host's -- and +`overheadbench` prices the difference in nanoseconds, gigabytes per +second, and operations per second, across the bare host, this chapter's +containers, and a real `systems-target` guest. + +--- + +

verified -- every +transcript and number above was produced this session on this host +(Fedora Linux 44, kernel 7.1.3-200.fc44.x86_64, 16 cpus, +podman 5.8.4 rootless): python3 scripts/test-all-examples.py --only +34-our-programs-in-containers --mode local printed +34-our-programs-in-containers PASS PASS PASS (3 passed, 0 +failed, 0 skipped); fresh podman builds measured +ubi10/ubi:10.2 at 220 MB, ubi10-micro:10.2 at +24.3 MB, and the three final images at cpp/go 27.2 MB and rust 24.9 MB; +ldd on binaries copied out of each image showed cpp linking +only libm/libc (no libstdc++), go +"not a dynamic executable", and rust linking only +libgcc_s/libc; a --cpus=2 --memory=128m +run reported cpu.max=200000/100000 and +mem.max=134217728 in all three languages with +effective_parallelism=16 for cpp against =2 for +go and rust, matching podman inspect's +CPUQuota=200000 CPUPeriod=100000 Memory=134217728; +podman exec, podman unshare nsenter, and a live +gdb -p attach all reached the same container PID 1 (exe +/usr/local/bin/app), while plain nsenter failed +EPERM; measured podman stop times were serve +cpp 192ms/go 60-87ms/rust 101ms (well within a 3s grace) against naive cpp +2108ms/rust 2074ms (full 2s grace, SIGKILLed)/go 60ms (runtime exception), +with zero shutting down lines from any naive +log; and the io_uring_setup probe returned +errno=14 (EFAULT) on the bare host and under +seccomp=unconfined, versus errno=38 (ENOSYS, +matching /usr/share/containers/seccomp.json's +defaultErrnoRet) under podman's default profile. Go 1.25/1.26's +container-aware GOMAXPROCS default and its +GODEBUG=containermaxprocs=0 opt-out were additionally checked +against the upstream Go blog and release notes at authoring time.

diff --git a/_docs/35-virtualization-and-kvm.md b/_docs/35-virtualization-and-kvm.md new file mode 100644 index 0000000..4d3080f --- /dev/null +++ b/_docs/35-virtualization-and-kvm.md @@ -0,0 +1,487 @@ +--- +title: "Virtualization and KVM" +order: 35 +part: "Containers and Virtualization" +description: "How a VM actually runs — VMX root/non-root modes, EPT, and virtio devices as the host kernel moonlighting as a hypervisor — the isolation spectrum from bare process through container and microVM to full VM, the rust-vmm/Firecracker/cloud-hypervisor landscape, and overheadbench's measured syscall/mem/io deltas across all three." +duration: "50 minutes" +--- + +Every daemon in this book has, until now, run either directly on the host or +inside a Podman container sharing the host's kernel. This chapter puts a +third option on the table: a full KVM guest with its own kernel, and asks the +one question a systems programmer actually needs answered before choosing +between them — not "which is more secure" in the abstract, but *what does +each one cost*, measured in the same units this book has used throughout: +nanoseconds per syscall, gigabytes per second, operations per second. +`overheadbench` is a tri-language microbenchmark built for exactly this: the +same binary, unmodified, runs on the bare host, inside the `systems-target` +KVM guest, and inside a rootless Podman container, and the three runs land in +one table. The isolation spectrum this chapter walks — process, container, +microVM, full VM — is not a taxonomy exercise; it is the same spectrum every +one of this book's daemons could be deployed onto, and this is the chapter +that prices each rung. + +The code is in `examples/35-virtualization-and-kvm/`. The run script there +builds and runs it; its `README.md` covers the CLI and the sanity ranges +`verify.lua` checks. + +{% include excalidraw.html + file="35-kvm-virtio-stack" + alt="Three horizontal bands. Top: host, VMX root mode, showing kvm.ko and kvm_intel.ko managing EPT tables, a QEMU/VMM userspace box running the vCPU thread and virtio-blk backend, and host NVMe/btrfs where a real fsync lands. Middle: guest, VMX non-root mode, showing the guest kernel serving getpid in-guest with no host trap, overheadbench in guest ring 3 issuing the syscall bench's raw getpid loop and the io bench's open/write/fsync calls, a virtio-blk front-end driver, and a virtqueue descriptor ring. Bottom: the trapped path for fsync — fsync in the guest causes a VM exit, the KVM exit handler hands off to the QEMU vCPU thread, the virtio-blk backend issues the real host fsync and triggers VM enter back into the guest. A fourth band gives the measured overheadbench deltas: syscall about 88ns/call on both host and guest because getpid never triggers a vmexit, mem about 14 to 15 GB/s on both because the EPT walk cost is amortized after the TLB warms up, and io about 1.3k ops/s on the host against about 0.5k ops/s on the guest, the real vmexit tax." + caption="Figure 35.1 — the VMX root/non-root boundary, the virtio-blk round trip that fsync actually pays, and where overheadbench's own measured host-vs-guest deltas land" %} + +> **Tools used** — `overheadbench` itself, `podman` (build and run, host), +> `ssh`/`scp` via `scripts/lab/deploy-to-vm.sh` and the Python runner +> (`scripts/test-all-examples.py`) (host), `virsh` (host, confirming the lab +> is up), `lscpu` (host and `systems-target` guest), `strace -c` (host and +> guest, the cross-check), `systemd-detect-virt` (guest). All of these are +> checked by `scripts/check-host.sh` or ship with Fedora's cloud image on the +> lab VMs. + +## What actually runs a VM + +A container's isolation is namespaces and cgroups layered over *one* kernel; +every syscall a containerized process makes is a syscall the host kernel +itself serves, at host cost. A KVM guest is a different animal: it has its +own kernel, its own page tables, its own idea of what hardware exists — and +the host kernel is the thing making that illusion real. Since 2006, Intel and +AMD CPUs have shipped hardware virtualization extensions (VMX on Intel, +confirmed on this host by `lscpu`'s `Virtualization: VT-x` line) that add a +second CPU mode dimension alongside ring 0/ring 3: **VMX root mode**, where +the host kernel and hypervisor run, and **VMX non-root mode**, where the +*entire* guest — its kernel at ring 0, its processes at ring 3 — runs mostly +unmodified. `/dev/kvm`, backed by the `kvm` and `kvm_intel` kernel modules +(both loaded on this host, `lsmod | grep kvm` confirms `kvm_intel`, `kvm`, +and `irqbypass`), is the interface QEMU drives: it calls `ioctl(KVM_RUN, ...)` +on a vCPU file descriptor, the CPU drops into non-root mode, and guest +instructions execute directly on the real CPU — no instruction-level +emulation — until something the guest cannot handle itself forces a **VM +exit** back to root mode. + +The key fact this chapter measures is *what actually causes an exit*. +Ordinary instructions — arithmetic, memory loads and stores, and crucially +**syscalls the guest kernel can serve itself**, like `getpid(2)` — never +leave non-root mode at all. The guest kernel's syscall entry point runs +exactly like it would on bare metal; there is no hypervisor in that path to +trap into. What *does* exit are instructions that touch state the guest does +not really own: port I/O, certain MSR accesses, and — the one this chapter's +`io` bench hits on every iteration — an access to an emulated device like +`virtio-blk`, the paravirtualized block device virtio defines so guest I/O +doesn't have to trap on every disk instruction the way full hardware +emulation would. `virtio-blk` still requires a round trip: the guest driver +places a request descriptor on a shared-memory virtqueue, the access that +notifies the host ("kick") causes a VM exit, QEMU's backend thread (in root +mode again) issues the *real* `fsync(2)` against the host filesystem, and a +VM entry resumes the guest. Extended Page Tables (EPT) are the third piece: +they let the guest maintain its own page tables while the CPU walks a second, +hardware-managed table translating guest-physical to host-physical addresses, +so ordinary memory access needs no hypervisor involvement either — only the +first touch of a page, which fills the EPT-backed TLB entry, costs anything +beyond native. + +## The isolation spectrum + +{% include excalidraw.html + file="35-isolation-spectrum" + alt="Four boxes left to right along a spectrum from shared host kernel to separate guest kernel. Bare process: no isolation boundary, syscall equals host syscall, escape equals root on host. Podman container: namespaces plus cgroup v2 plus seccomp, one host kernel and one syscall table, syscall and mem about equal host and io about equal host on a bind mount, escape equals a kernel bug in namespace or cgroup code. MicroVM, Firecracker or cloud-hypervisor built on rust-vmm: own guest kernel with a minimal virtio device set, about 125 milliseconds boot and under 5 MiB overhead for Firecracker, escape equals a bug in the roughly five devices the VMM exposes or in KVM itself. Full KVM/QEMU VM, the systems-target lab guest: own guest kernel with a full device model, syscall about equal host and io about three times slower due to virtio-blk, escape equals a bug in the full emulated device model or in KVM. A note below cites that rust-vmm crates such as kvm-ioctls, virtio-queue, and vm-memory underpin both Firecracker v1.16.1 and Cloud Hypervisor v53.0." + caption="Figure 35.2 — process, container, microVM, and full VM, each priced by escape surface and by this chapter's measured overheadbench numbers" %} + +Four points, not two. A **bare process** has no isolation boundary at all — a +syscall it makes *is* a host syscall, full stop, and an escape is simply +"you are already root, or you exploit a bug to become root." A **container** +(Chapter 34's Podman UBI images) adds namespaces and cgroup v2 as a soft +boundary layered over the *same* kernel: still one syscall table, so +`overheadbench`'s `syscall` and `mem` numbers inside a container should track +the host almost exactly — and the `io` bench, run against a directory bind- +mounted from the host's real filesystem rather than the container's overlay +storage, does too, because it is still the host's block layer doing the +work. The escape surface for a container is a kernel bug in the +namespace/cgroup/seccomp code path itself — the code the shared kernel runs +on the container's behalf. + +A **full KVM/QEMU VM** (`systems-target`, this book's lab guest) is the +opposite end: its own kernel, its own process table, its own view of +hardware, with a full emulated device model — network, block, USB, ACPI, +BIOS. The escape surface shrinks to "a bug in that entire device model, or in +KVM's guest/host boundary itself" — a much smaller, better-audited attack +surface than "the whole host kernel," at the cost of a full second kernel to +boot, patch, and pay overhead for on every path that has to leave the guest. + +Between them sits the **microVM** — Firecracker and cloud-hypervisor's +answer to "a real second kernel, but priced like a container." A microVM +still runs its own guest kernel under KVM, so the escape surface is the same +category as a full VM, but the device model is deliberately reduced: no +BIOS, no ACPI, no graphics, no USB — Firecracker (v1.16.1 as of this +writing) implements exactly five devices: virtio-net, virtio-block, +virtio-vsock, a serial console, and a minimal keyboard controller. That +minimalism is why Firecracker boots a guest in roughly 125 ms with under +5 MiB of per-instance memory overhead, and why AWS runs it under Lambda and +Fargate at up to 150 microVMs per second per host. Both Firecracker and +Cloud Hypervisor (v53.0, which added VFIO passthrough, CPU/memory hotplug, +and — recently — Windows guest support) are built from **rust-vmm**: a +community of reusable, independently audited Rust crates — +`kvm-ioctls`/`kvm-bindings` for the `/dev/kvm` ioctl surface, +`vm-memory` for guest address space management, `virtio-queue` and +per-device crates for the virtio transport. Firecracker (AWS) helped seed +rust-vmm; Cloud Hypervisor (originally Intel, now Linux Foundation-governed) +leans on the same crates for a broader feature set. The microVM's promise is +literally this chapter's spectrum collapsed to one point: KVM's isolation +guarantee at closer to container startup cost — which is why it is the +answer teams reach for when "container" isn't isolated enough and "full VM" +is too slow to spin up per request. + +## How the code works + +`overheadbench`'s three benches are intentionally tiny and identical across +languages, because the *comparison* is the content, not the algorithm. The +one worth walking in detail is `syscall`, since it is the bench whose result +this chapter's whole VMX argument hinges on: + +{% include codetabs.html langs="C++|Go|Rust" %} + +```cpp +// --- syscall bench: tight getpid(2) loop, ns/call. -------------------------- +double bench_syscall(std::uint64_t iters) { + const auto start = Clock::now(); + for (std::uint64_t i = 0; i < iters; ++i) { + // A raw syscall (not the libc wrapper) so every iteration really + // crosses into the kernel — this is the syscall-boundary cost the + // chapter contrasts against VM (vmexit trap) and container + // (near-zero extra) overhead. + ::syscall(SYS_getpid); + } + const auto elapsed = Clock::now() - start; + double ns = std::chrono::duration_cast>(elapsed).count(); + if (ns <= 0.0) { + ns = 1.0; + } + return ns / static_cast(iters); +} +``` + +```go +// benchSyscallRun: tight getpid(2) loop, ns/call. +func benchSyscallRun(iters uint64) float64 { + start := time.Now() + for i := uint64(0); i < iters; i++ { + // unix.Getpid issues the raw getpid(2) syscall on every call (the Go + // runtime does not cache it) — every iteration really crosses into + // the kernel, the syscall-boundary cost this chapter contrasts + // against VM (vmexit trap) and container (near-zero extra) overhead. + unix.Getpid() + } + ns := float64(time.Since(start).Nanoseconds()) + if ns <= 0 { + ns = 1 + } + return ns / float64(iters) +} +``` + +```rust +/// Tight getpid(2) loop, ns/call. +fn bench_syscall(iters: u64) -> f64 { + let start = Instant::now(); + for _ in 0..iters { + // rustix::process::getpid issues the raw getpid(2) syscall every + // call — every iteration really crosses into the kernel, the + // syscall-boundary cost this chapter contrasts against VM (vmexit + // trap) and container (near-zero extra) overhead. + black_box(rustix::process::getpid()); + } + let mut ns = start.elapsed().as_nanos() as f64; + if ns <= 0.0 { + ns = 1.0; + } + ns / iters as f64 +} +``` + +Three deliberate choices, identical in all three: the loop calls a **raw** +syscall rather than a libc-cached identity (glibc does not cache `getpid` +today, but the explicit `syscall(SYS_getpid)`/`unix.Getpid`/`rustix::getpid` +spelling makes that guarantee visible in the source, not implicit in a libc +version); the clock (`std::chrono::steady_clock`, `time.Now`/`Since`, +`Instant::now`) is a **monotonic** one, because Chapter 24 already +established that a wall-clock read can step backward mid-measurement; and +each language guards the compiler from proving the loop is dead — +`black_box` in Rust wraps the return value, C++ takes no extra step here +because the syscall itself is an unremovable side effect, and the Go build +relies on the same property. The `mem` bench needs an explicit anti-dead-code +guard because summing a buffer *is* pure, elidable computation: C++ stashes +the sum in a `volatile` sink, Rust calls `black_box(sum)`, Go checks the sum +against an unreachable sentinel inside an `if` whose body writes to stderr — +three different languages' answer to the same compiler-optimizer problem. +The `io` bench's fragile-but-deliberate choice is documented in its own +comment: the scratch directory is created **relative to the current working +directory**, not the system temp directory, because on this host (and many +dev hosts) `/tmp` is `tmpfs` — `fsync` there is nearly free, and the number +stops measuring storage at all. `examples/` sits on real NVMe-backed btrfs, +which is what makes the `io` numbers below meaningful as a storage-overhead +comparison rather than a memcpy benchmark in disguise. + +The other half of "how the code works" for this chapter is the multi-stage +`Containerfile` that gets the same Go binary into a rootless Podman +container — `examples/35-virtualization-and-kvm/go/Containerfile`, +reproduced verbatim: + +``` +# Multi-stage build for overheadbench (chapter 35). Build stage compiles a +# fully static binary against UBI 10's go-toolset (go1.26.5 — the same +# toolchain pin the host uses); the runtime stage is ubi10/ubi-micro, which +# carries no shell and no package manager, so the final image is just the +# static binary plus the handful of files ubi-micro itself ships. +FROM registry.access.redhat.com/ubi10/go-toolset:latest AS build +# Stay in the s2i image's own /opt/app-root/src: it is already writable by +# the image's arbitrary non-root uid (1001, gid 0); a fresh WORKDIR would be +# created root-owned and rootless podman's build user could not write to it. +COPY go.mod go.sum ./ +RUN go mod download +COPY main.go ./ +# CGO_ENABLED=0: unix.Getpid() is a raw syscall, no libc needed, so a fully +# static binary can run unmodified on ubi-micro's minimal userspace. +RUN CGO_ENABLED=0 go build -o overheadbench . + +FROM registry.access.redhat.com/ubi10/ubi-micro:latest +COPY --from=build /opt/app-root/src/overheadbench /usr/local/bin/overheadbench +# Rootless podman default: container runs as the invoking user's mapped uid, +# not root, and writes its scratch io-bench files into WORKDIR (set at run +# time via -w so the io bench lands on the mounted volume, not container +# storage). +ENTRYPOINT ["/usr/local/bin/overheadbench"] +CMD ["--bench", "all"] +``` + +The build stage is `ubi10/go-toolset` — Red Hat's s2i-flavored Go 1.26.5 +image, the exact same toolchain pin this book uses on the host, so the +container's binary is not just architecturally comparable to the host's, it +is a build of the identical source with the identical compiler version. It +compiles with `CGO_ENABLED=0`: `getpid(2)` needs no libc, so the result is a +fully static binary that runs unmodified on `ubi10/ubi-micro`, a runtime +image so minimal it has no shell, no package manager, nothing but the +handful of files UBI's licensing and CA-bundle policy require. The one +build-time trap worth stating plainly: setting an explicit `WORKDIR` before +the `COPY`/`RUN` steps would have the directory created root-owned at the +point the instruction runs, and every later `RUN` in a rootless build +executes as the image's own non-root user (uid 1001) — the fix is to stay in +the s2i image's pre-existing `/opt/app-root/src`, which the base image +already made writable for that uid. + +## Errors, three ways + +`overheadbench`'s error contract is the same shape this book has used since +Chapter 5: **argument-parsing failures exit 2** with the usage line on +stderr, and **bench-execution failures exit 1** without it, kept as two +distinct exit codes so a caller can tell "you asked wrong" from "it tried and +the environment failed." An unknown `--bench` value, a flag missing its +value, an unrecognized flag, or a non-numeric or zero `--iters` all land in +the first bucket in all three languages: C++'s `parse_args` returns +`std::expected` and the `main` that unwraps it prints +`cfg.error()` then `kUsage` before returning 2; Go's `parseArgs` returns a +plain `error` that `main` prints beside the same `usage` constant; Rust's +`parse_args` returns `Result` and the `main` match arm +does the identical two-line print. The second bucket — the `io` bench's +`open`/`write`/`fsync` calls actually failing, which none of the runs in +this chapter hit but which a full disk or a read-only bind mount would +trigger — propagates through `std::expected`, +`(float64, error)`, and `Result` respectively, each wrapped one +more time (`"io bench failed: " + ...`) so the message names which of the +three phases failed, and exits 1 with no usage line, because the arguments +were fine — the environment was not. + +## Concurrency lens + +This chapter's isolation spectrum is built entirely from **per-thread kernel +state**, and that constraint shows up on both the container and the VM side +in the same shape Chapter 14 already met with capabilities. On the VM side: +a vCPU file descriptor returned by `KVM_CREATE_VCPU` is only valid to drive +with `ioctl(KVM_RUN, ...)` **from the thread that created it** — call it from +any other thread and the ioctl fails. That is why QEMU pins exactly one host +thread per vCPU rather than dispatching `KVM_RUN` calls from a thread pool; +the vCPU's register state, its `KVM_RUN` mmap'd shared page, and its +in-kernel scheduling context are thread-local, not process-wide. On the +container side, entering an existing set of namespaces with `setns(2)` has +the *identical* constraint: namespace membership is per-thread state in the +kernel's `task_struct`, not a process-wide property, exactly like the uids +and capability sets Chapter 14 walked. A single-threaded C++ or Rust build +that calls `setns` then immediately does its namespaced work never notices; +a Go program is the case that bites, because the Go runtime is +multi-threaded and goroutines migrate between OS threads by default — a +goroutine that calls `setns` and then, without more, tries to act as though +it is still in the new namespace can be rescheduled onto a *different* OS +thread that never made that call. The fix is the same shape as Chapter 14's +Go build pushing the capability dance into the fork/exec child: +`runtime.LockOSThread()` before `setns`, pinning the calling goroutine to its +current OS thread for the rest of its life (or until `UnlockOSThread`), so +the namespace membership and the goroutine that believes it holds it stay +attached to the same kernel thread. The general rule repeats from Chapter +14: whenever a kernel API's state lives on the thread rather than the +process — credentials, capabilities, namespace membership, a KVM vCPU — a +runtime that hides its threading from you (Go's goroutines, but the same +risk applies to any green-thread or fiber scheduler) needs an explicit pin, +or the state and the code that thinks it owns the state quietly drift apart. + +## Build, run, observe + +Three environments, one binary. First, the bare host (Fedora 44, kernel +7.1.3-200.fc44, Intel i7-11800H, `Virtualization: VT-x` from `lscpu`): + +```bash +[host]$ cd examples/35-virtualization-and-kvm +[host]$ ./demo.sh go run --bench all +bench=syscall metric=84.84 unit=ns/call +bench=mem metric=12.59 unit=GB/s +bench=io metric=1504.89 unit=ops/s +``` + +Then the same binary deployed to the `systems-target` KVM guest (Fedora 44, +kernel 6.19.10-300.fc44, confirmed by `lscpu` in the guest as +`Virtualization type: full`, and by `systemd-detect-virt` printing `kvm`): + +```bash +[host]$ export LIBVIRT_DEFAULT_URI=qemu:///system TARGET=systems-target +[host]$ ./demo.sh go run --bench all +→ copying app to fedora@192.168.124.7:/home/fedora/app +→ running on systems-target (Ctrl-C to stop): +bench=syscall metric=87.69 unit=ns/call +bench=mem metric=14.32 unit=GB/s +bench=io metric=521.24 unit=ops/s +``` + +And the same source, rebuilt through the multi-stage `Containerfile` above, +run rootless with the `io` bench's scratch directory bind-mounted to a real +host path (`-w /scratch`, not the container's own overlay storage): + +```bash +[host]$ cd go +[host]$ podman build -t overheadbench-go -f Containerfile . +[host]$ mkdir -p container-scratch +[host]$ podman run --rm -v "$PWD/container-scratch:/scratch:Z" -w /scratch overheadbench-go --bench all +bench=syscall metric=94.01 unit=ns/call +bench=mem metric=15.39 unit=GB/s +bench=io metric=1473.58 unit=ops/s +``` + +Three repeats of each (host: 84.8–91.8 ns/call, 12.6–15.5 GB/s, 1105–1505 +ops/s; guest: 87.7–88.1 ns/call, 14.3–15.0 GB/s, 463–521 ops/s; container: +94.0–102.8 ns/call, 13.5–15.4 GB/s, 1474–1524 ops/s) put the pattern beyond +noise: **`syscall` and `mem` land in the same band across all three +environments** — a container shares the host's syscall table entirely, and +the guest's `getpid` never triggers a VM exit at all, matching Figure 35.1's +claim exactly. **`io` is where the VM actually pays**: roughly 2.9–3× slower +in the guest than on the host or in the container, the measured cost of +every `fsync` taking the virtio-blk round trip this chapter's diagram walks +— VM exit, QEMU's backend thread, the real host `fsync`, VM entry — instead +of running straight through in-kernel like the container's bind-mounted +write does. The full cross-language sweep on the host, run through this +book's standard runner, confirms the three languages track each other: + +```bash +[host]$ python3 scripts/test-all-examples.py --only 35-virtualization-and-kvm --mode local +example cpp go rust +35-virtualization-and-kvm PASS PASS PASS + +3 passed, 0 failed, 0 skipped (logs in build-logs/) +``` + +## Cross-check + +`overheadbench` prints its own timing, so an independent check has to verify +the printed numbers correspond to real syscalls actually made — not just +trust the arithmetic. `strace -c`, run on both sides of the VMX boundary, +does exactly that by counting syscalls a second way. On the host, wrapping +`--bench io --iters 50`: + +```bash +[host]$ strace -f -c -e trace=getpid,openat,write,fsync,close,unlink ./go/bin/app --bench io --iters 50 +bench=io metric=1233.15 unit=ops/s +% time seconds usecs/call calls errors syscall +------ ----------- ----------- --------- --------- ---------------- + 68.97 0.004160 83 50 fsync + 17.11 0.001032 19 54 openat + 7.68 0.000463 9 51 write + 6.07 0.000366 6 53 close +------ ----------- ----------- --------- --------- ---------------- +100.00 0.006032 28 210 total +``` + +Exactly 50 `fsync` calls for `--iters 50` — the count `overheadbench` claims +to have run is the count the kernel actually saw, independent of the +program's own bookkeeping. Run identically on `systems-target`: + +```bash +[vm]$ strace -f -c -e trace=openat,write,fsync,close,unlink /home/fedora/app --bench io --iters 50 +bench=io metric=512.96 unit=ops/s +------ ----------- ----------- --------- --------- ---------------- + 74.81 0.004972 99 50 fsync +------ ----------- ----------- --------- --------- ---------------- +``` + +Same 50 `fsync` calls, and the *unstraced* metric it printed (512.96 ops/s) +falls right inside the guest's 463–521 ops/s band from the build/run/observe +section above — the trace confirms the count without disturbing the +comparison this chapter is making. Note what `strace` cannot show, on either +side: it observes the syscall the guest kernel serves, never the VM exit +that syscall may or may not trigger underneath. `strace -c`'s own reported +`usecs/call` for `getpid` under tracing was 9,139 ns — two orders of +magnitude above the untraced ~85 ns this chapter reports — because `ptrace` +stops the tracee on every syscall entry and exit; that overhead is exactly +why `overheadbench` times itself with its own monotonic clock instead of +trusting a tracer's numbers, the same "the observation tool changes what it +measures" caveat Chapter 14 hit with `strace` across a privilege drop. There +is no dashboard panel for this chapter's numbers by design — `overheadbench` +is a one-shot CLI, and the table above *is* the observation surface. + +## What you learned + +- A VM exit is not triggered by "leaving the guest," it is triggered by + the guest touching state it does not itself own — `getpid(2)` never + exits, but `fsync(2)` against `virtio-blk` does, which is exactly why + `overheadbench`'s `syscall`/`mem` numbers track the host across all three + environments while `io` is the one that is measurably ~3× slower in a full + KVM guest. +- The isolation spectrum has four rungs, not two: bare process, container + (shared kernel, namespaces/cgroups as a soft boundary), microVM + (rust-vmm-built Firecracker v1.16.1 / Cloud Hypervisor v53.0 — a real + guest kernel behind a deliberately minimal device model), and full + KVM/QEMU VM — each trading escape-surface size against boot time and + per-path overhead. +- Kernel state that lives per-thread, not per-process, is a recurring trap: + a KVM vCPU fd must be driven from the thread that created it, and + `setns(2)` namespace membership is equally per-thread — a Go program must + call `runtime.LockOSThread()` before either, the same lesson Chapter 14's + capability drop taught for uids and capability sets. +- A benchmark's own numbers need an independent check: `strace -c`'s exact + syscall counts confirmed `overheadbench`'s claimed iteration counts on + both host and guest, while also showing where that same tool's overhead + (`ptrace`'s stop-on-every-syscall cost) makes it unfit to trust for timing. + +Next, Part 10 opens with **observability** — turning `/proc`, `/sys`, and +cgroup statistics into the kind of live picture this chapter had to build by +hand, one `overheadbench` run at a time. + +--- + +

verified — every transcript +and number above was produced this session. Host: Fedora 44, kernel +7.1.3-200.fc44.x86_64, Intel i7-11800H, lscpu reporting +Virtualization: VT-x, lsmod showing +kvm_intel/kvm/irqbypass loaded; +python3 scripts/test-all-examples.py --only 35-virtualization-and-kvm +--mode local printed PASS PASS PASS (3 passed, 0 +failed, 0 skipped). Guest: systems-target (Fedora 44, kernel +6.19.10-300.fc44.x86_64), lscpu reporting +Virtualization type: full, systemd-detect-virt +printing kvm, DMI sys_vendor=QEMU, +product_name=Standard PC (Q35 + ICH9, 2009). The +host/guest/container overheadbench table (syscall, mem, io) was +read directly off ./demo.sh go run, TARGET=systems-target +./demo.sh go run, and podman build + +podman run against the multi-stage Containerfile +in examples/35-virtualization-and-kvm/go/, each repeated three +times; the strace -c cross-check on both host and guest showed +exactly 50 fsync calls for --iters 50. Firecracker +v1.16.1 (released 2026-07-02) and Cloud Hypervisor v53.0 (released +2026-07-12) version numbers were confirmed against each project's GitHub +releases at authoring time and are not independently re-verified by any run +in this lab.

diff --git a/assets/diagrams/32-cgroup-hierarchy.excalidraw b/assets/diagrams/32-cgroup-hierarchy.excalidraw new file mode 100644 index 0000000..cdf1c96 --- /dev/null +++ b/assets/diagrams/32-cgroup-hierarchy.excalidraw @@ -0,0 +1,1353 @@ +{ + "type": "excalidraw", + "version": 2, + "source": "https://excalidraw.com", + "elements": [ + { + "id": "r606001073", + "type": "rectangle", + "x": 20, + "y": 20, + "width": 940, + "height": 130, + "angle": 0, + "strokeColor": "#dddddd", + "backgroundColor": "#eef4fb", + "fillStyle": "solid", + "strokeWidth": 2, + "strokeStyle": "solid", + "roughness": 1, + "opacity": 100, + "groupIds": [], + "frameId": null, + "roundness": { + "type": 3 + }, + "seed": 407770115, + "versionNonce": 351764233, + "isDeleted": false, + "boundElements": [], + "updated": 1, + "link": null, + "locked": false + }, + { + "id": "t65247963", + "type": "text", + "x": 32, + "y": 26, + "width": 442.20000000000005, + "height": 15.0, + "angle": 0, + "strokeColor": "#555555", + "backgroundColor": "transparent", + "fillStyle": "solid", + "strokeWidth": 1, + "strokeStyle": "solid", + "roughness": 1, + "opacity": 100, + "groupIds": [], + "frameId": null, + "roundness": null, + "seed": 913233213, + "versionNonce": 1086167278, + "isDeleted": false, + "boundElements": [], + "updated": 1, + "link": null, + "locked": false, + "text": "/sys/fs/cgroup (root \u2014 exempt from the \"no internal process\" rule)", + "fontSize": 12, + "fontFamily": 1, + "textAlign": "left", + "verticalAlign": "top", + "containerId": null, + "originalText": "/sys/fs/cgroup (root \u2014 exempt from the \"no internal process\" rule)", + "lineHeight": 1.25, + "baseline": 12 + }, + { + "id": "r1247526288", + "type": "rectangle", + "x": 20, + "y": 170, + "width": 940, + "height": 260, + "angle": 0, + "strokeColor": "#dddddd", + "backgroundColor": "#fff8ef", + "fillStyle": "solid", + "strokeWidth": 2, + "strokeStyle": "solid", + "roughness": 1, + "opacity": 100, + "groupIds": [], + "frameId": null, + "roundness": { + "type": 3 + }, + "seed": 1449742359, + "versionNonce": 1021626255, + "isDeleted": false, + "boundElements": [], + "updated": 1, + "link": null, + "locked": false + }, + { + "id": "t322898658", + "type": "text", + "x": 32, + "y": 176, + "width": 409.20000000000005, + "height": 15.0, + "angle": 0, + "strokeColor": "#555555", + "backgroundColor": "transparent", + "fillStyle": "solid", + "strokeWidth": 1, + "strokeStyle": "solid", + "roughness": 1, + "opacity": 100, + "groupIds": [], + "frameId": null, + "roundness": null, + "seed": 993907072, + "versionNonce": 1921835097, + "isDeleted": false, + "boundElements": [], + "updated": 1, + "link": null, + "locked": false, + "text": "/sys/fs/cgroup/pmon-oomdemo (leaf, created by setup_cgroup())", + "fontSize": 12, + "fontFamily": 1, + "textAlign": "left", + "verticalAlign": "top", + "containerId": null, + "originalText": "/sys/fs/cgroup/pmon-oomdemo (leaf, created by setup_cgroup())", + "lineHeight": 1.25, + "baseline": 12 + }, + { + "id": "r612626668", + "type": "rectangle", + "x": 20, + "y": 450, + "width": 940, + "height": 150, + "angle": 0, + "strokeColor": "#dddddd", + "backgroundColor": "#f4f4f4", + "fillStyle": "solid", + "strokeWidth": 2, + "strokeStyle": "solid", + "roughness": 1, + "opacity": 100, + "groupIds": [], + "frameId": null, + "roundness": { + "type": 3 + }, + "seed": 127954039, + "versionNonce": 109898593, + "isDeleted": false, + "boundElements": [], + "updated": 1, + "link": null, + "locked": false + }, + { + "id": "t374508796", + "type": "text", + "x": 32, + "y": 456, + "width": 310.20000000000005, + "height": 15.0, + "angle": 0, + "strokeColor": "#555555", + "backgroundColor": "transparent", + "fillStyle": "solid", + "strokeWidth": 1, + "strokeStyle": "solid", + "roughness": 1, + "opacity": 100, + "groupIds": [], + "frameId": null, + "roundness": null, + "seed": 670570085, + "versionNonce": 1634236293, + "isDeleted": false, + "boundElements": [], + "updated": 1, + "link": null, + "locked": false, + "text": "kernel \u2014 real dmesg from this session's OOM run", + "fontSize": 12, + "fontFamily": 1, + "textAlign": "left", + "verticalAlign": "top", + "containerId": null, + "originalText": "kernel \u2014 real dmesg from this session's OOM run", + "lineHeight": 1.25, + "baseline": 12 + }, + { + "id": "r113465090", + "type": "rectangle", + "x": 50, + "y": 60, + "width": 420, + "height": 60, + "angle": 0, + "strokeColor": "#111111", + "backgroundColor": "#ffffff", + "fillStyle": "solid", + "strokeWidth": 2, + "strokeStyle": "solid", + "roughness": 1, + "opacity": 100, + "groupIds": [], + "frameId": null, + "roundness": { + "type": 3 + }, + "seed": 986990948, + "versionNonce": 750811409, + "isDeleted": false, + "boundElements": [], + "updated": 1, + "link": null, + "locked": false + }, + { + "id": "t945753080", + "type": "text", + "x": 60, + "y": 68, + "width": 138.60000000000002, + "height": 17.5, + "angle": 0, + "strokeColor": "#111111", + "backgroundColor": "transparent", + "fillStyle": "solid", + "strokeWidth": 1, + "strokeStyle": "solid", + "roughness": 1, + "opacity": 100, + "groupIds": [], + "frameId": null, + "roundness": null, + "seed": 216577494, + "versionNonce": 1505939531, + "isDeleted": false, + "boundElements": [], + "updated": 1, + "link": null, + "locked": false, + "text": "cgroup.controllers", + "fontSize": 14, + "fontFamily": 1, + "textAlign": "left", + "verticalAlign": "top", + "containerId": null, + "originalText": "cgroup.controllers", + "lineHeight": 1.25, + "baseline": 14 + }, + { + "id": "t195884962", + "type": "text", + "x": 60, + "y": 88, + "width": 290.40000000000003, + "height": 13.75, + "angle": 0, + "strokeColor": "#555555", + "backgroundColor": "transparent", + "fillStyle": "solid", + "strokeWidth": 1, + "strokeStyle": "solid", + "roughness": 1, + "opacity": 100, + "groupIds": [], + "frameId": null, + "roundness": null, + "seed": 1225661664, + "versionNonce": 1327362839, + "isDeleted": false, + "boundElements": [], + "updated": 1, + "link": null, + "locked": false, + "text": "cpuset cpu io memory hugetlb pids rdma misc dmem", + "fontSize": 11, + "fontFamily": 1, + "textAlign": "left", + "verticalAlign": "top", + "containerId": null, + "originalText": "cpuset cpu io memory hugetlb pids rdma misc dmem", + "lineHeight": 1.25, + "baseline": 11 + }, + { + "id": "r259195472", + "type": "rectangle", + "x": 500, + "y": 60, + "width": 420, + "height": 60, + "angle": 0, + "strokeColor": "#e8870c", + "backgroundColor": "#fff8ef", + "fillStyle": "solid", + "strokeWidth": 2, + "strokeStyle": "solid", + "roughness": 1, + "opacity": 100, + "groupIds": [], + "frameId": null, + "roundness": { + "type": 3 + }, + "seed": 1988999969, + "versionNonce": 649291843, + "isDeleted": false, + "boundElements": [], + "updated": 1, + "link": null, + "locked": false + }, + { + "id": "t593095944", + "type": "text", + "x": 510, + "y": 68, + "width": 169.4, + "height": 17.5, + "angle": 0, + "strokeColor": "#111111", + "backgroundColor": "transparent", + "fillStyle": "solid", + "strokeWidth": 1, + "strokeStyle": "solid", + "roughness": 1, + "opacity": 100, + "groupIds": [], + "frameId": null, + "roundness": null, + "seed": 1787795640, + "versionNonce": 1661425886, + "isDeleted": false, + "boundElements": [], + "updated": 1, + "link": null, + "locked": false, + "text": "cgroup.subtree_control", + "fontSize": 14, + "fontFamily": 1, + "textAlign": "left", + "verticalAlign": "top", + "containerId": null, + "originalText": "cgroup.subtree_control", + "lineHeight": 1.25, + "baseline": 14 + }, + { + "id": "t423252069", + "type": "text", + "x": 510, + "y": 88, + "width": 223.85000000000002, + "height": 13.75, + "angle": 0, + "strokeColor": "#555555", + "backgroundColor": "transparent", + "fillStyle": "solid", + "strokeWidth": 1, + "strokeStyle": "solid", + "roughness": 1, + "opacity": 100, + "groupIds": [], + "frameId": null, + "roundness": null, + "seed": 1237143626, + "versionNonce": 1627866669, + "isDeleted": false, + "boundElements": [], + "updated": 1, + "link": null, + "locked": false, + "text": "+memory +cpu (delegated to children)", + "fontSize": 11, + "fontFamily": 1, + "textAlign": "left", + "verticalAlign": "top", + "containerId": null, + "originalText": "+memory +cpu (delegated to children)", + "lineHeight": 1.25, + "baseline": 11 + }, + { + "id": "r1202691552", + "type": "rectangle", + "x": 50, + "y": 210, + "width": 270, + "height": 55, + "angle": 0, + "strokeColor": "#e8870c", + "backgroundColor": "#fff8ef", + "fillStyle": "solid", + "strokeWidth": 2, + "strokeStyle": "solid", + "roughness": 1, + "opacity": 100, + "groupIds": [], + "frameId": null, + "roundness": { + "type": 3 + }, + "seed": 1794455566, + "versionNonce": 446299959, + "isDeleted": false, + "boundElements": [], + "updated": 1, + "link": null, + "locked": false + }, + { + "id": "t1007978663", + "type": "text", + "x": 60, + "y": 218, + "width": 77.0, + "height": 17.5, + "angle": 0, + "strokeColor": "#111111", + "backgroundColor": "transparent", + "fillStyle": "solid", + "strokeWidth": 1, + "strokeStyle": "solid", + "roughness": 1, + "opacity": 100, + "groupIds": [], + "frameId": null, + "roundness": null, + "seed": 1554301156, + "versionNonce": 386468128, + "isDeleted": false, + "boundElements": [], + "updated": 1, + "link": null, + "locked": false, + "text": "memory.max", + "fontSize": 14, + "fontFamily": 1, + "textAlign": "left", + "verticalAlign": "top", + "containerId": null, + "originalText": "memory.max", + "lineHeight": 1.25, + "baseline": 14 + }, + { + "id": "t1235517058", + "type": "text", + "x": 60, + "y": 238, + "width": 108.9, + "height": 13.75, + "angle": 0, + "strokeColor": "#555555", + "backgroundColor": "transparent", + "fillStyle": "solid", + "strokeWidth": 1, + "strokeStyle": "solid", + "roughness": 1, + "opacity": 100, + "groupIds": [], + "frameId": null, + "roundness": null, + "seed": 947379666, + "versionNonce": 731478845, + "isDeleted": false, + "boundElements": [], + "updated": 1, + "link": null, + "locked": false, + "text": "67108864 (64 MiB)", + "fontSize": 11, + "fontFamily": 1, + "textAlign": "left", + "verticalAlign": "top", + "containerId": null, + "originalText": "67108864 (64 MiB)", + "lineHeight": 1.25, + "baseline": 11 + }, + { + "id": "r1424913390", + "type": "rectangle", + "x": 340, + "y": 210, + "width": 270, + "height": 55, + "angle": 0, + "strokeColor": "#e8870c", + "backgroundColor": "#fff8ef", + "fillStyle": "solid", + "strokeWidth": 2, + "strokeStyle": "solid", + "roughness": 1, + "opacity": 100, + "groupIds": [], + "frameId": null, + "roundness": { + "type": 3 + }, + "seed": 1599087623, + "versionNonce": 1737007508, + "isDeleted": false, + "boundElements": [], + "updated": 1, + "link": null, + "locked": false + }, + { + "id": "t1458250897", + "type": "text", + "x": 350, + "y": 218, + "width": 115.50000000000001, + "height": 17.5, + "angle": 0, + "strokeColor": "#111111", + "backgroundColor": "transparent", + "fillStyle": "solid", + "strokeWidth": 1, + "strokeStyle": "solid", + "roughness": 1, + "opacity": 100, + "groupIds": [], + "frameId": null, + "roundness": null, + "seed": 1748342476, + "versionNonce": 343991084, + "isDeleted": false, + "boundElements": [], + "updated": 1, + "link": null, + "locked": false, + "text": "memory.swap.max", + "fontSize": 14, + "fontFamily": 1, + "textAlign": "left", + "verticalAlign": "top", + "containerId": null, + "originalText": "memory.swap.max", + "lineHeight": 1.25, + "baseline": 14 + }, + { + "id": "t1024238603", + "type": "text", + "x": 350, + "y": 238, + "width": 127.05000000000001, + "height": 13.75, + "angle": 0, + "strokeColor": "#555555", + "backgroundColor": "transparent", + "fillStyle": "solid", + "strokeWidth": 1, + "strokeStyle": "solid", + "roughness": 1, + "opacity": 100, + "groupIds": [], + "frameId": null, + "roundness": null, + "seed": 771440653, + "versionNonce": 1264581232, + "isDeleted": false, + "boundElements": [], + "updated": 1, + "link": null, + "locked": false, + "text": "0 (no swap headroom)", + "fontSize": 11, + "fontFamily": 1, + "textAlign": "left", + "verticalAlign": "top", + "containerId": null, + "originalText": "0 (no swap headroom)", + "lineHeight": 1.25, + "baseline": 11 + }, + { + "id": "r420651538", + "type": "rectangle", + "x": 630, + "y": 210, + "width": 300, + "height": 55, + "angle": 0, + "strokeColor": "#111111", + "backgroundColor": "#ffffff", + "fillStyle": "solid", + "strokeWidth": 2, + "strokeStyle": "solid", + "roughness": 1, + "opacity": 100, + "groupIds": [], + "frameId": null, + "roundness": { + "type": 3 + }, + "seed": 1290409603, + "versionNonce": 709873099, + "isDeleted": false, + "boundElements": [], + "updated": 1, + "link": null, + "locked": false + }, + { + "id": "t138452386", + "type": "text", + "x": 640, + "y": 218, + "width": 53.900000000000006, + "height": 17.5, + "angle": 0, + "strokeColor": "#111111", + "backgroundColor": "transparent", + "fillStyle": "solid", + "strokeWidth": 1, + "strokeStyle": "solid", + "roughness": 1, + "opacity": 100, + "groupIds": [], + "frameId": null, + "roundness": null, + "seed": 813108606, + "versionNonce": 1814267722, + "isDeleted": false, + "boundElements": [], + "updated": 1, + "link": null, + "locked": false, + "text": "cpu.max", + "fontSize": 14, + "fontFamily": 1, + "textAlign": "left", + "verticalAlign": "top", + "containerId": null, + "originalText": "cpu.max", + "lineHeight": 1.25, + "baseline": 14 + }, + { + "id": "t436181847", + "type": "text", + "x": 640, + "y": 238, + "width": 163.35000000000002, + "height": 13.75, + "angle": 0, + "strokeColor": "#555555", + "backgroundColor": "transparent", + "fillStyle": "solid", + "strokeWidth": 1, + "strokeStyle": "solid", + "roughness": 1, + "opacity": 100, + "groupIds": [], + "frameId": null, + "roundness": null, + "seed": 163086111, + "versionNonce": 543982720, + "isDeleted": false, + "boundElements": [], + "updated": 1, + "link": null, + "locked": false, + "text": "\"max 100000\" (unthrottled)", + "fontSize": 11, + "fontFamily": 1, + "textAlign": "left", + "verticalAlign": "top", + "containerId": null, + "originalText": "\"max 100000\" (unthrottled)", + "lineHeight": 1.25, + "baseline": 11 + }, + { + "id": "r1762277900", + "type": "rectangle", + "x": 50, + "y": 290, + "width": 270, + "height": 55, + "angle": 0, + "strokeColor": "#111111", + "backgroundColor": "#ffffff", + "fillStyle": "solid", + "strokeWidth": 2, + "strokeStyle": "solid", + "roughness": 1, + "opacity": 100, + "groupIds": [], + "frameId": null, + "roundness": { + "type": 3 + }, + "seed": 13416031, + "versionNonce": 1349268557, + "isDeleted": false, + "boundElements": [], + "updated": 1, + "link": null, + "locked": false + }, + { + "id": "t873198221", + "type": "text", + "x": 60, + "y": 298, + "width": 92.4, + "height": 17.5, + "angle": 0, + "strokeColor": "#111111", + "backgroundColor": "transparent", + "fillStyle": "solid", + "strokeWidth": 1, + "strokeStyle": "solid", + "roughness": 1, + "opacity": 100, + "groupIds": [], + "frameId": null, + "roundness": null, + "seed": 219440529, + "versionNonce": 1320833367, + "isDeleted": false, + "boundElements": [], + "updated": 1, + "link": null, + "locked": false, + "text": "cgroup.procs", + "fontSize": 14, + "fontFamily": 1, + "textAlign": "left", + "verticalAlign": "top", + "containerId": null, + "originalText": "cgroup.procs", + "lineHeight": 1.25, + "baseline": 14 + }, + { + "id": "t475716911", + "type": "text", + "x": 60, + "y": 318, + "width": 211.75000000000003, + "height": 13.75, + "angle": 0, + "strokeColor": "#555555", + "backgroundColor": "transparent", + "fillStyle": "solid", + "strokeWidth": 1, + "strokeStyle": "solid", + "roughness": 1, + "opacity": 100, + "groupIds": [], + "frameId": null, + "roundness": null, + "seed": 1075784158, + "versionNonce": 789633433, + "isDeleted": false, + "boundElements": [], + "updated": 1, + "link": null, + "locked": false, + "text": "pmon's own pid, written before fork", + "fontSize": 11, + "fontFamily": 1, + "textAlign": "left", + "verticalAlign": "top", + "containerId": null, + "originalText": "pmon's own pid, written before fork", + "lineHeight": 1.25, + "baseline": 11 + }, + { + "id": "r1704587343", + "type": "rectangle", + "x": 340, + "y": 290, + "width": 590, + "height": 55, + "angle": 0, + "strokeColor": "#111111", + "backgroundColor": "#ffffff", + "fillStyle": "solid", + "strokeWidth": 2, + "strokeStyle": "solid", + "roughness": 1, + "opacity": 100, + "groupIds": [], + "frameId": null, + "roundness": { + "type": 3 + }, + "seed": 125692972, + "versionNonce": 1190871709, + "isDeleted": false, + "boundElements": [], + "updated": 1, + "link": null, + "locked": false + }, + { + "id": "t1202240986", + "type": "text", + "x": 350, + "y": 298, + "width": 161.70000000000002, + "height": 17.5, + "angle": 0, + "strokeColor": "#111111", + "backgroundColor": "transparent", + "fillStyle": "solid", + "strokeWidth": 1, + "strokeStyle": "solid", + "roughness": 1, + "opacity": 100, + "groupIds": [], + "frameId": null, + "roundness": null, + "seed": 200540662, + "versionNonce": 225678365, + "isDeleted": false, + "boundElements": [], + "updated": 1, + "link": null, + "locked": false, + "text": "memory.pressure (PSI)", + "fontSize": 14, + "fontFamily": 1, + "textAlign": "left", + "verticalAlign": "top", + "containerId": null, + "originalText": "memory.pressure (PSI)", + "lineHeight": 1.25, + "baseline": 14 + }, + { + "id": "t1311660598", + "type": "text", + "x": 350, + "y": 318, + "width": 278.3, + "height": 13.75, + "angle": 0, + "strokeColor": "#555555", + "backgroundColor": "transparent", + "fillStyle": "solid", + "strokeWidth": 1, + "strokeStyle": "solid", + "roughness": 1, + "opacity": 100, + "groupIds": [], + "frameId": null, + "roundness": null, + "seed": 309142707, + "versionNonce": 1000248213, + "isDeleted": false, + "boundElements": [], + "updated": 1, + "link": null, + "locked": false, + "text": "some avg10=0.00 avg60=0.00 avg300=0.00 total=0", + "fontSize": 11, + "fontFamily": 1, + "textAlign": "left", + "verticalAlign": "top", + "containerId": null, + "originalText": "some avg10=0.00 avg60=0.00 avg300=0.00 total=0", + "lineHeight": 1.25, + "baseline": 11 + }, + { + "id": "r591857662", + "type": "rectangle", + "x": 50, + "y": 365, + "width": 880, + "height": 45, + "angle": 0, + "strokeColor": "#111111", + "backgroundColor": "#111111", + "fillStyle": "solid", + "strokeWidth": 2, + "strokeStyle": "solid", + "roughness": 1, + "opacity": 100, + "groupIds": [], + "frameId": null, + "roundness": { + "type": 3 + }, + "seed": 1318986971, + "versionNonce": 1721452947, + "isDeleted": false, + "boundElements": [], + "updated": 1, + "link": null, + "locked": false + }, + { + "id": "t1994304852", + "type": "text", + "x": 60, + "y": 373, + "width": 677.6, + "height": 17.5, + "angle": 0, + "strokeColor": "#ffffff", + "backgroundColor": "transparent", + "fillStyle": "solid", + "strokeWidth": 1, + "strokeStyle": "solid", + "roughness": 1, + "opacity": 100, + "groupIds": [], + "frameId": null, + "roundness": null, + "seed": 1251274441, + "versionNonce": 76062209, + "isDeleted": false, + "boundElements": [], + "updated": 1, + "link": null, + "locked": false, + "text": "hog (sh, real anon-rss 64384kB) writes past 64 MiB \u2192 allocation stalls, not merely slows", + "fontSize": 14, + "fontFamily": 1, + "textAlign": "left", + "verticalAlign": "top", + "containerId": null, + "originalText": "hog (sh, real anon-rss 64384kB) writes past 64 MiB \u2192 allocation stalls, not merely slows", + "lineHeight": 1.25, + "baseline": 14 + }, + { + "id": "r314314430", + "type": "rectangle", + "x": 50, + "y": 495, + "width": 880, + "height": 45, + "angle": 0, + "strokeColor": "#111111", + "backgroundColor": "#ffffff", + "fillStyle": "solid", + "strokeWidth": 2, + "strokeStyle": "solid", + "roughness": 1, + "opacity": 100, + "groupIds": [], + "frameId": null, + "roundness": { + "type": 3 + }, + "seed": 1892384278, + "versionNonce": 1416622876, + "isDeleted": false, + "boundElements": [], + "updated": 1, + "link": null, + "locked": false + }, + { + "id": "t719403984", + "type": "text", + "x": 60, + "y": 503, + "width": 800.8000000000001, + "height": 17.5, + "angle": 0, + "strokeColor": "#111111", + "backgroundColor": "transparent", + "fillStyle": "solid", + "strokeWidth": 1, + "strokeStyle": "solid", + "roughness": 1, + "opacity": 100, + "groupIds": [], + "frameId": null, + "roundness": null, + "seed": 1918070034, + "versionNonce": 1526928596, + "isDeleted": false, + "boundElements": [], + "updated": 1, + "link": null, + "locked": false, + "text": "oom-kill: constraint=CONSTRAINT_MEMCG oom_memcg=/pmon-oomdemo task_memcg=/pmon-oomdemo task=sh pid=27013", + "fontSize": 14, + "fontFamily": 1, + "textAlign": "left", + "verticalAlign": "top", + "containerId": null, + "originalText": "oom-kill: constraint=CONSTRAINT_MEMCG oom_memcg=/pmon-oomdemo task_memcg=/pmon-oomdemo task=sh pid=27013", + "lineHeight": 1.25, + "baseline": 14 + }, + { + "id": "r1162187097", + "type": "rectangle", + "x": 50, + "y": 550, + "width": 880, + "height": 45, + "angle": 0, + "strokeColor": "#111111", + "backgroundColor": "#ffffff", + "fillStyle": "solid", + "strokeWidth": 2, + "strokeStyle": "solid", + "roughness": 1, + "opacity": 100, + "groupIds": [], + "frameId": null, + "roundness": { + "type": 3 + }, + "seed": 1092149834, + "versionNonce": 462599393, + "isDeleted": false, + "boundElements": [], + "updated": 1, + "link": null, + "locked": false + }, + { + "id": "t1097101957", + "type": "text", + "x": 60, + "y": 558, + "width": 885.5000000000001, + "height": 17.5, + "angle": 0, + "strokeColor": "#111111", + "backgroundColor": "transparent", + "fillStyle": "solid", + "strokeWidth": 1, + "strokeStyle": "solid", + "roughness": 1, + "opacity": 100, + "groupIds": [], + "frameId": null, + "roundness": null, + "seed": 1748624501, + "versionNonce": 246589393, + "isDeleted": false, + "boundElements": [], + "updated": 1, + "link": null, + "locked": false, + "text": "Memory cgroup out of memory: Killed process 27013 (sh) anon-rss:64384kB \u2192 pmon reports: signal=9 (KILL), exit 137", + "fontSize": 14, + "fontFamily": 1, + "textAlign": "left", + "verticalAlign": "top", + "containerId": null, + "originalText": "Memory cgroup out of memory: Killed process 27013 (sh) anon-rss:64384kB \u2192 pmon reports: signal=9 (KILL), exit 137", + "lineHeight": 1.25, + "baseline": 14 + }, + { + "id": "a871506372", + "type": "arrow", + "x": 470, + "y": 90, + "width": 30, + "height": 0, + "angle": 0, + "strokeColor": "#b8650a", + "backgroundColor": "transparent", + "fillStyle": "solid", + "strokeWidth": 2, + "strokeStyle": "solid", + "roughness": 1, + "opacity": 100, + "groupIds": [], + "frameId": null, + "roundness": { + "type": 2 + }, + "seed": 668447526, + "versionNonce": 1565925272, + "isDeleted": false, + "boundElements": [], + "updated": 1, + "link": null, + "locked": false, + "points": [ + [ + 0, + 0 + ], + [ + 30, + 0 + ] + ], + "lastCommittedPoint": null, + "startBinding": null, + "endBinding": null, + "startArrowhead": null, + "endArrowhead": "arrow" + }, + { + "id": "t1226915396", + "type": "text", + "x": 485.0, + "y": 76.0, + "width": 42.35, + "height": 13.75, + "angle": 0, + "strokeColor": "#b8650a", + "backgroundColor": "transparent", + "fillStyle": "solid", + "strokeWidth": 1, + "strokeStyle": "solid", + "roughness": 1, + "opacity": 100, + "groupIds": [], + "frameId": null, + "roundness": null, + "seed": 146835258, + "versionNonce": 1998492447, + "isDeleted": false, + "boundElements": [], + "updated": 1, + "link": null, + "locked": false, + "text": "enables", + "fontSize": 11, + "fontFamily": 1, + "textAlign": "left", + "verticalAlign": "top", + "containerId": null, + "originalText": "enables", + "lineHeight": 1.25, + "baseline": 11 + }, + { + "id": "a273566582", + "type": "arrow", + "x": 265, + "y": 130, + "width": 80, + "height": 80, + "angle": 0, + "strokeColor": "#b8650a", + "backgroundColor": "transparent", + "fillStyle": "solid", + "strokeWidth": 2, + "strokeStyle": "solid", + "roughness": 1, + "opacity": 100, + "groupIds": [], + "frameId": null, + "roundness": { + "type": 2 + }, + "seed": 1349045877, + "versionNonce": 1708227481, + "isDeleted": false, + "boundElements": [], + "updated": 1, + "link": null, + "locked": false, + "points": [ + [ + 0, + 0 + ], + [ + -80, + 80 + ] + ], + "lastCommittedPoint": null, + "startBinding": null, + "endBinding": null, + "startArrowhead": null, + "endArrowhead": "arrow" + }, + { + "id": "a1232491265", + "type": "arrow", + "x": 730, + "y": 130, + "width": 50, + "height": 80, + "angle": 0, + "strokeColor": "#555555", + "backgroundColor": "transparent", + "fillStyle": "solid", + "strokeWidth": 2, + "strokeStyle": "solid", + "roughness": 1, + "opacity": 100, + "groupIds": [], + "frameId": null, + "roundness": { + "type": 2 + }, + "seed": 1869176948, + "versionNonce": 1889945078, + "isDeleted": false, + "boundElements": [], + "updated": 1, + "link": null, + "locked": false, + "points": [ + [ + 0, + 0 + ], + [ + 50, + 80 + ] + ], + "lastCommittedPoint": null, + "startBinding": null, + "endBinding": null, + "startArrowhead": null, + "endArrowhead": "arrow" + }, + { + "id": "a1108672718", + "type": "arrow", + "x": 490, + "y": 410, + "width": 0, + "height": 40, + "angle": 0, + "strokeColor": "#b8650a", + "backgroundColor": "transparent", + "fillStyle": "solid", + "strokeWidth": 2, + "strokeStyle": "solid", + "roughness": 1, + "opacity": 100, + "groupIds": [], + "frameId": null, + "roundness": { + "type": 2 + }, + "seed": 209378072, + "versionNonce": 1539549103, + "isDeleted": false, + "boundElements": [], + "updated": 1, + "link": null, + "locked": false, + "points": [ + [ + 0, + 0 + ], + [ + 0, + 40 + ] + ], + "lastCommittedPoint": null, + "startBinding": null, + "endBinding": null, + "startArrowhead": null, + "endArrowhead": "arrow" + }, + { + "id": "t1620890146", + "type": "text", + "x": 490.0, + "y": 416.0, + "width": 96.80000000000001, + "height": 13.75, + "angle": 0, + "strokeColor": "#b8650a", + "backgroundColor": "transparent", + "fillStyle": "solid", + "strokeWidth": 1, + "strokeStyle": "solid", + "roughness": 1, + "opacity": 100, + "groupIds": [], + "frameId": null, + "roundness": null, + "seed": 797894383, + "versionNonce": 604621249, + "isDeleted": false, + "boundElements": [], + "updated": 1, + "link": null, + "locked": false, + "text": "OOM killer fires", + "fontSize": 11, + "fontFamily": 1, + "textAlign": "left", + "verticalAlign": "top", + "containerId": null, + "originalText": "OOM killer fires", + "lineHeight": 1.25, + "baseline": 11 + }, + { + "id": "t801637876", + "type": "text", + "x": 30, + "y": 605, + "width": 897.6, + "height": 15.0, + "angle": 0, + "strokeColor": "#555555", + "backgroundColor": "transparent", + "fillStyle": "solid", + "strokeWidth": 1, + "strokeStyle": "solid", + "roughness": 1, + "opacity": 100, + "groupIds": [], + "frameId": null, + "roundness": null, + "seed": 44381408, + "versionNonce": 1009420273, + "isDeleted": false, + "boundElements": [], + "updated": 1, + "link": null, + "locked": false, + "text": "Every value above was read live from systems-target (kernel 6.19.10-300.fc44) during this session's containerize --mem-max 67108864 run.", + "fontSize": 12, + "fontFamily": 1, + "textAlign": "left", + "verticalAlign": "top", + "containerId": null, + "originalText": "Every value above was read live from systems-target (kernel 6.19.10-300.fc44) during this session's containerize --mem-max 67108864 run.", + "lineHeight": 1.25, + "baseline": 12 + } + ], + "appState": { + "viewBackgroundColor": "#ffffff", + "gridSize": null + }, + "files": {} +} \ No newline at end of file diff --git a/assets/diagrams/32-cgroup-hierarchy.svg b/assets/diagrams/32-cgroup-hierarchy.svg new file mode 100644 index 0000000..da9e2e6 --- /dev/null +++ b/assets/diagrams/32-cgroup-hierarchy.svg @@ -0,0 +1,45 @@ + + + +/sys/fs/cgroup (root — exempt from the "no internal process" rule) + +/sys/fs/cgroup/pmon-oomdemo (leaf, created by setup_cgroup()) + +kernel — real dmesg from this session's OOM run + +cgroup.controllers +cpuset cpu io memory hugetlb pids rdma misc dmem + +cgroup.subtree_control ++memory +cpu (delegated to children) + +memory.max +67108864 (64 MiB) + +memory.swap.max +0 (no swap headroom) + +cpu.max +"max 100000" (unthrottled) + +cgroup.procs +pmon's own pid, written before fork + +memory.pressure (PSI) +some avg10=0.00 avg60=0.00 avg300=0.00 total=0 + +hog (sh, real anon-rss 64384kB) writes past 64 MiB → allocation stalls, not merely slows + +oom-kill: constraint=CONSTRAINT_MEMCG oom_memcg=/pmon-oomdemo task_memcg=/pmon-oomdemo task=sh pid=27013 + +Memory cgroup out of memory: Killed process 27013 (sh) anon-rss:64384kB → pmon reports: signal=9 (KILL), exit 137 + +enables +enables + + + +OOM killer fires +OOM killer fires +Every value above was read live from systems-target (kernel 6.19.10-300.fc44) during this session's containerize --mem-max 67108864 run. + \ No newline at end of file diff --git a/assets/diagrams/32-clone3-flow.excalidraw b/assets/diagrams/32-clone3-flow.excalidraw new file mode 100644 index 0000000..161e0b4 --- /dev/null +++ b/assets/diagrams/32-clone3-flow.excalidraw @@ -0,0 +1,1552 @@ +{ + "type": "excalidraw", + "version": 2, + "source": "https://excalidraw.com", + "elements": [ + { + "id": "r453976816", + "type": "rectangle", + "x": 20, + "y": 20, + "width": 460, + "height": 250, + "angle": 0, + "strokeColor": "#dddddd", + "backgroundColor": "#eef4fb", + "fillStyle": "solid", + "strokeWidth": 2, + "strokeStyle": "solid", + "roughness": 1, + "opacity": 100, + "groupIds": [], + "frameId": null, + "roundness": { + "type": 3 + }, + "seed": 1414434839, + "versionNonce": 1424253873, + "isDeleted": false, + "boundElements": [], + "updated": 1, + "link": null, + "locked": false + }, + { + "id": "t1871561627", + "type": "text", + "x": 32, + "y": 26, + "width": 323.40000000000003, + "height": 15.0, + "angle": 0, + "strokeColor": "#555555", + "backgroundColor": "transparent", + "fillStyle": "solid", + "strokeWidth": 1, + "strokeStyle": "solid", + "roughness": 1, + "opacity": 100, + "groupIds": [], + "frameId": null, + "roundness": null, + "seed": 1115614384, + "versionNonce": 945316701, + "isDeleted": false, + "boundElements": [], + "updated": 1, + "link": null, + "locked": false, + "text": "C++ / Rust \u2014 unshare(2) then clone3/fork(flags=0)", + "fontSize": 12, + "fontFamily": 1, + "textAlign": "left", + "verticalAlign": "top", + "containerId": null, + "originalText": "C++ / Rust \u2014 unshare(2) then clone3/fork(flags=0)", + "lineHeight": 1.25, + "baseline": 12 + }, + { + "id": "r1521921204", + "type": "rectangle", + "x": 500, + "y": 20, + "width": 460, + "height": 250, + "angle": 0, + "strokeColor": "#dddddd", + "backgroundColor": "#fdeaea", + "fillStyle": "solid", + "strokeWidth": 2, + "strokeStyle": "solid", + "roughness": 1, + "opacity": 100, + "groupIds": [], + "frameId": null, + "roundness": { + "type": 3 + }, + "seed": 650886683, + "versionNonce": 1660980810, + "isDeleted": false, + "boundElements": [], + "updated": 1, + "link": null, + "locked": false + }, + { + "id": "t1508034820", + "type": "text", + "x": 512, + "y": 26, + "width": 363.00000000000006, + "height": 15.0, + "angle": 0, + "strokeColor": "#555555", + "backgroundColor": "transparent", + "fillStyle": "solid", + "strokeWidth": 1, + "strokeStyle": "solid", + "roughness": 1, + "opacity": 100, + "groupIds": [], + "frameId": null, + "roundness": null, + "seed": 1020951252, + "versionNonce": 1617664400, + "isDeleted": false, + "boundElements": [], + "updated": 1, + "link": null, + "locked": false, + "text": "Go, first attempt \u2014 unshare(2) then re-exec via os/exec", + "fontSize": 12, + "fontFamily": 1, + "textAlign": "left", + "verticalAlign": "top", + "containerId": null, + "originalText": "Go, first attempt \u2014 unshare(2) then re-exec via os/exec", + "lineHeight": 1.25, + "baseline": 12 + }, + { + "id": "r1598975596", + "type": "rectangle", + "x": 20, + "y": 290, + "width": 940, + "height": 250, + "angle": 0, + "strokeColor": "#dddddd", + "backgroundColor": "#fff8ef", + "fillStyle": "solid", + "strokeWidth": 2, + "strokeStyle": "solid", + "roughness": 1, + "opacity": 100, + "groupIds": [], + "frameId": null, + "roundness": { + "type": 3 + }, + "seed": 515249322, + "versionNonce": 972848427, + "isDeleted": false, + "boundElements": [], + "updated": 1, + "link": null, + "locked": false + }, + { + "id": "t376799899", + "type": "text", + "x": 32, + "y": 296, + "width": 376.20000000000005, + "height": 15.0, + "angle": 0, + "strokeColor": "#555555", + "backgroundColor": "transparent", + "fillStyle": "solid", + "strokeWidth": 1, + "strokeStyle": "solid", + "roughness": 1, + "opacity": 100, + "groupIds": [], + "frameId": null, + "roundness": null, + "seed": 498159516, + "versionNonce": 1708348160, + "isDeleted": false, + "boundElements": [], + "updated": 1, + "link": null, + "locked": false, + "text": "Go, shipped fix \u2014 Cloneflags on the re-exec's SysProcAttr", + "fontSize": 12, + "fontFamily": 1, + "textAlign": "left", + "verticalAlign": "top", + "containerId": null, + "originalText": "Go, shipped fix \u2014 Cloneflags on the re-exec's SysProcAttr", + "lineHeight": 1.25, + "baseline": 12 + }, + { + "id": "r102299349", + "type": "rectangle", + "x": 40, + "y": 60, + "width": 420, + "height": 45, + "angle": 0, + "strokeColor": "#111111", + "backgroundColor": "#ffffff", + "fillStyle": "solid", + "strokeWidth": 2, + "strokeStyle": "solid", + "roughness": 1, + "opacity": 100, + "groupIds": [], + "frameId": null, + "roundness": { + "type": 3 + }, + "seed": 174949813, + "versionNonce": 1221497657, + "isDeleted": false, + "boundElements": [], + "updated": 1, + "link": null, + "locked": false + }, + { + "id": "t561218553", + "type": "text", + "x": 50, + "y": 68, + "width": 269.5, + "height": 17.5, + "angle": 0, + "strokeColor": "#111111", + "backgroundColor": "transparent", + "fillStyle": "solid", + "strokeWidth": 1, + "strokeStyle": "solid", + "roughness": 1, + "opacity": 100, + "groupIds": [], + "frameId": null, + "roundness": null, + "seed": 388678803, + "versionNonce": 200439731, + "isDeleted": false, + "boundElements": [], + "updated": 1, + "link": null, + "locked": false, + "text": "unshare(NEWNS|NEWUTS|NEWNET|NEWPID)", + "fontSize": 14, + "fontFamily": 1, + "textAlign": "left", + "verticalAlign": "top", + "containerId": null, + "originalText": "unshare(NEWNS|NEWUTS|NEWNET|NEWPID)", + "lineHeight": 1.25, + "baseline": 14 + }, + { + "id": "t786810475", + "type": "text", + "x": 50, + "y": 88, + "width": 181.50000000000003, + "height": 13.75, + "angle": 0, + "strokeColor": "#555555", + "backgroundColor": "transparent", + "fillStyle": "solid", + "strokeWidth": 1, + "strokeStyle": "solid", + "roughness": 1, + "opacity": 100, + "groupIds": [], + "frameId": null, + "roundness": null, + "seed": 375563176, + "versionNonce": 939332085, + "isDeleted": false, + "boundElements": [], + "updated": 1, + "link": null, + "locked": false, + "text": "takes effect on the caller now", + "fontSize": 11, + "fontFamily": 1, + "textAlign": "left", + "verticalAlign": "top", + "containerId": null, + "originalText": "takes effect on the caller now", + "lineHeight": 1.25, + "baseline": 11 + }, + { + "id": "r1689459136", + "type": "rectangle", + "x": 40, + "y": 120, + "width": 420, + "height": 45, + "angle": 0, + "strokeColor": "#111111", + "backgroundColor": "#ffffff", + "fillStyle": "solid", + "strokeWidth": 2, + "strokeStyle": "solid", + "roughness": 1, + "opacity": 100, + "groupIds": [], + "frameId": null, + "roundness": { + "type": 3 + }, + "seed": 1495317021, + "versionNonce": 1799174660, + "isDeleted": false, + "boundElements": [], + "updated": 1, + "link": null, + "locked": false + }, + { + "id": "t967342087", + "type": "text", + "x": 50, + "y": 128, + "width": 184.8, + "height": 17.5, + "angle": 0, + "strokeColor": "#111111", + "backgroundColor": "transparent", + "fillStyle": "solid", + "strokeWidth": 1, + "strokeStyle": "solid", + "roughness": 1, + "opacity": 100, + "groupIds": [], + "frameId": null, + "roundness": null, + "seed": 1464367938, + "versionNonce": 1872405335, + "isDeleted": false, + "boundElements": [], + "updated": 1, + "link": null, + "locked": false, + "text": "clone3(flags=0, stack=0)", + "fontSize": 14, + "fontFamily": 1, + "textAlign": "left", + "verticalAlign": "top", + "containerId": null, + "originalText": "clone3(flags=0, stack=0)", + "lineHeight": 1.25, + "baseline": 14 + }, + { + "id": "t852784186", + "type": "text", + "x": 50, + "y": 148, + "width": 181.50000000000003, + "height": 13.75, + "angle": 0, + "strokeColor": "#555555", + "backgroundColor": "transparent", + "fillStyle": "solid", + "strokeWidth": 1, + "strokeStyle": "solid", + "roughness": 1, + "opacity": 100, + "groupIds": [], + "frameId": null, + "roundness": null, + "seed": 1467973869, + "versionNonce": 1344701661, + "isDeleted": false, + "boundElements": [], + "updated": 1, + "link": null, + "locked": false, + "text": "= fork(2): child becomes PID 1", + "fontSize": 11, + "fontFamily": 1, + "textAlign": "left", + "verticalAlign": "top", + "containerId": null, + "originalText": "= fork(2): child becomes PID 1", + "lineHeight": 1.25, + "baseline": 11 + }, + { + "id": "r517115927", + "type": "rectangle", + "x": 40, + "y": 180, + "width": 420, + "height": 55, + "angle": 0, + "strokeColor": "#e8870c", + "backgroundColor": "#fff8ef", + "fillStyle": "solid", + "strokeWidth": 2, + "strokeStyle": "solid", + "roughness": 1, + "opacity": 100, + "groupIds": [], + "frameId": null, + "roundness": { + "type": 3 + }, + "seed": 751326856, + "versionNonce": 606789096, + "isDeleted": false, + "boundElements": [], + "updated": 1, + "link": null, + "locked": false + }, + { + "id": "t399672096", + "type": "text", + "x": 50, + "y": 188, + "width": 292.6, + "height": 17.5, + "angle": 0, + "strokeColor": "#111111", + "backgroundColor": "transparent", + "fillStyle": "solid", + "strokeWidth": 1, + "strokeStyle": "solid", + "roughness": 1, + "opacity": 100, + "groupIds": [], + "frameId": null, + "roundness": null, + "seed": 1031469487, + "versionNonce": 277291480, + "isDeleted": false, + "boundElements": [], + "updated": 1, + "link": null, + "locked": false, + "text": "child: print pid/hostname, execvp(CMD)", + "fontSize": 14, + "fontFamily": 1, + "textAlign": "left", + "verticalAlign": "top", + "containerId": null, + "originalText": "child: print pid/hostname, execvp(CMD)", + "lineHeight": 1.25, + "baseline": 14 + }, + { + "id": "t703472936", + "type": "text", + "x": 50, + "y": 208, + "width": 175.45000000000002, + "height": 13.75, + "angle": 0, + "strokeColor": "#555555", + "backgroundColor": "transparent", + "fillStyle": "solid", + "strokeWidth": 1, + "strokeStyle": "solid", + "roughness": 1, + "opacity": 100, + "groupIds": [], + "frameId": null, + "roundness": null, + "seed": 351970335, + "versionNonce": 849513379, + "isDeleted": false, + "boundElements": [], + "updated": 1, + "link": null, + "locked": false, + "text": "PASS \u2014 real run, this session", + "fontSize": 11, + "fontFamily": 1, + "textAlign": "left", + "verticalAlign": "top", + "containerId": null, + "originalText": "PASS \u2014 real run, this session", + "lineHeight": 1.25, + "baseline": 11 + }, + { + "id": "r1101545239", + "type": "rectangle", + "x": 520, + "y": 60, + "width": 420, + "height": 45, + "angle": 0, + "strokeColor": "#111111", + "backgroundColor": "#ffffff", + "fillStyle": "solid", + "strokeWidth": 2, + "strokeStyle": "solid", + "roughness": 1, + "opacity": 100, + "groupIds": [], + "frameId": null, + "roundness": { + "type": 3 + }, + "seed": 559191448, + "versionNonce": 104309124, + "isDeleted": false, + "boundElements": [], + "updated": 1, + "link": null, + "locked": false + }, + { + "id": "t1514638559", + "type": "text", + "x": 530, + "y": 68, + "width": 277.20000000000005, + "height": 17.5, + "angle": 0, + "strokeColor": "#111111", + "backgroundColor": "transparent", + "fillStyle": "solid", + "strokeWidth": 1, + "strokeStyle": "solid", + "roughness": 1, + "opacity": 100, + "groupIds": [], + "frameId": null, + "roundness": null, + "seed": 1012088787, + "versionNonce": 126510688, + "isDeleted": false, + "boundElements": [], + "updated": 1, + "link": null, + "locked": false, + "text": "runtime.LockOSThread(); unshare(...)", + "fontSize": 14, + "fontFamily": 1, + "textAlign": "left", + "verticalAlign": "top", + "containerId": null, + "originalText": "runtime.LockOSThread(); unshare(...)", + "lineHeight": 1.25, + "baseline": 14 + }, + { + "id": "t806664213", + "type": "text", + "x": 530, + "y": 88, + "width": 175.45000000000002, + "height": 13.75, + "angle": 0, + "strokeColor": "#555555", + "backgroundColor": "transparent", + "fillStyle": "solid", + "strokeWidth": 1, + "strokeStyle": "solid", + "roughness": 1, + "opacity": 100, + "groupIds": [], + "frameId": null, + "roundness": null, + "seed": 1821916070, + "versionNonce": 807042078, + "isDeleted": false, + "boundElements": [], + "updated": 1, + "link": null, + "locked": false, + "text": "pins goroutine, then unshares", + "fontSize": 11, + "fontFamily": 1, + "textAlign": "left", + "verticalAlign": "top", + "containerId": null, + "originalText": "pins goroutine, then unshares", + "lineHeight": 1.25, + "baseline": 11 + }, + { + "id": "r787420600", + "type": "rectangle", + "x": 520, + "y": 120, + "width": 420, + "height": 45, + "angle": 0, + "strokeColor": "#111111", + "backgroundColor": "#ffffff", + "fillStyle": "solid", + "strokeWidth": 2, + "strokeStyle": "solid", + "roughness": 1, + "opacity": 100, + "groupIds": [], + "frameId": null, + "roundness": { + "type": 3 + }, + "seed": 767369652, + "versionNonce": 393760949, + "isDeleted": false, + "boundElements": [], + "updated": 1, + "link": null, + "locked": false + }, + { + "id": "t1951784294", + "type": "text", + "x": 530, + "y": 128, + "width": 223.3, + "height": 17.5, + "angle": 0, + "strokeColor": "#111111", + "backgroundColor": "transparent", + "fillStyle": "solid", + "strokeWidth": 1, + "strokeStyle": "solid", + "roughness": 1, + "opacity": 100, + "groupIds": [], + "frameId": null, + "roundness": null, + "seed": 291297184, + "versionNonce": 1720462606, + "isDeleted": false, + "boundElements": [], + "updated": 1, + "link": null, + "locked": false, + "text": "os/exec.Start() re-execs self", + "fontSize": 14, + "fontFamily": 1, + "textAlign": "left", + "verticalAlign": "top", + "containerId": null, + "originalText": "os/exec.Start() re-execs self", + "lineHeight": 1.25, + "baseline": 14 + }, + { + "id": "t59672442", + "type": "text", + "x": 530, + "y": 148, + "width": 223.85000000000002, + "height": 13.75, + "angle": 0, + "strokeColor": "#555555", + "backgroundColor": "transparent", + "fillStyle": "solid", + "strokeWidth": 1, + "strokeStyle": "solid", + "roughness": 1, + "opacity": 100, + "groupIds": [], + "frameId": null, + "roundness": null, + "seed": 399204827, + "versionNonce": 1233160095, + "isDeleted": false, + "boundElements": [], + "updated": 1, + "link": null, + "locked": false, + "text": "internal vfork/pidfd probe fork first", + "fontSize": 11, + "fontFamily": 1, + "textAlign": "left", + "verticalAlign": "top", + "containerId": null, + "originalText": "internal vfork/pidfd probe fork first", + "lineHeight": 1.25, + "baseline": 11 + }, + { + "id": "r1632707644", + "type": "rectangle", + "x": 520, + "y": 180, + "width": 420, + "height": 55, + "angle": 0, + "strokeColor": "#111111", + "backgroundColor": "#111111", + "fillStyle": "solid", + "strokeWidth": 2, + "strokeStyle": "solid", + "roughness": 1, + "opacity": 100, + "groupIds": [], + "frameId": null, + "roundness": { + "type": 3 + }, + "seed": 1677776105, + "versionNonce": 535699929, + "isDeleted": false, + "boundElements": [], + "updated": 1, + "link": null, + "locked": false + }, + { + "id": "t983823584", + "type": "text", + "x": 530, + "y": 188, + "width": 146.3, + "height": 17.5, + "angle": 0, + "strokeColor": "#ffffff", + "backgroundColor": "transparent", + "fillStyle": "solid", + "strokeWidth": 1, + "strokeStyle": "solid", + "roughness": 1, + "opacity": 100, + "groupIds": [], + "frameId": null, + "roundness": null, + "seed": 560136688, + "versionNonce": 1759426763, + "isDeleted": false, + "boundElements": [], + "updated": 1, + "link": null, + "locked": false, + "text": "clone() = -1 ENOMEM", + "fontSize": 14, + "fontFamily": 1, + "textAlign": "left", + "verticalAlign": "top", + "containerId": null, + "originalText": "clone() = -1 ENOMEM", + "lineHeight": 1.25, + "baseline": 14 + }, + { + "id": "t1877098375", + "type": "text", + "x": 530, + "y": 208, + "width": 338.8, + "height": 13.75, + "angle": 0, + "strokeColor": "#555555", + "backgroundColor": "transparent", + "fillStyle": "solid", + "strokeWidth": 1, + "strokeStyle": "solid", + "roughness": 1, + "opacity": 100, + "groupIds": [], + "frameId": null, + "roundness": null, + "seed": 1080559236, + "versionNonce": 1064999921, + "isDeleted": false, + "boundElements": [], + "updated": 1, + "link": null, + "locked": false, + "text": "\"fork/exec: cannot allocate memory\" \u2014 real, this session", + "fontSize": 11, + "fontFamily": 1, + "textAlign": "left", + "verticalAlign": "top", + "containerId": null, + "originalText": "\"fork/exec: cannot allocate memory\" \u2014 real, this session", + "lineHeight": 1.25, + "baseline": 11 + }, + { + "id": "r1580974413", + "type": "rectangle", + "x": 40, + "y": 335, + "width": 420, + "height": 45, + "angle": 0, + "strokeColor": "#111111", + "backgroundColor": "#ffffff", + "fillStyle": "solid", + "strokeWidth": 2, + "strokeStyle": "solid", + "roughness": 1, + "opacity": 100, + "groupIds": [], + "frameId": null, + "roundness": { + "type": 3 + }, + "seed": 657988076, + "versionNonce": 757795375, + "isDeleted": false, + "boundElements": [], + "updated": 1, + "link": null, + "locked": false + }, + { + "id": "t735638910", + "type": "text", + "x": 50, + "y": 343, + "width": 269.5, + "height": 17.5, + "angle": 0, + "strokeColor": "#111111", + "backgroundColor": "transparent", + "fillStyle": "solid", + "strokeWidth": 1, + "strokeStyle": "solid", + "roughness": 1, + "opacity": 100, + "groupIds": [], + "frameId": null, + "roundness": null, + "seed": 1413950527, + "versionNonce": 1119954600, + "isDeleted": false, + "boundElements": [], + "updated": 1, + "link": null, + "locked": false, + "text": "setup_cgroup(): write own pid first", + "fontSize": 14, + "fontFamily": 1, + "textAlign": "left", + "verticalAlign": "top", + "containerId": null, + "originalText": "setup_cgroup(): write own pid first", + "lineHeight": 1.25, + "baseline": 14 + }, + { + "id": "t114372061", + "type": "text", + "x": 50, + "y": 363, + "width": 211.75000000000003, + "height": 13.75, + "angle": 0, + "strokeColor": "#555555", + "backgroundColor": "transparent", + "fillStyle": "solid", + "strokeWidth": 1, + "strokeStyle": "solid", + "roughness": 1, + "opacity": 100, + "groupIds": [], + "frameId": null, + "roundness": null, + "seed": 1702218978, + "versionNonce": 1363567647, + "isDeleted": false, + "boundElements": [], + "updated": 1, + "link": null, + "locked": false, + "text": "(unchanged \u2014 inherited across fork)", + "fontSize": 11, + "fontFamily": 1, + "textAlign": "left", + "verticalAlign": "top", + "containerId": null, + "originalText": "(unchanged \u2014 inherited across fork)", + "lineHeight": 1.25, + "baseline": 11 + }, + { + "id": "r1613279655", + "type": "rectangle", + "x": 500, + "y": 335, + "width": 440, + "height": 45, + "angle": 0, + "strokeColor": "#e8870c", + "backgroundColor": "#fff8ef", + "fillStyle": "solid", + "strokeWidth": 2, + "strokeStyle": "solid", + "roughness": 1, + "opacity": 100, + "groupIds": [], + "frameId": null, + "roundness": { + "type": 3 + }, + "seed": 1792181671, + "versionNonce": 1055988542, + "isDeleted": false, + "boundElements": [], + "updated": 1, + "link": null, + "locked": false + }, + { + "id": "t729748689", + "type": "text", + "x": 510, + "y": 343, + "width": 300.3, + "height": 17.5, + "angle": 0, + "strokeColor": "#111111", + "backgroundColor": "transparent", + "fillStyle": "solid", + "strokeWidth": 1, + "strokeStyle": "solid", + "roughness": 1, + "opacity": 100, + "groupIds": [], + "frameId": null, + "roundness": null, + "seed": 1509397183, + "versionNonce": 366927065, + "isDeleted": false, + "boundElements": [], + "updated": 1, + "link": null, + "locked": false, + "text": "exec.Command(self, hostname, \"--\", CMD)", + "fontSize": 14, + "fontFamily": 1, + "textAlign": "left", + "verticalAlign": "top", + "containerId": null, + "originalText": "exec.Command(self, hostname, \"--\", CMD)", + "lineHeight": 1.25, + "baseline": 14 + }, + { + "id": "t368795056", + "type": "text", + "x": 510, + "y": 363, + "width": 308.55, + "height": 13.75, + "angle": 0, + "strokeColor": "#555555", + "backgroundColor": "transparent", + "fillStyle": "solid", + "strokeWidth": 1, + "strokeStyle": "solid", + "roughness": 1, + "opacity": 100, + "groupIds": [], + "frameId": null, + "roundness": null, + "seed": 1247520120, + "versionNonce": 336039688, + "isDeleted": false, + "boundElements": [], + "updated": 1, + "link": null, + "locked": false, + "text": "SysProcAttr.Cloneflags = NEWNS|NEWUTS|NEWNET|NEWPID", + "fontSize": 11, + "fontFamily": 1, + "textAlign": "left", + "verticalAlign": "top", + "containerId": null, + "originalText": "SysProcAttr.Cloneflags = NEWNS|NEWUTS|NEWNET|NEWPID", + "lineHeight": 1.25, + "baseline": 11 + }, + { + "id": "r1897388970", + "type": "rectangle", + "x": 40, + "y": 395, + "width": 900, + "height": 55, + "angle": 0, + "strokeColor": "#e8870c", + "backgroundColor": "#fff8ef", + "fillStyle": "solid", + "strokeWidth": 2, + "strokeStyle": "solid", + "roughness": 1, + "opacity": 100, + "groupIds": [], + "frameId": null, + "roundness": { + "type": 3 + }, + "seed": 1363744610, + "versionNonce": 1801021556, + "isDeleted": false, + "boundElements": [], + "updated": 1, + "link": null, + "locked": false + }, + { + "id": "t906355353", + "type": "text", + "x": 50, + "y": 403, + "width": 847.0000000000001, + "height": 17.5, + "angle": 0, + "strokeColor": "#111111", + "backgroundColor": "transparent", + "fillStyle": "solid", + "strokeWidth": 1, + "strokeStyle": "solid", + "roughness": 1, + "opacity": 100, + "groupIds": [], + "frameId": null, + "roundness": null, + "seed": 785596561, + "versionNonce": 1190527007, + "isDeleted": false, + "boundElements": [], + "updated": 1, + "link": null, + "locked": false, + "text": "ONE clone(2) creates the child already inside all four namespaces \u2014 no prior unshare(), no LockOSThread needed", + "fontSize": 14, + "fontFamily": 1, + "textAlign": "left", + "verticalAlign": "top", + "containerId": null, + "originalText": "ONE clone(2) creates the child already inside all four namespaces \u2014 no prior unshare(), no LockOSThread needed", + "lineHeight": 1.25, + "baseline": 14 + }, + { + "id": "t1751359256", + "type": "text", + "x": 50, + "y": 423, + "width": 665.5, + "height": 13.75, + "angle": 0, + "strokeColor": "#555555", + "backgroundColor": "transparent", + "fillStyle": "solid", + "strokeWidth": 1, + "strokeStyle": "solid", + "roughness": 1, + "opacity": 100, + "groupIds": [], + "frameId": null, + "roundness": null, + "seed": 1990043446, + "versionNonce": 1519557020, + "isDeleted": false, + "boundElements": [], + "updated": 1, + "link": null, + "locked": false, + "text": "child re-execs itself as __ns_init__: mounts / MS_PRIVATE, sethostname, prints proof lines, execs CMD as PID 1", + "fontSize": 11, + "fontFamily": 1, + "textAlign": "left", + "verticalAlign": "top", + "containerId": null, + "originalText": "child re-execs itself as __ns_init__: mounts / MS_PRIVATE, sethostname, prints proof lines, execs CMD as PID 1", + "lineHeight": 1.25, + "baseline": 11 + }, + { + "id": "r1815427810", + "type": "rectangle", + "x": 40, + "y": 465, + "width": 900, + "height": 45, + "angle": 0, + "strokeColor": "#111111", + "backgroundColor": "#ffffff", + "fillStyle": "solid", + "strokeWidth": 2, + "strokeStyle": "solid", + "roughness": 1, + "opacity": 100, + "groupIds": [], + "frameId": null, + "roundness": { + "type": 3 + }, + "seed": 1292311173, + "versionNonce": 407319255, + "isDeleted": false, + "boundElements": [], + "updated": 1, + "link": null, + "locked": false + }, + { + "id": "t1833461972", + "type": "text", + "x": 50, + "y": 473, + "width": 438.90000000000003, + "height": 17.5, + "angle": 0, + "strokeColor": "#111111", + "backgroundColor": "transparent", + "fillStyle": "solid", + "strokeWidth": 1, + "strokeStyle": "solid", + "roughness": 1, + "opacity": 100, + "groupIds": [], + "frameId": null, + "roundness": null, + "seed": 154168837, + "versionNonce": 496295604, + "isDeleted": false, + "boundElements": [], + "updated": 1, + "link": null, + "locked": false, + "text": "PASS 13/13 \u2014 cpp, go, rust \u2014 systems-target, this session", + "fontSize": 14, + "fontFamily": 1, + "textAlign": "left", + "verticalAlign": "top", + "containerId": null, + "originalText": "PASS 13/13 \u2014 cpp, go, rust \u2014 systems-target, this session", + "lineHeight": 1.25, + "baseline": 14 + }, + { + "id": "a1485866992", + "type": "arrow", + "x": 250, + "y": 105, + "width": 0, + "height": 15, + "angle": 0, + "strokeColor": "#555555", + "backgroundColor": "transparent", + "fillStyle": "solid", + "strokeWidth": 2, + "strokeStyle": "solid", + "roughness": 1, + "opacity": 100, + "groupIds": [], + "frameId": null, + "roundness": { + "type": 2 + }, + "seed": 515892033, + "versionNonce": 167234808, + "isDeleted": false, + "boundElements": [], + "updated": 1, + "link": null, + "locked": false, + "points": [ + [ + 0, + 0 + ], + [ + 0, + 15 + ] + ], + "lastCommittedPoint": null, + "startBinding": null, + "endBinding": null, + "startArrowhead": null, + "endArrowhead": "arrow" + }, + { + "id": "a1109961560", + "type": "arrow", + "x": 250, + "y": 165, + "width": 0, + "height": 15, + "angle": 0, + "strokeColor": "#555555", + "backgroundColor": "transparent", + "fillStyle": "solid", + "strokeWidth": 2, + "strokeStyle": "solid", + "roughness": 1, + "opacity": 100, + "groupIds": [], + "frameId": null, + "roundness": { + "type": 2 + }, + "seed": 368550576, + "versionNonce": 456697505, + "isDeleted": false, + "boundElements": [], + "updated": 1, + "link": null, + "locked": false, + "points": [ + [ + 0, + 0 + ], + [ + 0, + 15 + ] + ], + "lastCommittedPoint": null, + "startBinding": null, + "endBinding": null, + "startArrowhead": null, + "endArrowhead": "arrow" + }, + { + "id": "a1888335557", + "type": "arrow", + "x": 730, + "y": 105, + "width": 0, + "height": 15, + "angle": 0, + "strokeColor": "#555555", + "backgroundColor": "transparent", + "fillStyle": "solid", + "strokeWidth": 2, + "strokeStyle": "solid", + "roughness": 1, + "opacity": 100, + "groupIds": [], + "frameId": null, + "roundness": { + "type": 2 + }, + "seed": 1974413273, + "versionNonce": 610095333, + "isDeleted": false, + "boundElements": [], + "updated": 1, + "link": null, + "locked": false, + "points": [ + [ + 0, + 0 + ], + [ + 0, + 15 + ] + ], + "lastCommittedPoint": null, + "startBinding": null, + "endBinding": null, + "startArrowhead": null, + "endArrowhead": "arrow" + }, + { + "id": "a1144608629", + "type": "arrow", + "x": 730, + "y": 165, + "width": 0, + "height": 15, + "angle": 0, + "strokeColor": "#b8650a", + "backgroundColor": "transparent", + "fillStyle": "solid", + "strokeWidth": 2, + "strokeStyle": "solid", + "roughness": 1, + "opacity": 100, + "groupIds": [], + "frameId": null, + "roundness": { + "type": 2 + }, + "seed": 1392441650, + "versionNonce": 236297022, + "isDeleted": false, + "boundElements": [], + "updated": 1, + "link": null, + "locked": false, + "points": [ + [ + 0, + 0 + ], + [ + 0, + 15 + ] + ], + "lastCommittedPoint": null, + "startBinding": null, + "endBinding": null, + "startArrowhead": null, + "endArrowhead": "arrow" + }, + { + "id": "a1800210795", + "type": "arrow", + "x": 250, + "y": 380, + "width": 250, + "height": 22, + "angle": 0, + "strokeColor": "#555555", + "backgroundColor": "transparent", + "fillStyle": "solid", + "strokeWidth": 2, + "strokeStyle": "dashed", + "roughness": 1, + "opacity": 100, + "groupIds": [], + "frameId": null, + "roundness": { + "type": 2 + }, + "seed": 173962332, + "versionNonce": 620415622, + "isDeleted": false, + "boundElements": [], + "updated": 1, + "link": null, + "locked": false, + "points": [ + [ + 0, + 0 + ], + [ + 250, + -22 + ] + ], + "lastCommittedPoint": null, + "startBinding": null, + "endBinding": null, + "startArrowhead": null, + "endArrowhead": "arrow" + }, + { + "id": "t1716701492", + "type": "text", + "x": 375.0, + "y": 355.0, + "width": 229.9, + "height": 13.75, + "angle": 0, + "strokeColor": "#555555", + "backgroundColor": "transparent", + "fillStyle": "solid", + "strokeWidth": 1, + "strokeStyle": "solid", + "roughness": 1, + "opacity": 100, + "groupIds": [], + "frameId": null, + "roundness": null, + "seed": 1776347736, + "versionNonce": 450816834, + "isDeleted": false, + "boundElements": [], + "updated": 1, + "link": null, + "locked": false, + "text": "cgroup membership inherited either way", + "fontSize": 11, + "fontFamily": 1, + "textAlign": "left", + "verticalAlign": "top", + "containerId": null, + "originalText": "cgroup membership inherited either way", + "lineHeight": 1.25, + "baseline": 11 + }, + { + "id": "a1780036223", + "type": "arrow", + "x": 490, + "y": 380, + "width": 0, + "height": 15, + "angle": 0, + "strokeColor": "#b8650a", + "backgroundColor": "transparent", + "fillStyle": "solid", + "strokeWidth": 2, + "strokeStyle": "solid", + "roughness": 1, + "opacity": 100, + "groupIds": [], + "frameId": null, + "roundness": { + "type": 2 + }, + "seed": 331482516, + "versionNonce": 1920362251, + "isDeleted": false, + "boundElements": [], + "updated": 1, + "link": null, + "locked": false, + "points": [ + [ + 0, + 0 + ], + [ + 0, + 15 + ] + ], + "lastCommittedPoint": null, + "startBinding": null, + "endBinding": null, + "startArrowhead": null, + "endArrowhead": "arrow" + }, + { + "id": "a1068087179", + "type": "arrow", + "x": 490, + "y": 450, + "width": 0, + "height": 15, + "angle": 0, + "strokeColor": "#b8650a", + "backgroundColor": "transparent", + "fillStyle": "solid", + "strokeWidth": 2, + "strokeStyle": "solid", + "roughness": 1, + "opacity": 100, + "groupIds": [], + "frameId": null, + "roundness": { + "type": 2 + }, + "seed": 795288959, + "versionNonce": 492017222, + "isDeleted": false, + "boundElements": [], + "updated": 1, + "link": null, + "locked": false, + "points": [ + [ + 0, + 0 + ], + [ + 0, + 15 + ] + ], + "lastCommittedPoint": null, + "startBinding": null, + "endBinding": null, + "startArrowhead": null, + "endArrowhead": "arrow" + }, + { + "id": "t677717792", + "type": "text", + "x": 30, + "y": 545, + "width": 963.6, + "height": 15.0, + "angle": 0, + "strokeColor": "#555555", + "backgroundColor": "transparent", + "fillStyle": "solid", + "strokeWidth": 1, + "strokeStyle": "solid", + "roughness": 1, + "opacity": 100, + "groupIds": [], + "frameId": null, + "roundness": null, + "seed": 1310783058, + "versionNonce": 158224597, + "isDeleted": false, + "boundElements": [], + "updated": 1, + "link": null, + "locked": false, + "text": "The ENOMEM is a real, kernel-documented restriction (golang/go#19716): unshare(CLONE_NEWPID) lets the caller create exactly ONE child in the fresh", + "fontSize": 12, + "fontFamily": 1, + "textAlign": "left", + "verticalAlign": "top", + "containerId": null, + "originalText": "The ENOMEM is a real, kernel-documented restriction (golang/go#19716): unshare(CLONE_NEWPID) lets the caller create exactly ONE child in the fresh", + "lineHeight": 1.25, + "baseline": 12 + }, + { + "id": "t1084514099", + "type": "text", + "x": 30, + "y": 562, + "width": 976.8000000000001, + "height": 15.0, + "angle": 0, + "strokeColor": "#555555", + "backgroundColor": "transparent", + "fillStyle": "solid", + "strokeWidth": 1, + "strokeStyle": "solid", + "roughness": 1, + "opacity": 100, + "groupIds": [], + "frameId": null, + "roundness": null, + "seed": 572258032, + "versionNonce": 859510409, + "isDeleted": false, + "boundElements": [], + "updated": 1, + "link": null, + "locked": false, + "text": "namespace before further forks there return ENOMEM, and os/exec's own vfork/pidfd fast path performs an internal probe fork that silently spends it.", + "fontSize": 12, + "fontFamily": 1, + "textAlign": "left", + "verticalAlign": "top", + "containerId": null, + "originalText": "namespace before further forks there return ENOMEM, and os/exec's own vfork/pidfd fast path performs an internal probe fork that silently spends it.", + "lineHeight": 1.25, + "baseline": 12 + } + ], + "appState": { + "viewBackgroundColor": "#ffffff", + "gridSize": null + }, + "files": {} +} \ No newline at end of file diff --git a/assets/diagrams/32-clone3-flow.svg b/assets/diagrams/32-clone3-flow.svg new file mode 100644 index 0000000..f3b7417 --- /dev/null +++ b/assets/diagrams/32-clone3-flow.svg @@ -0,0 +1,49 @@ + + + +C++ / Rust — unshare(2) then clone3/fork(flags=0) + +Go, first attempt — unshare(2) then re-exec via os/exec + +Go, shipped fix — Cloneflags on the re-exec's SysProcAttr + +unshare(NEWNS|NEWUTS|NEWNET|NEWPID) +takes effect on the caller now + +clone3(flags=0, stack=0) += fork(2): child becomes PID 1 + +child: print pid/hostname, execvp(CMD) +PASS — real run, this session + +runtime.LockOSThread(); unshare(...) +pins goroutine, then unshares + +os/exec.Start() re-execs self +internal vfork/pidfd probe fork first + +clone() = -1 ENOMEM +"fork/exec: cannot allocate memory" — real, this session + +setup_cgroup(): write own pid first +(unchanged — inherited across fork) + +exec.Command(self, hostname, "--", CMD) +SysProcAttr.Cloneflags = NEWNS|NEWUTS|NEWNET|NEWPID + +ONE clone(2) creates the child already inside all four namespaces — no prior unshare(), no LockOSThread needed +child re-execs itself as __ns_init__: mounts / MS_PRIVATE, sethostname, prints proof lines, execs CMD as PID 1 + +PASS 13/13 — cpp, go, rust — systems-target, this session + + + + + +cgroup membership inherited either way +cgroup membership inherited either way + + +The ENOMEM is a real, kernel-documented restriction (golang/go#19716): unshare(CLONE_NEWPID) lets the caller create exactly ONE child in the fresh +namespace before further forks there return ENOMEM, and os/exec's own vfork/pidfd fast path performs an internal probe fork that silently spends it. + \ No newline at end of file diff --git a/assets/diagrams/32-namespace-overview.excalidraw b/assets/diagrams/32-namespace-overview.excalidraw new file mode 100644 index 0000000..97d16b4 --- /dev/null +++ b/assets/diagrams/32-namespace-overview.excalidraw @@ -0,0 +1,2126 @@ +{ + "type": "excalidraw", + "version": 2, + "source": "https://excalidraw.com", + "elements": [ + { + "id": "r730549430", + "type": "rectangle", + "x": 20, + "y": 20, + "width": 460, + "height": 560, + "angle": 0, + "strokeColor": "#dddddd", + "backgroundColor": "#eef4fb", + "fillStyle": "solid", + "strokeWidth": 2, + "strokeStyle": "solid", + "roughness": 1, + "opacity": 100, + "groupIds": [], + "frameId": null, + "roundness": { + "type": 3 + }, + "seed": 1563936846, + "versionNonce": 1889947267, + "isDeleted": false, + "boundElements": [], + "updated": 1, + "link": null, + "locked": false + }, + { + "id": "t656272823", + "type": "text", + "x": 32, + "y": 26, + "width": 250.8, + "height": 15.0, + "angle": 0, + "strokeColor": "#555555", + "backgroundColor": "transparent", + "fillStyle": "solid", + "strokeWidth": 1, + "strokeStyle": "solid", + "roughness": 1, + "opacity": 100, + "groupIds": [], + "frameId": null, + "roundness": null, + "seed": 1699719546, + "versionNonce": 416464291, + "isDeleted": false, + "boundElements": [], + "updated": 1, + "link": null, + "locked": false, + "text": "HOST \u2014 systems-target, pid 1 (systemd)", + "fontSize": 12, + "fontFamily": 1, + "textAlign": "left", + "verticalAlign": "top", + "containerId": null, + "originalText": "HOST \u2014 systems-target, pid 1 (systemd)", + "lineHeight": 1.25, + "baseline": 12 + }, + { + "id": "r1656973957", + "type": "rectangle", + "x": 500, + "y": 20, + "width": 460, + "height": 560, + "angle": 0, + "strokeColor": "#dddddd", + "backgroundColor": "#fff8ef", + "fillStyle": "solid", + "strokeWidth": 2, + "strokeStyle": "solid", + "roughness": 1, + "opacity": 100, + "groupIds": [], + "frameId": null, + "roundness": { + "type": 3 + }, + "seed": 1239306730, + "versionNonce": 1178034472, + "isDeleted": false, + "boundElements": [], + "updated": 1, + "link": null, + "locked": false + }, + { + "id": "t1853167774", + "type": "text", + "x": 512, + "y": 26, + "width": 264.0, + "height": 15.0, + "angle": 0, + "strokeColor": "#555555", + "backgroundColor": "transparent", + "fillStyle": "solid", + "strokeWidth": 1, + "strokeStyle": "solid", + "roughness": 1, + "opacity": 100, + "groupIds": [], + "frameId": null, + "roundness": null, + "seed": 1964818660, + "versionNonce": 1804996397, + "isDeleted": false, + "boundElements": [], + "updated": 1, + "link": null, + "locked": false, + "text": "pmon containerize child \u2014 new PID-1 task", + "fontSize": 12, + "fontFamily": 1, + "textAlign": "left", + "verticalAlign": "top", + "containerId": null, + "originalText": "pmon containerize child \u2014 new PID-1 task", + "lineHeight": 1.25, + "baseline": 12 + }, + { + "id": "r538754812", + "type": "rectangle", + "x": 40, + "y": 50, + "width": 420, + "height": 62, + "angle": 0, + "strokeColor": "#111111", + "backgroundColor": "#ffffff", + "fillStyle": "solid", + "strokeWidth": 2, + "strokeStyle": "solid", + "roughness": 1, + "opacity": 100, + "groupIds": [], + "frameId": null, + "roundness": { + "type": 3 + }, + "seed": 1949457030, + "versionNonce": 1673110478, + "isDeleted": false, + "boundElements": [], + "updated": 1, + "link": null, + "locked": false + }, + { + "id": "t972720252", + "type": "text", + "x": 50, + "y": 58, + "width": 40, + "height": 17.5, + "angle": 0, + "strokeColor": "#111111", + "backgroundColor": "transparent", + "fillStyle": "solid", + "strokeWidth": 1, + "strokeStyle": "solid", + "roughness": 1, + "opacity": 100, + "groupIds": [], + "frameId": null, + "roundness": null, + "seed": 1056122857, + "versionNonce": 1855283263, + "isDeleted": false, + "boundElements": [], + "updated": 1, + "link": null, + "locked": false, + "text": "PID", + "fontSize": 14, + "fontFamily": 1, + "textAlign": "left", + "verticalAlign": "top", + "containerId": null, + "originalText": "PID", + "lineHeight": 1.25, + "baseline": 14 + }, + { + "id": "t1363157110", + "type": "text", + "x": 50, + "y": 78, + "width": 96.80000000000001, + "height": 13.75, + "angle": 0, + "strokeColor": "#555555", + "backgroundColor": "transparent", + "fillStyle": "solid", + "strokeWidth": 1, + "strokeStyle": "solid", + "roughness": 1, + "opacity": 100, + "groupIds": [], + "frameId": null, + "roundness": null, + "seed": 428299106, + "versionNonce": 615444059, + "isDeleted": false, + "boundElements": [], + "updated": 1, + "link": null, + "locked": false, + "text": "pid:[4026531836]", + "fontSize": 11, + "fontFamily": 1, + "textAlign": "left", + "verticalAlign": "top", + "containerId": null, + "originalText": "pid:[4026531836]", + "lineHeight": 1.25, + "baseline": 11 + }, + { + "id": "r275728948", + "type": "rectangle", + "x": 520, + "y": 50, + "width": 420, + "height": 62, + "angle": 0, + "strokeColor": "#e8870c", + "backgroundColor": "#fff8ef", + "fillStyle": "solid", + "strokeWidth": 2, + "strokeStyle": "solid", + "roughness": 1, + "opacity": 100, + "groupIds": [], + "frameId": null, + "roundness": { + "type": 3 + }, + "seed": 1805440063, + "versionNonce": 1998489469, + "isDeleted": false, + "boundElements": [], + "updated": 1, + "link": null, + "locked": false + }, + { + "id": "t732871452", + "type": "text", + "x": 530, + "y": 58, + "width": 107.80000000000001, + "height": 17.5, + "angle": 0, + "strokeColor": "#111111", + "backgroundColor": "transparent", + "fillStyle": "solid", + "strokeWidth": 1, + "strokeStyle": "solid", + "roughness": 1, + "opacity": 100, + "groupIds": [], + "frameId": null, + "roundness": null, + "seed": 167279212, + "versionNonce": 1730293182, + "isDeleted": false, + "boundElements": [], + "updated": 1, + "link": null, + "locked": false, + "text": "PID (unshared)", + "fontSize": 14, + "fontFamily": 1, + "textAlign": "left", + "verticalAlign": "top", + "containerId": null, + "originalText": "PID (unshared)", + "lineHeight": 1.25, + "baseline": 14 + }, + { + "id": "t1148982698", + "type": "text", + "x": 530, + "y": 78, + "width": 96.80000000000001, + "height": 13.75, + "angle": 0, + "strokeColor": "#555555", + "backgroundColor": "transparent", + "fillStyle": "solid", + "strokeWidth": 1, + "strokeStyle": "solid", + "roughness": 1, + "opacity": 100, + "groupIds": [], + "frameId": null, + "roundness": null, + "seed": 1137216759, + "versionNonce": 62620490, + "isDeleted": false, + "boundElements": [], + "updated": 1, + "link": null, + "locked": false, + "text": "pid:[4026532530]", + "fontSize": 11, + "fontFamily": 1, + "textAlign": "left", + "verticalAlign": "top", + "containerId": null, + "originalText": "pid:[4026532530]", + "lineHeight": 1.25, + "baseline": 11 + }, + { + "id": "r198767061", + "type": "rectangle", + "x": 40, + "y": 130, + "width": 420, + "height": 62, + "angle": 0, + "strokeColor": "#111111", + "backgroundColor": "#ffffff", + "fillStyle": "solid", + "strokeWidth": 2, + "strokeStyle": "solid", + "roughness": 1, + "opacity": 100, + "groupIds": [], + "frameId": null, + "roundness": { + "type": 3 + }, + "seed": 1198481218, + "versionNonce": 1918746167, + "isDeleted": false, + "boundElements": [], + "updated": 1, + "link": null, + "locked": false + }, + { + "id": "t1711507601", + "type": "text", + "x": 50, + "y": 138, + "width": 40, + "height": 17.5, + "angle": 0, + "strokeColor": "#111111", + "backgroundColor": "transparent", + "fillStyle": "solid", + "strokeWidth": 1, + "strokeStyle": "solid", + "roughness": 1, + "opacity": 100, + "groupIds": [], + "frameId": null, + "roundness": null, + "seed": 508673179, + "versionNonce": 92192552, + "isDeleted": false, + "boundElements": [], + "updated": 1, + "link": null, + "locked": false, + "text": "Mount", + "fontSize": 14, + "fontFamily": 1, + "textAlign": "left", + "verticalAlign": "top", + "containerId": null, + "originalText": "Mount", + "lineHeight": 1.25, + "baseline": 14 + }, + { + "id": "t878047813", + "type": "text", + "x": 50, + "y": 158, + "width": 96.80000000000001, + "height": 13.75, + "angle": 0, + "strokeColor": "#555555", + "backgroundColor": "transparent", + "fillStyle": "solid", + "strokeWidth": 1, + "strokeStyle": "solid", + "roughness": 1, + "opacity": 100, + "groupIds": [], + "frameId": null, + "roundness": null, + "seed": 1597558583, + "versionNonce": 682772364, + "isDeleted": false, + "boundElements": [], + "updated": 1, + "link": null, + "locked": false, + "text": "mnt:[4026531832]", + "fontSize": 11, + "fontFamily": 1, + "textAlign": "left", + "verticalAlign": "top", + "containerId": null, + "originalText": "mnt:[4026531832]", + "lineHeight": 1.25, + "baseline": 11 + }, + { + "id": "r603514496", + "type": "rectangle", + "x": 520, + "y": 130, + "width": 420, + "height": 62, + "angle": 0, + "strokeColor": "#e8870c", + "backgroundColor": "#fff8ef", + "fillStyle": "solid", + "strokeWidth": 2, + "strokeStyle": "solid", + "roughness": 1, + "opacity": 100, + "groupIds": [], + "frameId": null, + "roundness": { + "type": 3 + }, + "seed": 1051103990, + "versionNonce": 751884763, + "isDeleted": false, + "boundElements": [], + "updated": 1, + "link": null, + "locked": false + }, + { + "id": "t22145396", + "type": "text", + "x": 530, + "y": 138, + "width": 123.20000000000002, + "height": 17.5, + "angle": 0, + "strokeColor": "#111111", + "backgroundColor": "transparent", + "fillStyle": "solid", + "strokeWidth": 1, + "strokeStyle": "solid", + "roughness": 1, + "opacity": 100, + "groupIds": [], + "frameId": null, + "roundness": null, + "seed": 384476464, + "versionNonce": 262656207, + "isDeleted": false, + "boundElements": [], + "updated": 1, + "link": null, + "locked": false, + "text": "Mount (unshared)", + "fontSize": 14, + "fontFamily": 1, + "textAlign": "left", + "verticalAlign": "top", + "containerId": null, + "originalText": "Mount (unshared)", + "lineHeight": 1.25, + "baseline": 14 + }, + { + "id": "t1353312934", + "type": "text", + "x": 530, + "y": 158, + "width": 96.80000000000001, + "height": 13.75, + "angle": 0, + "strokeColor": "#555555", + "backgroundColor": "transparent", + "fillStyle": "solid", + "strokeWidth": 1, + "strokeStyle": "solid", + "roughness": 1, + "opacity": 100, + "groupIds": [], + "frameId": null, + "roundness": null, + "seed": 1181085777, + "versionNonce": 1204093531, + "isDeleted": false, + "boundElements": [], + "updated": 1, + "link": null, + "locked": false, + "text": "mnt:[4026532528]", + "fontSize": 11, + "fontFamily": 1, + "textAlign": "left", + "verticalAlign": "top", + "containerId": null, + "originalText": "mnt:[4026532528]", + "lineHeight": 1.25, + "baseline": 11 + }, + { + "id": "r1369674197", + "type": "rectangle", + "x": 40, + "y": 210, + "width": 420, + "height": 62, + "angle": 0, + "strokeColor": "#111111", + "backgroundColor": "#ffffff", + "fillStyle": "solid", + "strokeWidth": 2, + "strokeStyle": "solid", + "roughness": 1, + "opacity": 100, + "groupIds": [], + "frameId": null, + "roundness": { + "type": 3 + }, + "seed": 973577472, + "versionNonce": 319651386, + "isDeleted": false, + "boundElements": [], + "updated": 1, + "link": null, + "locked": false + }, + { + "id": "t1994094896", + "type": "text", + "x": 50, + "y": 218, + "width": 40, + "height": 17.5, + "angle": 0, + "strokeColor": "#111111", + "backgroundColor": "transparent", + "fillStyle": "solid", + "strokeWidth": 1, + "strokeStyle": "solid", + "roughness": 1, + "opacity": 100, + "groupIds": [], + "frameId": null, + "roundness": null, + "seed": 1953810349, + "versionNonce": 1134357031, + "isDeleted": false, + "boundElements": [], + "updated": 1, + "link": null, + "locked": false, + "text": "UTS", + "fontSize": 14, + "fontFamily": 1, + "textAlign": "left", + "verticalAlign": "top", + "containerId": null, + "originalText": "UTS", + "lineHeight": 1.25, + "baseline": 14 + }, + { + "id": "t1067667600", + "type": "text", + "x": 50, + "y": 238, + "width": 96.80000000000001, + "height": 13.75, + "angle": 0, + "strokeColor": "#555555", + "backgroundColor": "transparent", + "fillStyle": "solid", + "strokeWidth": 1, + "strokeStyle": "solid", + "roughness": 1, + "opacity": 100, + "groupIds": [], + "frameId": null, + "roundness": null, + "seed": 1716926339, + "versionNonce": 339354388, + "isDeleted": false, + "boundElements": [], + "updated": 1, + "link": null, + "locked": false, + "text": "uts:[4026531838]", + "fontSize": 11, + "fontFamily": 1, + "textAlign": "left", + "verticalAlign": "top", + "containerId": null, + "originalText": "uts:[4026531838]", + "lineHeight": 1.25, + "baseline": 11 + }, + { + "id": "r1678544694", + "type": "rectangle", + "x": 520, + "y": 210, + "width": 420, + "height": 62, + "angle": 0, + "strokeColor": "#e8870c", + "backgroundColor": "#fff8ef", + "fillStyle": "solid", + "strokeWidth": 2, + "strokeStyle": "solid", + "roughness": 1, + "opacity": 100, + "groupIds": [], + "frameId": null, + "roundness": { + "type": 3 + }, + "seed": 185385367, + "versionNonce": 420324327, + "isDeleted": false, + "boundElements": [], + "updated": 1, + "link": null, + "locked": false + }, + { + "id": "t1245687302", + "type": "text", + "x": 530, + "y": 218, + "width": 107.80000000000001, + "height": 17.5, + "angle": 0, + "strokeColor": "#111111", + "backgroundColor": "transparent", + "fillStyle": "solid", + "strokeWidth": 1, + "strokeStyle": "solid", + "roughness": 1, + "opacity": 100, + "groupIds": [], + "frameId": null, + "roundness": null, + "seed": 939882666, + "versionNonce": 189008910, + "isDeleted": false, + "boundElements": [], + "updated": 1, + "link": null, + "locked": false, + "text": "UTS (unshared)", + "fontSize": 14, + "fontFamily": 1, + "textAlign": "left", + "verticalAlign": "top", + "containerId": null, + "originalText": "UTS (unshared)", + "lineHeight": 1.25, + "baseline": 14 + }, + { + "id": "t718041104", + "type": "text", + "x": 530, + "y": 238, + "width": 96.80000000000001, + "height": 13.75, + "angle": 0, + "strokeColor": "#555555", + "backgroundColor": "transparent", + "fillStyle": "solid", + "strokeWidth": 1, + "strokeStyle": "solid", + "roughness": 1, + "opacity": 100, + "groupIds": [], + "frameId": null, + "roundness": null, + "seed": 945177123, + "versionNonce": 1767568899, + "isDeleted": false, + "boundElements": [], + "updated": 1, + "link": null, + "locked": false, + "text": "uts:[4026532529]", + "fontSize": 11, + "fontFamily": 1, + "textAlign": "left", + "verticalAlign": "top", + "containerId": null, + "originalText": "uts:[4026532529]", + "lineHeight": 1.25, + "baseline": 11 + }, + { + "id": "r1840704368", + "type": "rectangle", + "x": 40, + "y": 290, + "width": 420, + "height": 62, + "angle": 0, + "strokeColor": "#111111", + "backgroundColor": "#ffffff", + "fillStyle": "solid", + "strokeWidth": 2, + "strokeStyle": "solid", + "roughness": 1, + "opacity": 100, + "groupIds": [], + "frameId": null, + "roundness": { + "type": 3 + }, + "seed": 961215991, + "versionNonce": 1255802411, + "isDeleted": false, + "boundElements": [], + "updated": 1, + "link": null, + "locked": false + }, + { + "id": "t563639951", + "type": "text", + "x": 50, + "y": 298, + "width": 53.900000000000006, + "height": 17.5, + "angle": 0, + "strokeColor": "#111111", + "backgroundColor": "transparent", + "fillStyle": "solid", + "strokeWidth": 1, + "strokeStyle": "solid", + "roughness": 1, + "opacity": 100, + "groupIds": [], + "frameId": null, + "roundness": null, + "seed": 837101959, + "versionNonce": 1147968767, + "isDeleted": false, + "boundElements": [], + "updated": 1, + "link": null, + "locked": false, + "text": "Network", + "fontSize": 14, + "fontFamily": 1, + "textAlign": "left", + "verticalAlign": "top", + "containerId": null, + "originalText": "Network", + "lineHeight": 1.25, + "baseline": 14 + }, + { + "id": "t824385312", + "type": "text", + "x": 50, + "y": 318, + "width": 96.80000000000001, + "height": 13.75, + "angle": 0, + "strokeColor": "#555555", + "backgroundColor": "transparent", + "fillStyle": "solid", + "strokeWidth": 1, + "strokeStyle": "solid", + "roughness": 1, + "opacity": 100, + "groupIds": [], + "frameId": null, + "roundness": null, + "seed": 1292244116, + "versionNonce": 1667945007, + "isDeleted": false, + "boundElements": [], + "updated": 1, + "link": null, + "locked": false, + "text": "net:[4026531833]", + "fontSize": 11, + "fontFamily": 1, + "textAlign": "left", + "verticalAlign": "top", + "containerId": null, + "originalText": "net:[4026531833]", + "lineHeight": 1.25, + "baseline": 11 + }, + { + "id": "r1713390894", + "type": "rectangle", + "x": 520, + "y": 290, + "width": 420, + "height": 62, + "angle": 0, + "strokeColor": "#e8870c", + "backgroundColor": "#fff8ef", + "fillStyle": "solid", + "strokeWidth": 2, + "strokeStyle": "solid", + "roughness": 1, + "opacity": 100, + "groupIds": [], + "frameId": null, + "roundness": { + "type": 3 + }, + "seed": 1471529599, + "versionNonce": 251258587, + "isDeleted": false, + "boundElements": [], + "updated": 1, + "link": null, + "locked": false + }, + { + "id": "t169067300", + "type": "text", + "x": 530, + "y": 298, + "width": 138.60000000000002, + "height": 17.5, + "angle": 0, + "strokeColor": "#111111", + "backgroundColor": "transparent", + "fillStyle": "solid", + "strokeWidth": 1, + "strokeStyle": "solid", + "roughness": 1, + "opacity": 100, + "groupIds": [], + "frameId": null, + "roundness": null, + "seed": 116448509, + "versionNonce": 1276743168, + "isDeleted": false, + "boundElements": [], + "updated": 1, + "link": null, + "locked": false, + "text": "Network (unshared)", + "fontSize": 14, + "fontFamily": 1, + "textAlign": "left", + "verticalAlign": "top", + "containerId": null, + "originalText": "Network (unshared)", + "lineHeight": 1.25, + "baseline": 14 + }, + { + "id": "t525924106", + "type": "text", + "x": 530, + "y": 318, + "width": 96.80000000000001, + "height": 13.75, + "angle": 0, + "strokeColor": "#555555", + "backgroundColor": "transparent", + "fillStyle": "solid", + "strokeWidth": 1, + "strokeStyle": "solid", + "roughness": 1, + "opacity": 100, + "groupIds": [], + "frameId": null, + "roundness": null, + "seed": 1770622123, + "versionNonce": 1664100368, + "isDeleted": false, + "boundElements": [], + "updated": 1, + "link": null, + "locked": false, + "text": "net:[4026532531]", + "fontSize": 11, + "fontFamily": 1, + "textAlign": "left", + "verticalAlign": "top", + "containerId": null, + "originalText": "net:[4026532531]", + "lineHeight": 1.25, + "baseline": 11 + }, + { + "id": "r545104879", + "type": "rectangle", + "x": 40, + "y": 370, + "width": 420, + "height": 62, + "angle": 0, + "strokeColor": "#111111", + "backgroundColor": "#ffffff", + "fillStyle": "solid", + "strokeWidth": 2, + "strokeStyle": "solid", + "roughness": 1, + "opacity": 100, + "groupIds": [], + "frameId": null, + "roundness": { + "type": 3 + }, + "seed": 1139503734, + "versionNonce": 1415423758, + "isDeleted": false, + "boundElements": [], + "updated": 1, + "link": null, + "locked": false + }, + { + "id": "t525349183", + "type": "text", + "x": 50, + "y": 378, + "width": 40, + "height": 17.5, + "angle": 0, + "strokeColor": "#111111", + "backgroundColor": "transparent", + "fillStyle": "solid", + "strokeWidth": 1, + "strokeStyle": "solid", + "roughness": 1, + "opacity": 100, + "groupIds": [], + "frameId": null, + "roundness": null, + "seed": 1322892966, + "versionNonce": 207239270, + "isDeleted": false, + "boundElements": [], + "updated": 1, + "link": null, + "locked": false, + "text": "Time", + "fontSize": 14, + "fontFamily": 1, + "textAlign": "left", + "verticalAlign": "top", + "containerId": null, + "originalText": "Time", + "lineHeight": 1.25, + "baseline": 14 + }, + { + "id": "t708563433", + "type": "text", + "x": 50, + "y": 398, + "width": 102.85000000000001, + "height": 13.75, + "angle": 0, + "strokeColor": "#555555", + "backgroundColor": "transparent", + "fillStyle": "solid", + "strokeWidth": 1, + "strokeStyle": "solid", + "roughness": 1, + "opacity": 100, + "groupIds": [], + "frameId": null, + "roundness": null, + "seed": 1488961995, + "versionNonce": 1511277206, + "isDeleted": false, + "boundElements": [], + "updated": 1, + "link": null, + "locked": false, + "text": "time:[4026531834]", + "fontSize": 11, + "fontFamily": 1, + "textAlign": "left", + "verticalAlign": "top", + "containerId": null, + "originalText": "time:[4026531834]", + "lineHeight": 1.25, + "baseline": 11 + }, + { + "id": "r350726504", + "type": "rectangle", + "x": 520, + "y": 370, + "width": 420, + "height": 62, + "angle": 0, + "strokeColor": "#999999", + "backgroundColor": "#ffffff", + "fillStyle": "solid", + "strokeWidth": 2, + "strokeStyle": "dashed", + "roughness": 1, + "opacity": 100, + "groupIds": [], + "frameId": null, + "roundness": { + "type": 3 + }, + "seed": 1242237834, + "versionNonce": 823827127, + "isDeleted": false, + "boundElements": [], + "updated": 1, + "link": null, + "locked": false + }, + { + "id": "t508228479", + "type": "text", + "x": 530, + "y": 378, + "width": 177.10000000000002, + "height": 17.5, + "angle": 0, + "strokeColor": "#111111", + "backgroundColor": "transparent", + "fillStyle": "solid", + "strokeWidth": 1, + "strokeStyle": "solid", + "roughness": 1, + "opacity": 100, + "groupIds": [], + "frameId": null, + "roundness": null, + "seed": 1482122539, + "versionNonce": 573548812, + "isDeleted": false, + "boundElements": [], + "updated": 1, + "link": null, + "locked": false, + "text": "Time (shared with host)", + "fontSize": 14, + "fontFamily": 1, + "textAlign": "left", + "verticalAlign": "top", + "containerId": null, + "originalText": "Time (shared with host)", + "lineHeight": 1.25, + "baseline": 14 + }, + { + "id": "t664514990", + "type": "text", + "x": 530, + "y": 398, + "width": 102.85000000000001, + "height": 13.75, + "angle": 0, + "strokeColor": "#555555", + "backgroundColor": "transparent", + "fillStyle": "solid", + "strokeWidth": 1, + "strokeStyle": "solid", + "roughness": 1, + "opacity": 100, + "groupIds": [], + "frameId": null, + "roundness": null, + "seed": 889587577, + "versionNonce": 397670965, + "isDeleted": false, + "boundElements": [], + "updated": 1, + "link": null, + "locked": false, + "text": "time:[4026531834]", + "fontSize": 11, + "fontFamily": 1, + "textAlign": "left", + "verticalAlign": "top", + "containerId": null, + "originalText": "time:[4026531834]", + "lineHeight": 1.25, + "baseline": 11 + }, + { + "id": "r669417447", + "type": "rectangle", + "x": 40, + "y": 450, + "width": 420, + "height": 62, + "angle": 0, + "strokeColor": "#111111", + "backgroundColor": "#ffffff", + "fillStyle": "solid", + "strokeWidth": 2, + "strokeStyle": "solid", + "roughness": 1, + "opacity": 100, + "groupIds": [], + "frameId": null, + "roundness": { + "type": 3 + }, + "seed": 357960644, + "versionNonce": 1502102390, + "isDeleted": false, + "boundElements": [], + "updated": 1, + "link": null, + "locked": false + }, + { + "id": "t587377995", + "type": "text", + "x": 50, + "y": 458, + "width": 40, + "height": 17.5, + "angle": 0, + "strokeColor": "#111111", + "backgroundColor": "transparent", + "fillStyle": "solid", + "strokeWidth": 1, + "strokeStyle": "solid", + "roughness": 1, + "opacity": 100, + "groupIds": [], + "frameId": null, + "roundness": null, + "seed": 789325198, + "versionNonce": 1129962394, + "isDeleted": false, + "boundElements": [], + "updated": 1, + "link": null, + "locked": false, + "text": "IPC", + "fontSize": 14, + "fontFamily": 1, + "textAlign": "left", + "verticalAlign": "top", + "containerId": null, + "originalText": "IPC", + "lineHeight": 1.25, + "baseline": 14 + }, + { + "id": "t782119772", + "type": "text", + "x": 50, + "y": 478, + "width": 96.80000000000001, + "height": 13.75, + "angle": 0, + "strokeColor": "#555555", + "backgroundColor": "transparent", + "fillStyle": "solid", + "strokeWidth": 1, + "strokeStyle": "solid", + "roughness": 1, + "opacity": 100, + "groupIds": [], + "frameId": null, + "roundness": null, + "seed": 212586293, + "versionNonce": 485289905, + "isDeleted": false, + "boundElements": [], + "updated": 1, + "link": null, + "locked": false, + "text": "ipc:[4026531839]", + "fontSize": 11, + "fontFamily": 1, + "textAlign": "left", + "verticalAlign": "top", + "containerId": null, + "originalText": "ipc:[4026531839]", + "lineHeight": 1.25, + "baseline": 11 + }, + { + "id": "r1091598397", + "type": "rectangle", + "x": 520, + "y": 450, + "width": 420, + "height": 62, + "angle": 0, + "strokeColor": "#999999", + "backgroundColor": "#ffffff", + "fillStyle": "solid", + "strokeWidth": 2, + "strokeStyle": "dashed", + "roughness": 1, + "opacity": 100, + "groupIds": [], + "frameId": null, + "roundness": { + "type": 3 + }, + "seed": 1053001440, + "versionNonce": 1545532260, + "isDeleted": false, + "boundElements": [], + "updated": 1, + "link": null, + "locked": false + }, + { + "id": "t1304832762", + "type": "text", + "x": 530, + "y": 458, + "width": 169.4, + "height": 17.5, + "angle": 0, + "strokeColor": "#111111", + "backgroundColor": "transparent", + "fillStyle": "solid", + "strokeWidth": 1, + "strokeStyle": "solid", + "roughness": 1, + "opacity": 100, + "groupIds": [], + "frameId": null, + "roundness": null, + "seed": 1877463476, + "versionNonce": 24438534, + "isDeleted": false, + "boundElements": [], + "updated": 1, + "link": null, + "locked": false, + "text": "IPC (shared with host)", + "fontSize": 14, + "fontFamily": 1, + "textAlign": "left", + "verticalAlign": "top", + "containerId": null, + "originalText": "IPC (shared with host)", + "lineHeight": 1.25, + "baseline": 14 + }, + { + "id": "t486981186", + "type": "text", + "x": 530, + "y": 478, + "width": 96.80000000000001, + "height": 13.75, + "angle": 0, + "strokeColor": "#555555", + "backgroundColor": "transparent", + "fillStyle": "solid", + "strokeWidth": 1, + "strokeStyle": "solid", + "roughness": 1, + "opacity": 100, + "groupIds": [], + "frameId": null, + "roundness": null, + "seed": 1973874053, + "versionNonce": 352765687, + "isDeleted": false, + "boundElements": [], + "updated": 1, + "link": null, + "locked": false, + "text": "ipc:[4026531839]", + "fontSize": 11, + "fontFamily": 1, + "textAlign": "left", + "verticalAlign": "top", + "containerId": null, + "originalText": "ipc:[4026531839]", + "lineHeight": 1.25, + "baseline": 11 + }, + { + "id": "r684995067", + "type": "rectangle", + "x": 40, + "y": 530, + "width": 420, + "height": 62, + "angle": 0, + "strokeColor": "#111111", + "backgroundColor": "#ffffff", + "fillStyle": "solid", + "strokeWidth": 2, + "strokeStyle": "solid", + "roughness": 1, + "opacity": 100, + "groupIds": [], + "frameId": null, + "roundness": { + "type": 3 + }, + "seed": 549273456, + "versionNonce": 1657609628, + "isDeleted": false, + "boundElements": [], + "updated": 1, + "link": null, + "locked": false + }, + { + "id": "t726598529", + "type": "text", + "x": 50, + "y": 538, + "width": 100.10000000000001, + "height": 17.5, + "angle": 0, + "strokeColor": "#111111", + "backgroundColor": "transparent", + "fillStyle": "solid", + "strokeWidth": 1, + "strokeStyle": "solid", + "roughness": 1, + "opacity": 100, + "groupIds": [], + "frameId": null, + "roundness": null, + "seed": 1864783867, + "versionNonce": 934919270, + "isDeleted": false, + "boundElements": [], + "updated": 1, + "link": null, + "locked": false, + "text": "User / Cgroup", + "fontSize": 14, + "fontFamily": 1, + "textAlign": "left", + "verticalAlign": "top", + "containerId": null, + "originalText": "User / Cgroup", + "lineHeight": 1.25, + "baseline": 14 + }, + { + "id": "t335680215", + "type": "text", + "x": 50, + "y": 558, + "width": 211.75000000000003, + "height": 13.75, + "angle": 0, + "strokeColor": "#555555", + "backgroundColor": "transparent", + "fillStyle": "solid", + "strokeWidth": 1, + "strokeStyle": "solid", + "roughness": 1, + "opacity": 100, + "groupIds": [], + "frameId": null, + "roundness": null, + "seed": 1153272878, + "versionNonce": 97854005, + "isDeleted": false, + "boundElements": [], + "updated": 1, + "link": null, + "locked": false, + "text": "user/cgroup:[4026531837/4026531835]", + "fontSize": 11, + "fontFamily": 1, + "textAlign": "left", + "verticalAlign": "top", + "containerId": null, + "originalText": "user/cgroup:[4026531837/4026531835]", + "lineHeight": 1.25, + "baseline": 11 + }, + { + "id": "r607899931", + "type": "rectangle", + "x": 520, + "y": 530, + "width": 420, + "height": 62, + "angle": 0, + "strokeColor": "#999999", + "backgroundColor": "#ffffff", + "fillStyle": "solid", + "strokeWidth": 2, + "strokeStyle": "dashed", + "roughness": 1, + "opacity": 100, + "groupIds": [], + "frameId": null, + "roundness": { + "type": 3 + }, + "seed": 552770886, + "versionNonce": 1744473633, + "isDeleted": false, + "boundElements": [], + "updated": 1, + "link": null, + "locked": false + }, + { + "id": "t966694459", + "type": "text", + "x": 530, + "y": 538, + "width": 246.40000000000003, + "height": 17.5, + "angle": 0, + "strokeColor": "#111111", + "backgroundColor": "transparent", + "fillStyle": "solid", + "strokeWidth": 1, + "strokeStyle": "solid", + "roughness": 1, + "opacity": 100, + "groupIds": [], + "frameId": null, + "roundness": null, + "seed": 533749199, + "versionNonce": 1941672580, + "isDeleted": false, + "boundElements": [], + "updated": 1, + "link": null, + "locked": false, + "text": "User / Cgroup (shared with host)", + "fontSize": 14, + "fontFamily": 1, + "textAlign": "left", + "verticalAlign": "top", + "containerId": null, + "originalText": "User / Cgroup (shared with host)", + "lineHeight": 1.25, + "baseline": 14 + }, + { + "id": "t668282055", + "type": "text", + "x": 530, + "y": 558, + "width": 211.75000000000003, + "height": 13.75, + "angle": 0, + "strokeColor": "#555555", + "backgroundColor": "transparent", + "fillStyle": "solid", + "strokeWidth": 1, + "strokeStyle": "solid", + "roughness": 1, + "opacity": 100, + "groupIds": [], + "frameId": null, + "roundness": null, + "seed": 831776877, + "versionNonce": 189325756, + "isDeleted": false, + "boundElements": [], + "updated": 1, + "link": null, + "locked": false, + "text": "user/cgroup:[4026531837/4026531835]", + "fontSize": 11, + "fontFamily": 1, + "textAlign": "left", + "verticalAlign": "top", + "containerId": null, + "originalText": "user/cgroup:[4026531837/4026531835]", + "lineHeight": 1.25, + "baseline": 11 + }, + { + "id": "a1233880352", + "type": "arrow", + "x": 460, + "y": 81, + "width": 60, + "height": 0, + "angle": 0, + "strokeColor": "#b8650a", + "backgroundColor": "transparent", + "fillStyle": "solid", + "strokeWidth": 2, + "strokeStyle": "solid", + "roughness": 1, + "opacity": 100, + "groupIds": [], + "frameId": null, + "roundness": { + "type": 2 + }, + "seed": 1843273728, + "versionNonce": 239248101, + "isDeleted": false, + "boundElements": [], + "updated": 1, + "link": null, + "locked": false, + "points": [ + [ + 0, + 0 + ], + [ + 60, + 0 + ] + ], + "lastCommittedPoint": null, + "startBinding": null, + "endBinding": null, + "startArrowhead": null, + "endArrowhead": "arrow" + }, + { + "id": "t1964734558", + "type": "text", + "x": 490.0, + "y": 67.0, + "width": 42.35, + "height": 13.75, + "angle": 0, + "strokeColor": "#b8650a", + "backgroundColor": "transparent", + "fillStyle": "solid", + "strokeWidth": 1, + "strokeStyle": "solid", + "roughness": 1, + "opacity": 100, + "groupIds": [], + "frameId": null, + "roundness": null, + "seed": 1869404594, + "versionNonce": 35851803, + "isDeleted": false, + "boundElements": [], + "updated": 1, + "link": null, + "locked": false, + "text": "differs", + "fontSize": 11, + "fontFamily": 1, + "textAlign": "left", + "verticalAlign": "top", + "containerId": null, + "originalText": "differs", + "lineHeight": 1.25, + "baseline": 11 + }, + { + "id": "a1987082369", + "type": "arrow", + "x": 460, + "y": 161, + "width": 60, + "height": 0, + "angle": 0, + "strokeColor": "#b8650a", + "backgroundColor": "transparent", + "fillStyle": "solid", + "strokeWidth": 2, + "strokeStyle": "solid", + "roughness": 1, + "opacity": 100, + "groupIds": [], + "frameId": null, + "roundness": { + "type": 2 + }, + "seed": 206293974, + "versionNonce": 864839748, + "isDeleted": false, + "boundElements": [], + "updated": 1, + "link": null, + "locked": false, + "points": [ + [ + 0, + 0 + ], + [ + 60, + 0 + ] + ], + "lastCommittedPoint": null, + "startBinding": null, + "endBinding": null, + "startArrowhead": null, + "endArrowhead": "arrow" + }, + { + "id": "t333792366", + "type": "text", + "x": 490.0, + "y": 147.0, + "width": 42.35, + "height": 13.75, + "angle": 0, + "strokeColor": "#b8650a", + "backgroundColor": "transparent", + "fillStyle": "solid", + "strokeWidth": 1, + "strokeStyle": "solid", + "roughness": 1, + "opacity": 100, + "groupIds": [], + "frameId": null, + "roundness": null, + "seed": 1145070023, + "versionNonce": 352399609, + "isDeleted": false, + "boundElements": [], + "updated": 1, + "link": null, + "locked": false, + "text": "differs", + "fontSize": 11, + "fontFamily": 1, + "textAlign": "left", + "verticalAlign": "top", + "containerId": null, + "originalText": "differs", + "lineHeight": 1.25, + "baseline": 11 + }, + { + "id": "a870130486", + "type": "arrow", + "x": 460, + "y": 241, + "width": 60, + "height": 0, + "angle": 0, + "strokeColor": "#b8650a", + "backgroundColor": "transparent", + "fillStyle": "solid", + "strokeWidth": 2, + "strokeStyle": "solid", + "roughness": 1, + "opacity": 100, + "groupIds": [], + "frameId": null, + "roundness": { + "type": 2 + }, + "seed": 728771874, + "versionNonce": 501000875, + "isDeleted": false, + "boundElements": [], + "updated": 1, + "link": null, + "locked": false, + "points": [ + [ + 0, + 0 + ], + [ + 60, + 0 + ] + ], + "lastCommittedPoint": null, + "startBinding": null, + "endBinding": null, + "startArrowhead": null, + "endArrowhead": "arrow" + }, + { + "id": "t818731103", + "type": "text", + "x": 490.0, + "y": 227.0, + "width": 42.35, + "height": 13.75, + "angle": 0, + "strokeColor": "#b8650a", + "backgroundColor": "transparent", + "fillStyle": "solid", + "strokeWidth": 1, + "strokeStyle": "solid", + "roughness": 1, + "opacity": 100, + "groupIds": [], + "frameId": null, + "roundness": null, + "seed": 829405956, + "versionNonce": 1491547764, + "isDeleted": false, + "boundElements": [], + "updated": 1, + "link": null, + "locked": false, + "text": "differs", + "fontSize": 11, + "fontFamily": 1, + "textAlign": "left", + "verticalAlign": "top", + "containerId": null, + "originalText": "differs", + "lineHeight": 1.25, + "baseline": 11 + }, + { + "id": "a1708755674", + "type": "arrow", + "x": 460, + "y": 321, + "width": 60, + "height": 0, + "angle": 0, + "strokeColor": "#b8650a", + "backgroundColor": "transparent", + "fillStyle": "solid", + "strokeWidth": 2, + "strokeStyle": "solid", + "roughness": 1, + "opacity": 100, + "groupIds": [], + "frameId": null, + "roundness": { + "type": 2 + }, + "seed": 1556446005, + "versionNonce": 52985085, + "isDeleted": false, + "boundElements": [], + "updated": 1, + "link": null, + "locked": false, + "points": [ + [ + 0, + 0 + ], + [ + 60, + 0 + ] + ], + "lastCommittedPoint": null, + "startBinding": null, + "endBinding": null, + "startArrowhead": null, + "endArrowhead": "arrow" + }, + { + "id": "t588704360", + "type": "text", + "x": 490.0, + "y": 307.0, + "width": 42.35, + "height": 13.75, + "angle": 0, + "strokeColor": "#b8650a", + "backgroundColor": "transparent", + "fillStyle": "solid", + "strokeWidth": 1, + "strokeStyle": "solid", + "roughness": 1, + "opacity": 100, + "groupIds": [], + "frameId": null, + "roundness": null, + "seed": 1450184150, + "versionNonce": 1993593383, + "isDeleted": false, + "boundElements": [], + "updated": 1, + "link": null, + "locked": false, + "text": "differs", + "fontSize": 11, + "fontFamily": 1, + "textAlign": "left", + "verticalAlign": "top", + "containerId": null, + "originalText": "differs", + "lineHeight": 1.25, + "baseline": 11 + }, + { + "id": "a518171346", + "type": "arrow", + "x": 460, + "y": 401, + "width": 60, + "height": 0, + "angle": 0, + "strokeColor": "#555555", + "backgroundColor": "transparent", + "fillStyle": "solid", + "strokeWidth": 2, + "strokeStyle": "dashed", + "roughness": 1, + "opacity": 100, + "groupIds": [], + "frameId": null, + "roundness": { + "type": 2 + }, + "seed": 788632443, + "versionNonce": 657813861, + "isDeleted": false, + "boundElements": [], + "updated": 1, + "link": null, + "locked": false, + "points": [ + [ + 0, + 0 + ], + [ + 60, + 0 + ] + ], + "lastCommittedPoint": null, + "startBinding": null, + "endBinding": null, + "startArrowhead": null, + "endArrowhead": "arrow" + }, + { + "id": "t843527113", + "type": "text", + "x": 490.0, + "y": 387.0, + "width": 60.50000000000001, + "height": 13.75, + "angle": 0, + "strokeColor": "#555555", + "backgroundColor": "transparent", + "fillStyle": "solid", + "strokeWidth": 1, + "strokeStyle": "solid", + "roughness": 1, + "opacity": 100, + "groupIds": [], + "frameId": null, + "roundness": null, + "seed": 1302527742, + "versionNonce": 95057269, + "isDeleted": false, + "boundElements": [], + "updated": 1, + "link": null, + "locked": false, + "text": "same inode", + "fontSize": 11, + "fontFamily": 1, + "textAlign": "left", + "verticalAlign": "top", + "containerId": null, + "originalText": "same inode", + "lineHeight": 1.25, + "baseline": 11 + }, + { + "id": "a1104826113", + "type": "arrow", + "x": 460, + "y": 481, + "width": 60, + "height": 0, + "angle": 0, + "strokeColor": "#555555", + "backgroundColor": "transparent", + "fillStyle": "solid", + "strokeWidth": 2, + "strokeStyle": "dashed", + "roughness": 1, + "opacity": 100, + "groupIds": [], + "frameId": null, + "roundness": { + "type": 2 + }, + "seed": 719056253, + "versionNonce": 1545499183, + "isDeleted": false, + "boundElements": [], + "updated": 1, + "link": null, + "locked": false, + "points": [ + [ + 0, + 0 + ], + [ + 60, + 0 + ] + ], + "lastCommittedPoint": null, + "startBinding": null, + "endBinding": null, + "startArrowhead": null, + "endArrowhead": "arrow" + }, + { + "id": "t1299473586", + "type": "text", + "x": 490.0, + "y": 467.0, + "width": 60.50000000000001, + "height": 13.75, + "angle": 0, + "strokeColor": "#555555", + "backgroundColor": "transparent", + "fillStyle": "solid", + "strokeWidth": 1, + "strokeStyle": "solid", + "roughness": 1, + "opacity": 100, + "groupIds": [], + "frameId": null, + "roundness": null, + "seed": 407096312, + "versionNonce": 232355792, + "isDeleted": false, + "boundElements": [], + "updated": 1, + "link": null, + "locked": false, + "text": "same inode", + "fontSize": 11, + "fontFamily": 1, + "textAlign": "left", + "verticalAlign": "top", + "containerId": null, + "originalText": "same inode", + "lineHeight": 1.25, + "baseline": 11 + }, + { + "id": "a61765111", + "type": "arrow", + "x": 460, + "y": 561, + "width": 60, + "height": 0, + "angle": 0, + "strokeColor": "#555555", + "backgroundColor": "transparent", + "fillStyle": "solid", + "strokeWidth": 2, + "strokeStyle": "dashed", + "roughness": 1, + "opacity": 100, + "groupIds": [], + "frameId": null, + "roundness": { + "type": 2 + }, + "seed": 1821885162, + "versionNonce": 466144663, + "isDeleted": false, + "boundElements": [], + "updated": 1, + "link": null, + "locked": false, + "points": [ + [ + 0, + 0 + ], + [ + 60, + 0 + ] + ], + "lastCommittedPoint": null, + "startBinding": null, + "endBinding": null, + "startArrowhead": null, + "endArrowhead": "arrow" + }, + { + "id": "t388761624", + "type": "text", + "x": 490.0, + "y": 547.0, + "width": 60.50000000000001, + "height": 13.75, + "angle": 0, + "strokeColor": "#555555", + "backgroundColor": "transparent", + "fillStyle": "solid", + "strokeWidth": 1, + "strokeStyle": "solid", + "roughness": 1, + "opacity": 100, + "groupIds": [], + "frameId": null, + "roundness": null, + "seed": 1543210021, + "versionNonce": 792173145, + "isDeleted": false, + "boundElements": [], + "updated": 1, + "link": null, + "locked": false, + "text": "same inode", + "fontSize": 11, + "fontFamily": 1, + "textAlign": "left", + "verticalAlign": "top", + "containerId": null, + "originalText": "same inode", + "lineHeight": 1.25, + "baseline": 11 + }, + { + "id": "t420796487", + "type": "text", + "x": 30, + "y": 595, + "width": 957.0000000000001, + "height": 15.0, + "angle": 0, + "strokeColor": "#555555", + "backgroundColor": "transparent", + "fillStyle": "solid", + "strokeWidth": 1, + "strokeStyle": "solid", + "roughness": 1, + "opacity": 100, + "groupIds": [], + "frameId": null, + "roundness": null, + "seed": 551779620, + "versionNonce": 1472238603, + "isDeleted": false, + "boundElements": [], + "updated": 1, + "link": null, + "locked": false, + "text": "pmon containerize requests CLONE_NEWNS|CLONE_NEWUTS|CLONE_NEWNET|CLONE_NEWPID only \u2014 amber boxes are namespaces it actually unshares; grey/dashed", + "fontSize": 12, + "fontFamily": 1, + "textAlign": "left", + "verticalAlign": "top", + "containerId": null, + "originalText": "pmon containerize requests CLONE_NEWNS|CLONE_NEWUTS|CLONE_NEWNET|CLONE_NEWPID only \u2014 amber boxes are namespaces it actually unshares; grey/dashed", + "lineHeight": 1.25, + "baseline": 12 + }, + { + "id": "t191977621", + "type": "text", + "x": 30, + "y": 612, + "width": 970.2, + "height": 15.0, + "angle": 0, + "strokeColor": "#555555", + "backgroundColor": "transparent", + "fillStyle": "solid", + "strokeWidth": 1, + "strokeStyle": "solid", + "roughness": 1, + "opacity": 100, + "groupIds": [], + "frameId": null, + "roundness": null, + "seed": 159532815, + "versionNonce": 995481716, + "isDeleted": false, + "boundElements": [], + "updated": 1, + "link": null, + "locked": false, + "text": "boxes are namespaces it deliberately leaves alone (time, IPC, user, cgroup) \u2014 every id above is a real readlink /proc//ns/* on systems-target.", + "fontSize": 12, + "fontFamily": 1, + "textAlign": "left", + "verticalAlign": "top", + "containerId": null, + "originalText": "boxes are namespaces it deliberately leaves alone (time, IPC, user, cgroup) \u2014 every id above is a real readlink /proc//ns/* on systems-target.", + "lineHeight": 1.25, + "baseline": 12 + } + ], + "appState": { + "viewBackgroundColor": "#ffffff", + "gridSize": null + }, + "files": {} +} \ No newline at end of file diff --git a/assets/diagrams/32-namespace-overview.svg b/assets/diagrams/32-namespace-overview.svg new file mode 100644 index 0000000..3579b08 --- /dev/null +++ b/assets/diagrams/32-namespace-overview.svg @@ -0,0 +1,72 @@ + + + +HOST — systems-target, pid 1 (systemd) + +pmon containerize child — new PID-1 task + +PID +pid:[4026531836] + +PID (unshared) +pid:[4026532530] + +Mount +mnt:[4026531832] + +Mount (unshared) +mnt:[4026532528] + +UTS +uts:[4026531838] + +UTS (unshared) +uts:[4026532529] + +Network +net:[4026531833] + +Network (unshared) +net:[4026532531] + +Time +time:[4026531834] + +Time (shared with host) +time:[4026531834] + +IPC +ipc:[4026531839] + +IPC (shared with host) +ipc:[4026531839] + +User / Cgroup +user/cgroup:[4026531837/4026531835] + +User / Cgroup (shared with host) +user/cgroup:[4026531837/4026531835] + +differs +differs + +differs +differs + +differs +differs + +differs +differs + +same inode +same inode + +same inode +same inode + +same inode +same inode +pmon containerize requests CLONE_NEWNS|CLONE_NEWUTS|CLONE_NEWNET|CLONE_NEWPID only — amber boxes are namespaces it actually unshares; grey/dashed +boxes are namespaces it deliberately leaves alone (time, IPC, user, cgroup) — every id above is a real readlink /proc/<pid>/ns/* on systems-target. + \ No newline at end of file diff --git a/assets/diagrams/33-landlock-ruleset.excalidraw b/assets/diagrams/33-landlock-ruleset.excalidraw new file mode 100644 index 0000000..8a42217 --- /dev/null +++ b/assets/diagrams/33-landlock-ruleset.excalidraw @@ -0,0 +1,1469 @@ +{ + "type": "excalidraw", + "version": 2, + "source": "https://excalidraw.com", + "elements": [ + { + "id": "r986015859", + "type": "rectangle", + "x": 20, + "y": 30, + "width": 900, + "height": 230, + "angle": 0, + "strokeColor": "#dddddd", + "backgroundColor": "#eef4fb", + "fillStyle": "solid", + "strokeWidth": 2, + "strokeStyle": "solid", + "roughness": 1, + "opacity": 100, + "groupIds": [], + "frameId": null, + "roundness": { + "type": 3 + }, + "seed": 1740857548, + "versionNonce": 932498535, + "isDeleted": false, + "boundElements": [], + "updated": 1, + "link": null, + "locked": false + }, + { + "id": "t220685604", + "type": "text", + "x": 32, + "y": 36, + "width": 389.40000000000003, + "height": 15.0, + "angle": 0, + "strokeColor": "#555555", + "backgroundColor": "transparent", + "fillStyle": "solid", + "strokeWidth": 1, + "strokeStyle": "solid", + "roughness": 1, + "opacity": 100, + "groupIds": [], + "frameId": null, + "roundness": null, + "seed": 943674156, + "versionNonce": 828699827, + "isDeleted": false, + "boundElements": [], + "updated": 1, + "link": null, + "locked": false, + "text": "building the ruleset (unprivileged \u2014 no root, no VM needed)", + "fontSize": 12, + "fontFamily": 1, + "textAlign": "left", + "verticalAlign": "top", + "containerId": null, + "originalText": "building the ruleset (unprivileged \u2014 no root, no VM needed)", + "lineHeight": 1.25, + "baseline": 12 + }, + { + "id": "r1202485095", + "type": "rectangle", + "x": 20, + "y": 280, + "width": 900, + "height": 170, + "angle": 0, + "strokeColor": "#dddddd", + "backgroundColor": "#fafafa", + "fillStyle": "solid", + "strokeWidth": 2, + "strokeStyle": "solid", + "roughness": 1, + "opacity": 100, + "groupIds": [], + "frameId": null, + "roundness": { + "type": 3 + }, + "seed": 757592862, + "versionNonce": 850745098, + "isDeleted": false, + "boundElements": [], + "updated": 1, + "link": null, + "locked": false + }, + { + "id": "t1651648840", + "type": "text", + "x": 32, + "y": 286, + "width": 534.6, + "height": 15.0, + "angle": 0, + "strokeColor": "#555555", + "backgroundColor": "transparent", + "fillStyle": "solid", + "strokeWidth": 1, + "strokeStyle": "solid", + "roughness": 1, + "opacity": 100, + "groupIds": [], + "frameId": null, + "roundness": null, + "seed": 569894947, + "versionNonce": 1141233007, + "isDeleted": false, + "boundElements": [], + "updated": 1, + "link": null, + "locked": false, + "text": "after landlock_restrict_self() \u2014 enforced against this process + every descendant", + "fontSize": 12, + "fontFamily": 1, + "textAlign": "left", + "verticalAlign": "top", + "containerId": null, + "originalText": "after landlock_restrict_self() \u2014 enforced against this process + every descendant", + "lineHeight": 1.25, + "baseline": 12 + }, + { + "id": "r1541927571", + "type": "rectangle", + "x": 50, + "y": 65, + "width": 400, + "height": 55, + "angle": 0, + "strokeColor": "#2f6db5", + "backgroundColor": "#eef4fb", + "fillStyle": "solid", + "strokeWidth": 2, + "strokeStyle": "solid", + "roughness": 1, + "opacity": 100, + "groupIds": [], + "frameId": null, + "roundness": { + "type": 3 + }, + "seed": 1884995974, + "versionNonce": 1037255618, + "isDeleted": false, + "boundElements": [], + "updated": 1, + "link": null, + "locked": false + }, + { + "id": "t999157363", + "type": "text", + "x": 60, + "y": 73, + "width": 315.70000000000005, + "height": 17.5, + "angle": 0, + "strokeColor": "#111111", + "backgroundColor": "transparent", + "fillStyle": "solid", + "strokeWidth": 1, + "strokeStyle": "solid", + "roughness": 1, + "opacity": 100, + "groupIds": [], + "frameId": null, + "roundness": null, + "seed": 613633743, + "versionNonce": 1174984624, + "isDeleted": false, + "boundElements": [], + "updated": 1, + "link": null, + "locked": false, + "text": "landlock_create_ruleset(NULL, 0, VERSION)", + "fontSize": 14, + "fontFamily": 1, + "textAlign": "left", + "verticalAlign": "top", + "containerId": null, + "originalText": "landlock_create_ruleset(NULL, 0, VERSION)", + "lineHeight": 1.25, + "baseline": 14 + }, + { + "id": "t919842257", + "type": "text", + "x": 60, + "y": 93, + "width": 193.60000000000002, + "height": 13.75, + "angle": 0, + "strokeColor": "#555555", + "backgroundColor": "transparent", + "fillStyle": "solid", + "strokeWidth": 1, + "strokeStyle": "solid", + "roughness": 1, + "opacity": 100, + "groupIds": [], + "frameId": null, + "roundness": null, + "seed": 78478062, + "versionNonce": 510374609, + "isDeleted": false, + "boundElements": [], + "updated": 1, + "link": null, + "locked": false, + "text": "probes the ABI \u2014 never hardcoded", + "fontSize": 11, + "fontFamily": 1, + "textAlign": "left", + "verticalAlign": "top", + "containerId": null, + "originalText": "probes the ABI \u2014 never hardcoded", + "lineHeight": 1.25, + "baseline": 11 + }, + { + "id": "r806370805", + "type": "rectangle", + "x": 500, + "y": 65, + "width": 400, + "height": 55, + "angle": 0, + "strokeColor": "#111111", + "backgroundColor": "#ffffff", + "fillStyle": "solid", + "strokeWidth": 2, + "strokeStyle": "solid", + "roughness": 1, + "opacity": 100, + "groupIds": [], + "frameId": null, + "roundness": { + "type": 3 + }, + "seed": 1062327033, + "versionNonce": 677309677, + "isDeleted": false, + "boundElements": [], + "updated": 1, + "link": null, + "locked": false + }, + { + "id": "t1249758966", + "type": "text", + "x": 510, + "y": 73, + "width": 331.1, + "height": 17.5, + "angle": 0, + "strokeColor": "#111111", + "backgroundColor": "transparent", + "fillStyle": "solid", + "strokeWidth": 1, + "strokeStyle": "solid", + "roughness": 1, + "opacity": 100, + "groupIds": [], + "frameId": null, + "roundness": null, + "seed": 13751825, + "versionNonce": 1392956561, + "isDeleted": false, + "boundElements": [], + "updated": 1, + "link": null, + "locked": false, + "text": "host: ABI=9 \u00b7 systems-target guest: ABI=7", + "fontSize": 14, + "fontFamily": 1, + "textAlign": "left", + "verticalAlign": "top", + "containerId": null, + "originalText": "host: ABI=9 \u00b7 systems-target guest: ABI=7", + "lineHeight": 1.25, + "baseline": 14 + }, + { + "id": "t195189224", + "type": "text", + "x": 510, + "y": 93, + "width": 242.00000000000003, + "height": 13.75, + "angle": 0, + "strokeColor": "#555555", + "backgroundColor": "transparent", + "fillStyle": "solid", + "strokeWidth": 1, + "strokeStyle": "solid", + "roughness": 1, + "opacity": 100, + "groupIds": [], + "frameId": null, + "roundness": null, + "seed": 659333348, + "versionNonce": 500276714, + "isDeleted": false, + "boundElements": [], + "updated": 1, + "link": null, + "locked": false, + "text": "same kernel line, different Landlock ABI", + "fontSize": 11, + "fontFamily": 1, + "textAlign": "left", + "verticalAlign": "top", + "containerId": null, + "originalText": "same kernel line, different Landlock ABI", + "lineHeight": 1.25, + "baseline": 11 + }, + { + "id": "r424912557", + "type": "rectangle", + "x": 50, + "y": 140, + "width": 400, + "height": 55, + "angle": 0, + "strokeColor": "#999999", + "backgroundColor": "#ffffff", + "fillStyle": "solid", + "strokeWidth": 2, + "strokeStyle": "solid", + "roughness": 1, + "opacity": 100, + "groupIds": [], + "frameId": null, + "roundness": { + "type": 3 + }, + "seed": 1524691148, + "versionNonce": 1649439089, + "isDeleted": false, + "boundElements": [], + "updated": 1, + "link": null, + "locked": false + }, + { + "id": "t1912218973", + "type": "text", + "x": 60, + "y": 148, + "width": 223.3, + "height": 17.5, + "angle": 0, + "strokeColor": "#111111", + "backgroundColor": "transparent", + "fillStyle": "solid", + "strokeWidth": 1, + "strokeStyle": "solid", + "roughness": 1, + "opacity": 100, + "groupIds": [], + "frameId": null, + "roundness": null, + "seed": 882857998, + "versionNonce": 1103698761, + "isDeleted": false, + "boundElements": [], + "updated": 1, + "link": null, + "locked": false, + "text": "landlock_create_ruleset(attr)", + "fontSize": 14, + "fontFamily": 1, + "textAlign": "left", + "verticalAlign": "top", + "containerId": null, + "originalText": "landlock_create_ruleset(attr)", + "lineHeight": 1.25, + "baseline": 14 + }, + { + "id": "t126330005", + "type": "text", + "x": 60, + "y": 168, + "width": 229.9, + "height": 13.75, + "angle": 0, + "strokeColor": "#555555", + "backgroundColor": "transparent", + "fillStyle": "solid", + "strokeWidth": 1, + "strokeStyle": "solid", + "roughness": 1, + "opacity": 100, + "groupIds": [], + "frameId": null, + "roundness": null, + "seed": 220907025, + "versionNonce": 338730516, + "isDeleted": false, + "boundElements": [], + "updated": 1, + "link": null, + "locked": false, + "text": "handled_access_fs = READ_FILE|READ_DIR", + "fontSize": 11, + "fontFamily": 1, + "textAlign": "left", + "verticalAlign": "top", + "containerId": null, + "originalText": "handled_access_fs = READ_FILE|READ_DIR", + "lineHeight": 1.25, + "baseline": 11 + }, + { + "id": "r191555498", + "type": "rectangle", + "x": 500, + "y": 140, + "width": 400, + "height": 55, + "angle": 0, + "strokeColor": "#999999", + "backgroundColor": "#ffffff", + "fillStyle": "solid", + "strokeWidth": 2, + "strokeStyle": "solid", + "roughness": 1, + "opacity": 100, + "groupIds": [], + "frameId": null, + "roundness": { + "type": 3 + }, + "seed": 1448337007, + "versionNonce": 1082904968, + "isDeleted": false, + "boundElements": [], + "updated": 1, + "link": null, + "locked": false + }, + { + "id": "t1906191602", + "type": "text", + "x": 510, + "y": 148, + "width": 454.3, + "height": 17.5, + "angle": 0, + "strokeColor": "#111111", + "backgroundColor": "transparent", + "fillStyle": "solid", + "strokeWidth": 1, + "strokeStyle": "solid", + "roughness": 1, + "opacity": 100, + "groupIds": [], + "frameId": null, + "roundness": null, + "seed": 1342779799, + "versionNonce": 1371773517, + "isDeleted": false, + "boundElements": [], + "updated": 1, + "link": null, + "locked": false, + "text": "landlock_add_rule(PATH_BENEATH, parent_fd=open(DIR,O_PATH))", + "fontSize": 14, + "fontFamily": 1, + "textAlign": "left", + "verticalAlign": "top", + "containerId": null, + "originalText": "landlock_add_rule(PATH_BENEATH, parent_fd=open(DIR,O_PATH))", + "lineHeight": 1.25, + "baseline": 14 + }, + { + "id": "t71592625", + "type": "text", + "x": 510, + "y": 168, + "width": 181.50000000000003, + "height": 13.75, + "angle": 0, + "strokeColor": "#555555", + "backgroundColor": "transparent", + "fillStyle": "solid", + "strokeWidth": 1, + "strokeStyle": "solid", + "roughness": 1, + "opacity": 100, + "groupIds": [], + "frameId": null, + "roundness": null, + "seed": 212007186, + "versionNonce": 541692540, + "isDeleted": false, + "boundElements": [], + "updated": 1, + "link": null, + "locked": false, + "text": "binds those rights to DIR only", + "fontSize": 11, + "fontFamily": 1, + "textAlign": "left", + "verticalAlign": "top", + "containerId": null, + "originalText": "binds those rights to DIR only", + "lineHeight": 1.25, + "baseline": 11 + }, + { + "id": "r505794563", + "type": "rectangle", + "x": 50, + "y": 210, + "width": 400, + "height": 40, + "angle": 0, + "strokeColor": "#999999", + "backgroundColor": "#ffffff", + "fillStyle": "solid", + "strokeWidth": 2, + "strokeStyle": "solid", + "roughness": 1, + "opacity": 100, + "groupIds": [], + "frameId": null, + "roundness": { + "type": 3 + }, + "seed": 1495706757, + "versionNonce": 37045717, + "isDeleted": false, + "boundElements": [], + "updated": 1, + "link": null, + "locked": false + }, + { + "id": "t105941385", + "type": "text", + "x": 60, + "y": 218, + "width": 223.3, + "height": 17.5, + "angle": 0, + "strokeColor": "#111111", + "backgroundColor": "transparent", + "fillStyle": "solid", + "strokeWidth": 1, + "strokeStyle": "solid", + "roughness": 1, + "opacity": 100, + "groupIds": [], + "frameId": null, + "roundness": null, + "seed": 1090975565, + "versionNonce": 1972715718, + "isDeleted": false, + "boundElements": [], + "updated": 1, + "link": null, + "locked": false, + "text": "prctl(PR_SET_NO_NEW_PRIVS, 1)", + "fontSize": 14, + "fontFamily": 1, + "textAlign": "left", + "verticalAlign": "top", + "containerId": null, + "originalText": "prctl(PR_SET_NO_NEW_PRIVS, 1)", + "lineHeight": 1.25, + "baseline": 14 + }, + { + "id": "t53970204", + "type": "text", + "x": 60, + "y": 238, + "width": 175.45000000000002, + "height": 13.75, + "angle": 0, + "strokeColor": "#555555", + "backgroundColor": "transparent", + "fillStyle": "solid", + "strokeWidth": 1, + "strokeStyle": "solid", + "roughness": 1, + "opacity": 100, + "groupIds": [], + "frameId": null, + "roundness": null, + "seed": 1736799895, + "versionNonce": 608460526, + "isDeleted": false, + "boundElements": [], + "updated": 1, + "link": null, + "locked": false, + "text": "required before restrict_self", + "fontSize": 11, + "fontFamily": 1, + "textAlign": "left", + "verticalAlign": "top", + "containerId": null, + "originalText": "required before restrict_self", + "lineHeight": 1.25, + "baseline": 11 + }, + { + "id": "r1040406026", + "type": "rectangle", + "x": 500, + "y": 210, + "width": 400, + "height": 40, + "angle": 0, + "strokeColor": "#e8870c", + "backgroundColor": "#fff8ef", + "fillStyle": "solid", + "strokeWidth": 2, + "strokeStyle": "solid", + "roughness": 1, + "opacity": 100, + "groupIds": [], + "frameId": null, + "roundness": { + "type": 3 + }, + "seed": 1093346005, + "versionNonce": 18744642, + "isDeleted": false, + "boundElements": [], + "updated": 1, + "link": null, + "locked": false + }, + { + "id": "t488891178", + "type": "text", + "x": 510, + "y": 218, + "width": 331.1, + "height": 17.5, + "angle": 0, + "strokeColor": "#111111", + "backgroundColor": "transparent", + "fillStyle": "solid", + "strokeWidth": 1, + "strokeStyle": "solid", + "roughness": 1, + "opacity": 100, + "groupIds": [], + "frameId": null, + "roundness": null, + "seed": 657638706, + "versionNonce": 52450755, + "isDeleted": false, + "boundElements": [], + "updated": 1, + "link": null, + "locked": false, + "text": "landlock_restrict_self(ruleset_fd[, TSYNC])", + "fontSize": 14, + "fontFamily": 1, + "textAlign": "left", + "verticalAlign": "top", + "containerId": null, + "originalText": "landlock_restrict_self(ruleset_fd[, TSYNC])", + "lineHeight": 1.25, + "baseline": 14 + }, + { + "id": "t1145034255", + "type": "text", + "x": 510, + "y": 238, + "width": 266.20000000000005, + "height": 13.75, + "angle": 0, + "strokeColor": "#555555", + "backgroundColor": "transparent", + "fillStyle": "solid", + "strokeWidth": 1, + "strokeStyle": "solid", + "roughness": 1, + "opacity": 100, + "groupIds": [], + "frameId": null, + "roundness": null, + "seed": 387704043, + "versionNonce": 625467722, + "isDeleted": false, + "boundElements": [], + "updated": 1, + "link": null, + "locked": false, + "text": "irrevocable; TSYNC covers every Go OS thread", + "fontSize": 11, + "fontFamily": 1, + "textAlign": "left", + "verticalAlign": "top", + "containerId": null, + "originalText": "irrevocable; TSYNC covers every Go OS thread", + "lineHeight": 1.25, + "baseline": 11 + }, + { + "id": "r1518256095", + "type": "rectangle", + "x": 60, + "y": 310, + "width": 380, + "height": 60, + "angle": 0, + "strokeColor": "#e8870c", + "backgroundColor": "#fff8ef", + "fillStyle": "solid", + "strokeWidth": 2, + "strokeStyle": "solid", + "roughness": 1, + "opacity": 100, + "groupIds": [], + "frameId": null, + "roundness": { + "type": 3 + }, + "seed": 430485017, + "versionNonce": 1756171502, + "isDeleted": false, + "boundElements": [], + "updated": 1, + "link": null, + "locked": false + }, + { + "id": "t1895866379", + "type": "text", + "x": 70, + "y": 318, + "width": 246.40000000000003, + "height": 17.5, + "angle": 0, + "strokeColor": "#111111", + "backgroundColor": "transparent", + "fillStyle": "solid", + "strokeWidth": 1, + "strokeStyle": "solid", + "roughness": 1, + "opacity": 100, + "groupIds": [], + "frameId": null, + "roundness": null, + "seed": 1822125665, + "versionNonce": 1024447833, + "isDeleted": false, + "boundElements": [], + "updated": 1, + "link": null, + "locked": false, + "text": "/tmp/lsp33-watch-sandbox (= DIR)", + "fontSize": 14, + "fontFamily": 1, + "textAlign": "left", + "verticalAlign": "top", + "containerId": null, + "originalText": "/tmp/lsp33-watch-sandbox (= DIR)", + "lineHeight": 1.25, + "baseline": 14 + }, + { + "id": "t1980073061", + "type": "text", + "x": 70, + "y": 338, + "width": 326.70000000000005, + "height": 13.75, + "angle": 0, + "strokeColor": "#555555", + "backgroundColor": "transparent", + "fillStyle": "solid", + "strokeWidth": 1, + "strokeStyle": "solid", + "roughness": 1, + "opacity": 100, + "groupIds": [], + "frameId": null, + "roundness": null, + "seed": 1512763698, + "versionNonce": 1586026815, + "isDeleted": false, + "boundElements": [], + "updated": 1, + "link": null, + "locked": false, + "text": "READ_FILE|READ_DIR allowed \u2014 watch loop works normally", + "fontSize": 11, + "fontFamily": 1, + "textAlign": "left", + "verticalAlign": "top", + "containerId": null, + "originalText": "READ_FILE|READ_DIR allowed \u2014 watch loop works normally", + "lineHeight": 1.25, + "baseline": 11 + }, + { + "id": "r799900309", + "type": "rectangle", + "x": 500, + "y": 310, + "width": 380, + "height": 60, + "angle": 0, + "strokeColor": "#999999", + "backgroundColor": "#ffffff", + "fillStyle": "solid", + "strokeWidth": 2, + "strokeStyle": "dashed", + "roughness": 1, + "opacity": 100, + "groupIds": [], + "frameId": null, + "roundness": { + "type": 3 + }, + "seed": 982819049, + "versionNonce": 867694840, + "isDeleted": false, + "boundElements": [], + "updated": 1, + "link": null, + "locked": false + }, + { + "id": "t1005776729", + "type": "text", + "x": 510, + "y": 318, + "width": 277.20000000000005, + "height": 17.5, + "angle": 0, + "strokeColor": "#111111", + "backgroundColor": "transparent", + "fillStyle": "solid", + "strokeWidth": 1, + "strokeStyle": "solid", + "roughness": 1, + "opacity": 100, + "groupIds": [], + "frameId": null, + "roundness": null, + "seed": 836273343, + "versionNonce": 1569470299, + "isDeleted": false, + "boundElements": [], + "updated": 1, + "link": null, + "locked": false, + "text": "/tmp/lsp33-outside-target/secret.txt", + "fontSize": 14, + "fontFamily": 1, + "textAlign": "left", + "verticalAlign": "top", + "containerId": null, + "originalText": "/tmp/lsp33-outside-target/secret.txt", + "lineHeight": 1.25, + "baseline": 14 + }, + { + "id": "t141640565", + "type": "text", + "x": 510, + "y": 338, + "width": 157.3, + "height": 13.75, + "angle": 0, + "strokeColor": "#555555", + "backgroundColor": "transparent", + "fillStyle": "solid", + "strokeWidth": 1, + "strokeStyle": "solid", + "roughness": 1, + "opacity": 100, + "groupIds": [], + "frameId": null, + "roundness": null, + "seed": 1538876218, + "versionNonce": 1485772489, + "isDeleted": false, + "boundElements": [], + "updated": 1, + "link": null, + "locked": false, + "text": "open(O_RDONLY) \u2192 -1 EACCES", + "fontSize": 11, + "fontFamily": 1, + "textAlign": "left", + "verticalAlign": "top", + "containerId": null, + "originalText": "open(O_RDONLY) \u2192 -1 EACCES", + "lineHeight": 1.25, + "baseline": 11 + }, + { + "id": "r1835723398", + "type": "rectangle", + "x": 60, + "y": 390, + "width": 820, + "height": 45, + "angle": 0, + "strokeColor": "#999999", + "backgroundColor": "#ffffff", + "fillStyle": "solid", + "strokeWidth": 2, + "strokeStyle": "solid", + "roughness": 1, + "opacity": 100, + "groupIds": [], + "frameId": null, + "roundness": { + "type": 3 + }, + "seed": 815651457, + "versionNonce": 1626077973, + "isDeleted": false, + "boundElements": [], + "updated": 1, + "link": null, + "locked": false + }, + { + "id": "t1794649746", + "type": "text", + "x": 70, + "y": 398, + "width": 592.9000000000001, + "height": 17.5, + "angle": 0, + "strokeColor": "#111111", + "backgroundColor": "transparent", + "fillStyle": "solid", + "strokeWidth": 1, + "strokeStyle": "solid", + "roughness": 1, + "opacity": 100, + "groupIds": [], + "frameId": null, + "roundness": null, + "seed": 831384969, + "versionNonce": 629151126, + "isDeleted": false, + "boundElements": [], + "updated": 1, + "link": null, + "locked": false, + "text": "handled_access_net left at 0 in this ruleset \u2014 no network mediation requested", + "fontSize": 14, + "fontFamily": 1, + "textAlign": "left", + "verticalAlign": "top", + "containerId": null, + "originalText": "handled_access_net left at 0 in this ruleset \u2014 no network mediation requested", + "lineHeight": 1.25, + "baseline": 14 + }, + { + "id": "t719715710", + "type": "text", + "x": 70, + "y": 418, + "width": 393.25000000000006, + "height": 13.75, + "angle": 0, + "strokeColor": "#555555", + "backgroundColor": "transparent", + "fillStyle": "solid", + "strokeWidth": 1, + "strokeStyle": "solid", + "roughness": 1, + "opacity": 100, + "groupIds": [], + "frameId": null, + "roundness": null, + "seed": 95767015, + "versionNonce": 1825389012, + "isDeleted": false, + "boundElements": [], + "updated": 1, + "link": null, + "locked": false, + "text": "TCP bind/connect rights exist since ABI v4; UDP not until ABI v10", + "fontSize": 11, + "fontFamily": 1, + "textAlign": "left", + "verticalAlign": "top", + "containerId": null, + "originalText": "TCP bind/connect rights exist since ABI v4; UDP not until ABI v10", + "lineHeight": 1.25, + "baseline": 11 + }, + { + "id": "a1470779892", + "type": "arrow", + "x": 450, + "y": 92, + "width": 50, + "height": 0, + "angle": 0, + "strokeColor": "#b8650a", + "backgroundColor": "transparent", + "fillStyle": "solid", + "strokeWidth": 2, + "strokeStyle": "solid", + "roughness": 1, + "opacity": 100, + "groupIds": [], + "frameId": null, + "roundness": { + "type": 2 + }, + "seed": 1810136438, + "versionNonce": 1257655707, + "isDeleted": false, + "boundElements": [], + "updated": 1, + "link": null, + "locked": false, + "points": [ + [ + 0, + 0 + ], + [ + 50, + 0 + ] + ], + "lastCommittedPoint": null, + "startBinding": null, + "endBinding": null, + "startArrowhead": null, + "endArrowhead": "arrow" + }, + { + "id": "a490221154", + "type": "arrow", + "x": 250, + "y": 120, + "width": 0, + "height": 20, + "angle": 0, + "strokeColor": "#555555", + "backgroundColor": "transparent", + "fillStyle": "solid", + "strokeWidth": 2, + "strokeStyle": "solid", + "roughness": 1, + "opacity": 100, + "groupIds": [], + "frameId": null, + "roundness": { + "type": 2 + }, + "seed": 1351426538, + "versionNonce": 1269401149, + "isDeleted": false, + "boundElements": [], + "updated": 1, + "link": null, + "locked": false, + "points": [ + [ + 0, + 0 + ], + [ + 0, + 20 + ] + ], + "lastCommittedPoint": null, + "startBinding": null, + "endBinding": null, + "startArrowhead": null, + "endArrowhead": "arrow" + }, + { + "id": "a1914858604", + "type": "arrow", + "x": 700, + "y": 120, + "width": 0, + "height": 20, + "angle": 0, + "strokeColor": "#555555", + "backgroundColor": "transparent", + "fillStyle": "solid", + "strokeWidth": 2, + "strokeStyle": "solid", + "roughness": 1, + "opacity": 100, + "groupIds": [], + "frameId": null, + "roundness": { + "type": 2 + }, + "seed": 40880852, + "versionNonce": 350299181, + "isDeleted": false, + "boundElements": [], + "updated": 1, + "link": null, + "locked": false, + "points": [ + [ + 0, + 0 + ], + [ + 0, + 20 + ] + ], + "lastCommittedPoint": null, + "startBinding": null, + "endBinding": null, + "startArrowhead": null, + "endArrowhead": "arrow" + }, + { + "id": "a1158280993", + "type": "arrow", + "x": 250, + "y": 195, + "width": 0, + "height": 15, + "angle": 0, + "strokeColor": "#555555", + "backgroundColor": "transparent", + "fillStyle": "solid", + "strokeWidth": 2, + "strokeStyle": "solid", + "roughness": 1, + "opacity": 100, + "groupIds": [], + "frameId": null, + "roundness": { + "type": 2 + }, + "seed": 1590480164, + "versionNonce": 1631755657, + "isDeleted": false, + "boundElements": [], + "updated": 1, + "link": null, + "locked": false, + "points": [ + [ + 0, + 0 + ], + [ + 0, + 15 + ] + ], + "lastCommittedPoint": null, + "startBinding": null, + "endBinding": null, + "startArrowhead": null, + "endArrowhead": "arrow" + }, + { + "id": "a156154182", + "type": "arrow", + "x": 700, + "y": 195, + "width": 0, + "height": 15, + "angle": 0, + "strokeColor": "#555555", + "backgroundColor": "transparent", + "fillStyle": "solid", + "strokeWidth": 2, + "strokeStyle": "solid", + "roughness": 1, + "opacity": 100, + "groupIds": [], + "frameId": null, + "roundness": { + "type": 2 + }, + "seed": 1380307746, + "versionNonce": 1218071576, + "isDeleted": false, + "boundElements": [], + "updated": 1, + "link": null, + "locked": false, + "points": [ + [ + 0, + 0 + ], + [ + 0, + 15 + ] + ], + "lastCommittedPoint": null, + "startBinding": null, + "endBinding": null, + "startArrowhead": null, + "endArrowhead": "arrow" + }, + { + "id": "a870723866", + "type": "arrow", + "x": 450, + "y": 230, + "width": 50, + "height": 0, + "angle": 0, + "strokeColor": "#b8650a", + "backgroundColor": "transparent", + "fillStyle": "solid", + "strokeWidth": 2, + "strokeStyle": "solid", + "roughness": 1, + "opacity": 100, + "groupIds": [], + "frameId": null, + "roundness": { + "type": 2 + }, + "seed": 781257211, + "versionNonce": 215558227, + "isDeleted": false, + "boundElements": [], + "updated": 1, + "link": null, + "locked": false, + "points": [ + [ + 0, + 0 + ], + [ + 50, + 0 + ] + ], + "lastCommittedPoint": null, + "startBinding": null, + "endBinding": null, + "startArrowhead": null, + "endArrowhead": "arrow" + }, + { + "id": "a310060078", + "type": "arrow", + "x": 250, + "y": 250, + "width": 0, + "height": 60, + "angle": 0, + "strokeColor": "#555555", + "backgroundColor": "transparent", + "fillStyle": "solid", + "strokeWidth": 2, + "strokeStyle": "solid", + "roughness": 1, + "opacity": 100, + "groupIds": [], + "frameId": null, + "roundness": { + "type": 2 + }, + "seed": 1650601292, + "versionNonce": 1581337696, + "isDeleted": false, + "boundElements": [], + "updated": 1, + "link": null, + "locked": false, + "points": [ + [ + 0, + 0 + ], + [ + 0, + 60 + ] + ], + "lastCommittedPoint": null, + "startBinding": null, + "endBinding": null, + "startArrowhead": null, + "endArrowhead": "arrow" + }, + { + "id": "t1966593519", + "type": "text", + "x": 250.0, + "y": 266.0, + "width": 40, + "height": 13.75, + "angle": 0, + "strokeColor": "#555555", + "backgroundColor": "transparent", + "fillStyle": "solid", + "strokeWidth": 1, + "strokeStyle": "solid", + "roughness": 1, + "opacity": 100, + "groupIds": [], + "frameId": null, + "roundness": null, + "seed": 1183696325, + "versionNonce": 152196402, + "isDeleted": false, + "boundElements": [], + "updated": 1, + "link": null, + "locked": false, + "text": "grant", + "fontSize": 11, + "fontFamily": 1, + "textAlign": "left", + "verticalAlign": "top", + "containerId": null, + "originalText": "grant", + "lineHeight": 1.25, + "baseline": 11 + }, + { + "id": "a674171145", + "type": "arrow", + "x": 700, + "y": 250, + "width": 0, + "height": 60, + "angle": 0, + "strokeColor": "#555555", + "backgroundColor": "transparent", + "fillStyle": "solid", + "strokeWidth": 2, + "strokeStyle": "dashed", + "roughness": 1, + "opacity": 100, + "groupIds": [], + "frameId": null, + "roundness": { + "type": 2 + }, + "seed": 1101936118, + "versionNonce": 682077226, + "isDeleted": false, + "boundElements": [], + "updated": 1, + "link": null, + "locked": false, + "points": [ + [ + 0, + 0 + ], + [ + 0, + 60 + ] + ], + "lastCommittedPoint": null, + "startBinding": null, + "endBinding": null, + "startArrowhead": null, + "endArrowhead": "arrow" + }, + { + "id": "t98241283", + "type": "text", + "x": 700.0, + "y": 266.0, + "width": 40, + "height": 13.75, + "angle": 0, + "strokeColor": "#555555", + "backgroundColor": "transparent", + "fillStyle": "solid", + "strokeWidth": 1, + "strokeStyle": "solid", + "roughness": 1, + "opacity": 100, + "groupIds": [], + "frameId": null, + "roundness": null, + "seed": 1489145817, + "versionNonce": 1496996170, + "isDeleted": false, + "boundElements": [], + "updated": 1, + "link": null, + "locked": false, + "text": "deny", + "fontSize": 11, + "fontFamily": 1, + "textAlign": "left", + "verticalAlign": "top", + "containerId": null, + "originalText": "deny", + "lineHeight": 1.25, + "baseline": 11 + }, + { + "id": "t369402072", + "type": "text", + "x": 470, + "y": 10, + "width": 417.45000000000005, + "height": 13.75, + "angle": 0, + "strokeColor": "#1d1d1d", + "backgroundColor": "transparent", + "fillStyle": "solid", + "strokeWidth": 1, + "strokeStyle": "solid", + "roughness": 1, + "opacity": 100, + "groupIds": [], + "frameId": null, + "roundness": null, + "seed": 1922468035, + "versionNonce": 184685670, + "isDeleted": false, + "boundElements": [], + "updated": 1, + "link": null, + "locked": false, + "text": "Figure 33.2 \u2014 the Landlock ruleset/rule model, ABI-probed not assumed", + "fontSize": 11, + "fontFamily": 1, + "textAlign": "left", + "verticalAlign": "top", + "containerId": null, + "originalText": "Figure 33.2 \u2014 the Landlock ruleset/rule model, ABI-probed not assumed", + "lineHeight": 1.25, + "baseline": 11 + } + ], + "appState": { + "viewBackgroundColor": "#ffffff", + "gridSize": null + }, + "files": {} +} \ No newline at end of file diff --git a/assets/diagrams/33-landlock-ruleset.svg b/assets/diagrams/33-landlock-ruleset.svg new file mode 100644 index 0000000..2bae41d --- /dev/null +++ b/assets/diagrams/33-landlock-ruleset.svg @@ -0,0 +1,47 @@ + + + +building the ruleset (unprivileged — no root, no VM needed) + +after landlock_restrict_self() — enforced against this process + every descendant + +landlock_create_ruleset(NULL, 0, VERSION) +probes the ABI — never hardcoded + +host: ABI=9 · systems-target guest: ABI=7 +same kernel line, different Landlock ABI + +landlock_create_ruleset(attr) +handled_access_fs = READ_FILE|READ_DIR + +landlock_add_rule(PATH_BENEATH, parent_fd=open(DIR,O_PATH)) +binds those rights to DIR only + +prctl(PR_SET_NO_NEW_PRIVS, 1) +required before restrict_self + +landlock_restrict_self(ruleset_fd[, TSYNC]) +irrevocable; TSYNC covers every Go OS thread + +/tmp/lsp33-watch-sandbox (= DIR) +READ_FILE|READ_DIR allowed — watch loop works normally + +/tmp/lsp33-outside-target/secret.txt +open(O_RDONLY) → -1 EACCES + +handled_access_net left at 0 in this ruleset — no network mediation requested +TCP bind/connect rights exist since ABI v4; UDP not until ABI v10 + + + + + + + +grant +grant + +deny +deny +Figure 33.2 — the Landlock ruleset/rule model, ABI-probed not assumed + \ No newline at end of file diff --git a/assets/diagrams/33-seccomp-filter-flow.excalidraw b/assets/diagrams/33-seccomp-filter-flow.excalidraw new file mode 100644 index 0000000..9727b29 --- /dev/null +++ b/assets/diagrams/33-seccomp-filter-flow.excalidraw @@ -0,0 +1,1538 @@ +{ + "type": "excalidraw", + "version": 2, + "source": "https://excalidraw.com", + "elements": [ + { + "id": "r1330882817", + "type": "rectangle", + "x": 20, + "y": 30, + "width": 900, + "height": 90, + "angle": 0, + "strokeColor": "#dddddd", + "backgroundColor": "#eef4fb", + "fillStyle": "solid", + "strokeWidth": 2, + "strokeStyle": "solid", + "roughness": 1, + "opacity": 100, + "groupIds": [], + "frameId": null, + "roundness": { + "type": 3 + }, + "seed": 266637735, + "versionNonce": 1345446178, + "isDeleted": false, + "boundElements": [], + "updated": 1, + "link": null, + "locked": false + }, + { + "id": "t769905287", + "type": "text", + "x": 32, + "y": 36, + "width": 184.8, + "height": 15.0, + "angle": 0, + "strokeColor": "#555555", + "backgroundColor": "transparent", + "fillStyle": "solid", + "strokeWidth": 1, + "strokeStyle": "solid", + "roughness": 1, + "opacity": 100, + "groupIds": [], + "frameId": null, + "roundness": null, + "seed": 1144639544, + "versionNonce": 1600862455, + "isDeleted": false, + "boundElements": [], + "updated": 1, + "link": null, + "locked": false, + "text": "userspace: fwatch watch loop", + "fontSize": 12, + "fontFamily": 1, + "textAlign": "left", + "verticalAlign": "top", + "containerId": null, + "originalText": "userspace: fwatch watch loop", + "lineHeight": 1.25, + "baseline": 12 + }, + { + "id": "r1046748495", + "type": "rectangle", + "x": 20, + "y": 140, + "width": 900, + "height": 200, + "angle": 0, + "strokeColor": "#dddddd", + "backgroundColor": "#fafafa", + "fillStyle": "solid", + "strokeWidth": 2, + "strokeStyle": "solid", + "roughness": 1, + "opacity": 100, + "groupIds": [], + "frameId": null, + "roundness": { + "type": 3 + }, + "seed": 1181621924, + "versionNonce": 1685078501, + "isDeleted": false, + "boundElements": [], + "updated": 1, + "link": null, + "locked": false + }, + { + "id": "t930087594", + "type": "text", + "x": 32, + "y": 146, + "width": 521.4000000000001, + "height": 15.0, + "angle": 0, + "strokeColor": "#555555", + "backgroundColor": "transparent", + "fillStyle": "solid", + "strokeWidth": 1, + "strokeStyle": "solid", + "roughness": 1, + "opacity": 100, + "groupIds": [], + "frameId": null, + "roundness": null, + "seed": 1424524508, + "versionNonce": 1008606236, + "isDeleted": false, + "boundElements": [], + "updated": 1, + "link": null, + "locked": false, + "text": "kernel: seccomp-bpf filter \u2014 one cBPF program, RUN AGAIN on every syscall entry", + "fontSize": 12, + "fontFamily": 1, + "textAlign": "left", + "verticalAlign": "top", + "containerId": null, + "originalText": "kernel: seccomp-bpf filter \u2014 one cBPF program, RUN AGAIN on every syscall entry", + "lineHeight": 1.25, + "baseline": 12 + }, + { + "id": "r149483224", + "type": "rectangle", + "x": 50, + "y": 65, + "width": 380, + "height": 45, + "angle": 0, + "strokeColor": "#111111", + "backgroundColor": "#ffffff", + "fillStyle": "solid", + "strokeWidth": 2, + "strokeStyle": "solid", + "roughness": 1, + "opacity": 100, + "groupIds": [], + "frameId": null, + "roundness": { + "type": 3 + }, + "seed": 1425035588, + "versionNonce": 842396018, + "isDeleted": false, + "boundElements": [], + "updated": 1, + "link": null, + "locked": false + }, + { + "id": "t1063521969", + "type": "text", + "x": 60, + "y": 73, + "width": 130.9, + "height": 17.5, + "angle": 0, + "strokeColor": "#111111", + "backgroundColor": "transparent", + "fillStyle": "solid", + "strokeWidth": 1, + "strokeStyle": "solid", + "roughness": 1, + "opacity": 100, + "groupIds": [], + "frameId": null, + "roundness": null, + "seed": 1497581030, + "versionNonce": 1607144463, + "isDeleted": false, + "boundElements": [], + "updated": 1, + "link": null, + "locked": false, + "text": "syscall(2) issued", + "fontSize": 14, + "fontFamily": 1, + "textAlign": "left", + "verticalAlign": "top", + "containerId": null, + "originalText": "syscall(2) issued", + "lineHeight": 1.25, + "baseline": 14 + }, + { + "id": "t1405177853", + "type": "text", + "x": 60, + "y": 93, + "width": 175.45000000000002, + "height": 13.75, + "angle": 0, + "strokeColor": "#555555", + "backgroundColor": "transparent", + "fillStyle": "solid", + "strokeWidth": 1, + "strokeStyle": "solid", + "roughness": 1, + "opacity": 100, + "groupIds": [], + "frameId": null, + "roundness": null, + "seed": 1816510846, + "versionNonce": 660980236, + "isDeleted": false, + "boundElements": [], + "updated": 1, + "link": null, + "locked": false, + "text": "e.g. epoll_wait, read, socket", + "fontSize": 11, + "fontFamily": 1, + "textAlign": "left", + "verticalAlign": "top", + "containerId": null, + "originalText": "e.g. epoll_wait, read, socket", + "lineHeight": 1.25, + "baseline": 11 + }, + { + "id": "r334666300", + "type": "rectangle", + "x": 500, + "y": 65, + "width": 400, + "height": 45, + "angle": 0, + "strokeColor": "#2f6db5", + "backgroundColor": "#eef4fb", + "fillStyle": "solid", + "strokeWidth": 2, + "strokeStyle": "solid", + "roughness": 1, + "opacity": 100, + "groupIds": [], + "frameId": null, + "roundness": { + "type": 3 + }, + "seed": 1084598553, + "versionNonce": 608036561, + "isDeleted": false, + "boundElements": [], + "updated": 1, + "link": null, + "locked": false + }, + { + "id": "t278434415", + "type": "text", + "x": 510, + "y": 73, + "width": 200.20000000000002, + "height": 17.5, + "angle": 0, + "strokeColor": "#111111", + "backgroundColor": "transparent", + "fillStyle": "solid", + "strokeWidth": 1, + "strokeStyle": "solid", + "roughness": 1, + "opacity": 100, + "groupIds": [], + "frameId": null, + "roundness": null, + "seed": 134347611, + "versionNonce": 970125105, + "isDeleted": false, + "boundElements": [], + "updated": 1, + "link": null, + "locked": false, + "text": "this IS classic BPF (cBPF)", + "fontSize": 14, + "fontFamily": 1, + "textAlign": "left", + "verticalAlign": "top", + "containerId": null, + "originalText": "this IS classic BPF (cBPF)", + "lineHeight": 1.25, + "baseline": 14 + }, + { + "id": "t1737506054", + "type": "text", + "x": 510, + "y": 93, + "width": 381.15000000000003, + "height": 13.75, + "angle": 0, + "strokeColor": "#555555", + "backgroundColor": "transparent", + "fillStyle": "solid", + "strokeWidth": 1, + "strokeStyle": "solid", + "roughness": 1, + "opacity": 100, + "groupIds": [], + "frameId": null, + "roundness": null, + "seed": 1551434788, + "versionNonce": 403983532, + "isDeleted": false, + "boundElements": [], + "updated": 1, + "link": null, + "locked": false, + "text": "you write the filter \u2014 Part 8's bpftrace observes; this decides", + "fontSize": 11, + "fontFamily": 1, + "textAlign": "left", + "verticalAlign": "top", + "containerId": null, + "originalText": "you write the filter \u2014 Part 8's bpftrace observes; this decides", + "lineHeight": 1.25, + "baseline": 11 + }, + { + "id": "r729630275", + "type": "rectangle", + "x": 50, + "y": 175, + "width": 230, + "height": 55, + "angle": 0, + "strokeColor": "#999999", + "backgroundColor": "#ffffff", + "fillStyle": "solid", + "strokeWidth": 2, + "strokeStyle": "solid", + "roughness": 1, + "opacity": 100, + "groupIds": [], + "frameId": null, + "roundness": { + "type": 3 + }, + "seed": 335263203, + "versionNonce": 951397104, + "isDeleted": false, + "boundElements": [], + "updated": 1, + "link": null, + "locked": false + }, + { + "id": "t73661649", + "type": "text", + "x": 60, + "y": 183, + "width": 154.0, + "height": 17.5, + "angle": 0, + "strokeColor": "#111111", + "backgroundColor": "transparent", + "fillStyle": "solid", + "strokeWidth": 1, + "strokeStyle": "solid", + "roughness": 1, + "opacity": 100, + "groupIds": [], + "frameId": null, + "roundness": null, + "seed": 1931350521, + "versionNonce": 308267055, + "isDeleted": false, + "boundElements": [], + "updated": 1, + "link": null, + "locked": false, + "text": "BPF_LD|BPF_W|BPF_ABS", + "fontSize": 14, + "fontFamily": 1, + "textAlign": "left", + "verticalAlign": "top", + "containerId": null, + "originalText": "BPF_LD|BPF_W|BPF_ABS", + "lineHeight": 1.25, + "baseline": 14 + }, + { + "id": "t665368463", + "type": "text", + "x": 60, + "y": 203, + "width": 127.05000000000001, + "height": 13.75, + "angle": 0, + "strokeColor": "#555555", + "backgroundColor": "transparent", + "fillStyle": "solid", + "strokeWidth": 1, + "strokeStyle": "solid", + "roughness": 1, + "opacity": 100, + "groupIds": [], + "frameId": null, + "roundness": null, + "seed": 1956871770, + "versionNonce": 679845973, + "isDeleted": false, + "boundElements": [], + "updated": 1, + "link": null, + "locked": false, + "text": "load nr + arch offset", + "fontSize": 11, + "fontFamily": 1, + "textAlign": "left", + "verticalAlign": "top", + "containerId": null, + "originalText": "load nr + arch offset", + "lineHeight": 1.25, + "baseline": 11 + }, + { + "id": "r1160515681", + "type": "rectangle", + "x": 310, + "y": 175, + "width": 300, + "height": 55, + "angle": 0, + "strokeColor": "#e8870c", + "backgroundColor": "#fff8ef", + "fillStyle": "solid", + "strokeWidth": 2, + "strokeStyle": "solid", + "roughness": 1, + "opacity": 100, + "groupIds": [], + "frameId": null, + "roundness": { + "type": 3 + }, + "seed": 1693002477, + "versionNonce": 261475217, + "isDeleted": false, + "boundElements": [], + "updated": 1, + "link": null, + "locked": false + }, + { + "id": "t470260725", + "type": "text", + "x": 320, + "y": 183, + "width": 284.90000000000003, + "height": 17.5, + "angle": 0, + "strokeColor": "#111111", + "backgroundColor": "transparent", + "fillStyle": "solid", + "strokeWidth": 1, + "strokeStyle": "solid", + "roughness": 1, + "opacity": 100, + "groupIds": [], + "frameId": null, + "roundness": null, + "seed": 1008259129, + "versionNonce": 1323230075, + "isDeleted": false, + "boundElements": [], + "updated": 1, + "link": null, + "locked": false, + "text": "jump table: nr == read? epoll_wait? \u2026", + "fontSize": 14, + "fontFamily": 1, + "textAlign": "left", + "verticalAlign": "top", + "containerId": null, + "originalText": "jump table: nr == read? epoll_wait? \u2026", + "lineHeight": 1.25, + "baseline": 14 + }, + { + "id": "t781131265", + "type": "text", + "x": 320, + "y": 203, + "width": 187.55, + "height": 13.75, + "angle": 0, + "strokeColor": "#555555", + "backgroundColor": "transparent", + "fillStyle": "solid", + "strokeWidth": 1, + "strokeStyle": "solid", + "roughness": 1, + "opacity": 100, + "groupIds": [], + "frameId": null, + "roundness": null, + "seed": 373672168, + "versionNonce": 175241129, + "isDeleted": false, + "boundElements": [], + "updated": 1, + "link": null, + "locked": false, + "text": "26 entries (cpp/rust) \u00b7 39 (go)", + "fontSize": 11, + "fontFamily": 1, + "textAlign": "left", + "verticalAlign": "top", + "containerId": null, + "originalText": "26 entries (cpp/rust) \u00b7 39 (go)", + "lineHeight": 1.25, + "baseline": 11 + }, + { + "id": "r1002274835", + "type": "rectangle", + "x": 650, + "y": 175, + "width": 240, + "height": 55, + "angle": 0, + "strokeColor": "#111111", + "backgroundColor": "#ffffff", + "fillStyle": "solid", + "strokeWidth": 2, + "strokeStyle": "solid", + "roughness": 1, + "opacity": 100, + "groupIds": [], + "frameId": null, + "roundness": { + "type": 3 + }, + "seed": 656938478, + "versionNonce": 398198723, + "isDeleted": false, + "boundElements": [], + "updated": 1, + "link": null, + "locked": false + }, + { + "id": "t1987699094", + "type": "text", + "x": 660, + "y": 183, + "width": 192.50000000000003, + "height": 17.5, + "angle": 0, + "strokeColor": "#111111", + "backgroundColor": "transparent", + "fillStyle": "solid", + "strokeWidth": 1, + "strokeStyle": "solid", + "roughness": 1, + "opacity": 100, + "groupIds": [], + "frameId": null, + "roundness": null, + "seed": 137041023, + "versionNonce": 643436042, + "isDeleted": false, + "boundElements": [], + "updated": 1, + "link": null, + "locked": false, + "text": "match \u2192 SECCOMP_RET_ALLOW", + "fontSize": 14, + "fontFamily": 1, + "textAlign": "left", + "verticalAlign": "top", + "containerId": null, + "originalText": "match \u2192 SECCOMP_RET_ALLOW", + "lineHeight": 1.25, + "baseline": 14 + }, + { + "id": "t493579600", + "type": "text", + "x": 660, + "y": 203, + "width": 127.05000000000001, + "height": 13.75, + "angle": 0, + "strokeColor": "#555555", + "backgroundColor": "transparent", + "fillStyle": "solid", + "strokeWidth": 1, + "strokeStyle": "solid", + "roughness": 1, + "opacity": 100, + "groupIds": [], + "frameId": null, + "roundness": null, + "seed": 1474808769, + "versionNonce": 1115894009, + "isDeleted": false, + "boundElements": [], + "updated": 1, + "link": null, + "locked": false, + "text": "syscall actually runs", + "fontSize": 11, + "fontFamily": 1, + "textAlign": "left", + "verticalAlign": "top", + "containerId": null, + "originalText": "syscall actually runs", + "lineHeight": 1.25, + "baseline": 11 + }, + { + "id": "r171764142", + "type": "rectangle", + "x": 310, + "y": 260, + "width": 300, + "height": 50, + "angle": 0, + "strokeColor": "#999999", + "backgroundColor": "#ffffff", + "fillStyle": "solid", + "strokeWidth": 2, + "strokeStyle": "solid", + "roughness": 1, + "opacity": 100, + "groupIds": [], + "frameId": null, + "roundness": { + "type": 3 + }, + "seed": 352840307, + "versionNonce": 705586860, + "isDeleted": false, + "boundElements": [], + "updated": 1, + "link": null, + "locked": false + }, + { + "id": "t1211786392", + "type": "text", + "x": 320, + "y": 268, + "width": 192.50000000000003, + "height": 17.5, + "angle": 0, + "strokeColor": "#111111", + "backgroundColor": "transparent", + "fillStyle": "solid", + "strokeWidth": 1, + "strokeStyle": "solid", + "roughness": 1, + "opacity": 100, + "groupIds": [], + "frameId": null, + "roundness": null, + "seed": 965113195, + "versionNonce": 1918390893, + "isDeleted": false, + "boundElements": [], + "updated": 1, + "link": null, + "locked": false, + "text": "no match \u2192 default action", + "fontSize": 14, + "fontFamily": 1, + "textAlign": "left", + "verticalAlign": "top", + "containerId": null, + "originalText": "no match \u2192 default action", + "lineHeight": 1.25, + "baseline": 14 + }, + { + "id": "t1550783002", + "type": "text", + "x": 320, + "y": 288, + "width": 163.35000000000002, + "height": 13.75, + "angle": 0, + "strokeColor": "#555555", + "backgroundColor": "transparent", + "fillStyle": "solid", + "strokeWidth": 1, + "strokeStyle": "solid", + "roughness": 1, + "opacity": 100, + "groupIds": [], + "frameId": null, + "roundness": null, + "seed": 1430874561, + "versionNonce": 1926837898, + "isDeleted": false, + "boundElements": [], + "updated": 1, + "link": null, + "locked": false, + "text": "(seccomp_init's action arg)", + "fontSize": 11, + "fontFamily": 1, + "textAlign": "left", + "verticalAlign": "top", + "containerId": null, + "originalText": "(seccomp_init's action arg)", + "lineHeight": 1.25, + "baseline": 11 + }, + { + "id": "r881655752", + "type": "rectangle", + "x": 60, + "y": 340, + "width": 380, + "height": 55, + "angle": 0, + "strokeColor": "#2f6db5", + "backgroundColor": "#eef4fb", + "fillStyle": "solid", + "strokeWidth": 2, + "strokeStyle": "solid", + "roughness": 1, + "opacity": 100, + "groupIds": [], + "frameId": null, + "roundness": { + "type": 3 + }, + "seed": 1758569515, + "versionNonce": 1959485630, + "isDeleted": false, + "boundElements": [], + "updated": 1, + "link": null, + "locked": false + }, + { + "id": "t1704721819", + "type": "text", + "x": 70, + "y": 348, + "width": 300.3, + "height": 17.5, + "angle": 0, + "strokeColor": "#111111", + "backgroundColor": "transparent", + "fillStyle": "solid", + "strokeWidth": 1, + "strokeStyle": "solid", + "roughness": 1, + "opacity": 100, + "groupIds": [], + "frameId": null, + "roundness": null, + "seed": 1679786817, + "versionNonce": 1911282390, + "isDeleted": false, + "boundElements": [], + "updated": 1, + "link": null, + "locked": false, + "text": "SCMP_ACT_ERRNO(EPERM) \u2014 fwatch's choice", + "fontSize": 14, + "fontFamily": 1, + "textAlign": "left", + "verticalAlign": "top", + "containerId": null, + "originalText": "SCMP_ACT_ERRNO(EPERM) \u2014 fwatch's choice", + "lineHeight": 1.25, + "baseline": 14 + }, + { + "id": "t496862780", + "type": "text", + "x": 70, + "y": 368, + "width": 260.15000000000003, + "height": 13.75, + "angle": 0, + "strokeColor": "#555555", + "backgroundColor": "transparent", + "fillStyle": "solid", + "strokeWidth": 1, + "strokeStyle": "solid", + "roughness": 1, + "opacity": 100, + "groupIds": [], + "frameId": null, + "roundness": null, + "seed": 1377432089, + "versionNonce": 650806404, + "isDeleted": false, + "boundElements": [], + "updated": 1, + "link": null, + "locked": false, + "text": "socket(2) = -1 EPERM; process keeps running", + "fontSize": 11, + "fontFamily": 1, + "textAlign": "left", + "verticalAlign": "top", + "containerId": null, + "originalText": "socket(2) = -1 EPERM; process keeps running", + "lineHeight": 1.25, + "baseline": 11 + }, + { + "id": "r632191574", + "type": "rectangle", + "x": 500, + "y": 340, + "width": 380, + "height": 55, + "angle": 0, + "strokeColor": "#999999", + "backgroundColor": "#ffffff", + "fillStyle": "solid", + "strokeWidth": 2, + "strokeStyle": "dashed", + "roughness": 1, + "opacity": 100, + "groupIds": [], + "frameId": null, + "roundness": { + "type": 3 + }, + "seed": 357863112, + "versionNonce": 631049171, + "isDeleted": false, + "boundElements": [], + "updated": 1, + "link": null, + "locked": false + }, + { + "id": "t1310100722", + "type": "text", + "x": 510, + "y": 348, + "width": 323.40000000000003, + "height": 17.5, + "angle": 0, + "strokeColor": "#111111", + "backgroundColor": "transparent", + "fillStyle": "solid", + "strokeWidth": 1, + "strokeStyle": "solid", + "roughness": 1, + "opacity": 100, + "groupIds": [], + "frameId": null, + "roundness": null, + "seed": 1306494300, + "versionNonce": 857354916, + "isDeleted": false, + "boundElements": [], + "updated": 1, + "link": null, + "locked": false, + "text": "SCMP_ACT_KILL_PROCESS \u2014 not used by fwatch", + "fontSize": 14, + "fontFamily": 1, + "textAlign": "left", + "verticalAlign": "top", + "containerId": null, + "originalText": "SCMP_ACT_KILL_PROCESS \u2014 not used by fwatch", + "lineHeight": 1.25, + "baseline": 14 + }, + { + "id": "t1395788909", + "type": "text", + "x": 510, + "y": 368, + "width": 254.10000000000002, + "height": 13.75, + "angle": 0, + "strokeColor": "#555555", + "backgroundColor": "transparent", + "fillStyle": "solid", + "strokeWidth": 1, + "strokeStyle": "solid", + "roughness": 1, + "opacity": 100, + "groupIds": [], + "frameId": null, + "roundness": null, + "seed": 264380677, + "versionNonce": 154809065, + "isDeleted": false, + "boundElements": [], + "updated": 1, + "link": null, + "locked": false, + "text": "getpid() denied \u2192 +++ killed by SIGSYS +++", + "fontSize": 11, + "fontFamily": 1, + "textAlign": "left", + "verticalAlign": "top", + "containerId": null, + "originalText": "getpid() denied \u2192 +++ killed by SIGSYS +++", + "lineHeight": 1.25, + "baseline": 11 + }, + { + "id": "a1561052323", + "type": "arrow", + "x": 240, + "y": 110, + "width": 0, + "height": 65, + "angle": 0, + "strokeColor": "#b8650a", + "backgroundColor": "transparent", + "fillStyle": "solid", + "strokeWidth": 2, + "strokeStyle": "solid", + "roughness": 1, + "opacity": 100, + "groupIds": [], + "frameId": null, + "roundness": { + "type": 2 + }, + "seed": 396662529, + "versionNonce": 830126169, + "isDeleted": false, + "boundElements": [], + "updated": 1, + "link": null, + "locked": false, + "points": [ + [ + 0, + 0 + ], + [ + 0, + 65 + ] + ], + "lastCommittedPoint": null, + "startBinding": null, + "endBinding": null, + "startArrowhead": null, + "endArrowhead": "arrow" + }, + { + "id": "t712627593", + "type": "text", + "x": 240.0, + "y": 128.5, + "width": 102.85000000000001, + "height": 13.75, + "angle": 0, + "strokeColor": "#b8650a", + "backgroundColor": "transparent", + "fillStyle": "solid", + "strokeWidth": 1, + "strokeStyle": "solid", + "roughness": 1, + "opacity": 100, + "groupIds": [], + "frameId": null, + "roundness": null, + "seed": 618120498, + "versionNonce": 1829174288, + "isDeleted": false, + "boundElements": [], + "updated": 1, + "link": null, + "locked": false, + "text": "traps into kernel", + "fontSize": 11, + "fontFamily": 1, + "textAlign": "left", + "verticalAlign": "top", + "containerId": null, + "originalText": "traps into kernel", + "lineHeight": 1.25, + "baseline": 11 + }, + { + "id": "a1640790065", + "type": "arrow", + "x": 280, + "y": 202, + "width": 30, + "height": 0, + "angle": 0, + "strokeColor": "#555555", + "backgroundColor": "transparent", + "fillStyle": "solid", + "strokeWidth": 2, + "strokeStyle": "solid", + "roughness": 1, + "opacity": 100, + "groupIds": [], + "frameId": null, + "roundness": { + "type": 2 + }, + "seed": 1355331388, + "versionNonce": 236071183, + "isDeleted": false, + "boundElements": [], + "updated": 1, + "link": null, + "locked": false, + "points": [ + [ + 0, + 0 + ], + [ + 30, + 0 + ] + ], + "lastCommittedPoint": null, + "startBinding": null, + "endBinding": null, + "startArrowhead": null, + "endArrowhead": "arrow" + }, + { + "id": "a1763162771", + "type": "arrow", + "x": 610, + "y": 202, + "width": 40, + "height": 0, + "angle": 0, + "strokeColor": "#b8650a", + "backgroundColor": "transparent", + "fillStyle": "solid", + "strokeWidth": 2, + "strokeStyle": "solid", + "roughness": 1, + "opacity": 100, + "groupIds": [], + "frameId": null, + "roundness": { + "type": 2 + }, + "seed": 114658230, + "versionNonce": 1480716898, + "isDeleted": false, + "boundElements": [], + "updated": 1, + "link": null, + "locked": false, + "points": [ + [ + 0, + 0 + ], + [ + 40, + 0 + ] + ], + "lastCommittedPoint": null, + "startBinding": null, + "endBinding": null, + "startArrowhead": null, + "endArrowhead": "arrow" + }, + { + "id": "t747262755", + "type": "text", + "x": 630.0, + "y": 188.0, + "width": 40, + "height": 13.75, + "angle": 0, + "strokeColor": "#b8650a", + "backgroundColor": "transparent", + "fillStyle": "solid", + "strokeWidth": 1, + "strokeStyle": "solid", + "roughness": 1, + "opacity": 100, + "groupIds": [], + "frameId": null, + "roundness": null, + "seed": 1816920837, + "versionNonce": 11601135, + "isDeleted": false, + "boundElements": [], + "updated": 1, + "link": null, + "locked": false, + "text": "match", + "fontSize": 11, + "fontFamily": 1, + "textAlign": "left", + "verticalAlign": "top", + "containerId": null, + "originalText": "match", + "lineHeight": 1.25, + "baseline": 11 + }, + { + "id": "a1368358534", + "type": "arrow", + "x": 770, + "y": 175, + "width": 0, + "height": 110, + "angle": 0, + "strokeColor": "#b8650a", + "backgroundColor": "transparent", + "fillStyle": "solid", + "strokeWidth": 2, + "strokeStyle": "solid", + "roughness": 1, + "opacity": 100, + "groupIds": [], + "frameId": null, + "roundness": { + "type": 2 + }, + "seed": 1416363045, + "versionNonce": 1254311840, + "isDeleted": false, + "boundElements": [], + "updated": 1, + "link": null, + "locked": false, + "points": [ + [ + 0, + 0 + ], + [ + 0, + -110 + ] + ], + "lastCommittedPoint": null, + "startBinding": null, + "endBinding": null, + "startArrowhead": null, + "endArrowhead": "arrow" + }, + { + "id": "t1940668535", + "type": "text", + "x": 770.0, + "y": 106.0, + "width": 96.80000000000001, + "height": 13.75, + "angle": 0, + "strokeColor": "#b8650a", + "backgroundColor": "transparent", + "fillStyle": "solid", + "strokeWidth": 1, + "strokeStyle": "solid", + "roughness": 1, + "opacity": 100, + "groupIds": [], + "frameId": null, + "roundness": null, + "seed": 1174150746, + "versionNonce": 1803124562, + "isDeleted": false, + "boundElements": [], + "updated": 1, + "link": null, + "locked": false, + "text": "return to caller", + "fontSize": 11, + "fontFamily": 1, + "textAlign": "left", + "verticalAlign": "top", + "containerId": null, + "originalText": "return to caller", + "lineHeight": 1.25, + "baseline": 11 + }, + { + "id": "a332109020", + "type": "arrow", + "x": 460, + "y": 230, + "width": 0, + "height": 30, + "angle": 0, + "strokeColor": "#555555", + "backgroundColor": "transparent", + "fillStyle": "solid", + "strokeWidth": 2, + "strokeStyle": "dashed", + "roughness": 1, + "opacity": 100, + "groupIds": [], + "frameId": null, + "roundness": { + "type": 2 + }, + "seed": 854044961, + "versionNonce": 1890147705, + "isDeleted": false, + "boundElements": [], + "updated": 1, + "link": null, + "locked": false, + "points": [ + [ + 0, + 0 + ], + [ + 0, + 30 + ] + ], + "lastCommittedPoint": null, + "startBinding": null, + "endBinding": null, + "startArrowhead": null, + "endArrowhead": "arrow" + }, + { + "id": "t1658671849", + "type": "text", + "x": 460.0, + "y": 231.0, + "width": 48.400000000000006, + "height": 13.75, + "angle": 0, + "strokeColor": "#555555", + "backgroundColor": "transparent", + "fillStyle": "solid", + "strokeWidth": 1, + "strokeStyle": "solid", + "roughness": 1, + "opacity": 100, + "groupIds": [], + "frameId": null, + "roundness": null, + "seed": 773247796, + "versionNonce": 808250572, + "isDeleted": false, + "boundElements": [], + "updated": 1, + "link": null, + "locked": false, + "text": "no match", + "fontSize": 11, + "fontFamily": 1, + "textAlign": "left", + "verticalAlign": "top", + "containerId": null, + "originalText": "no match", + "lineHeight": 1.25, + "baseline": 11 + }, + { + "id": "a1851542360", + "type": "arrow", + "x": 400, + "y": 310, + "width": 150, + "height": 30, + "angle": 0, + "strokeColor": "#555555", + "backgroundColor": "transparent", + "fillStyle": "solid", + "strokeWidth": 2, + "strokeStyle": "solid", + "roughness": 1, + "opacity": 100, + "groupIds": [], + "frameId": null, + "roundness": { + "type": 2 + }, + "seed": 687545755, + "versionNonce": 607909489, + "isDeleted": false, + "boundElements": [], + "updated": 1, + "link": null, + "locked": false, + "points": [ + [ + 0, + 0 + ], + [ + -150, + 30 + ] + ], + "lastCommittedPoint": null, + "startBinding": null, + "endBinding": null, + "startArrowhead": null, + "endArrowhead": "arrow" + }, + { + "id": "t1857532236", + "type": "text", + "x": 325.0, + "y": 311.0, + "width": 133.10000000000002, + "height": 13.75, + "angle": 0, + "strokeColor": "#555555", + "backgroundColor": "transparent", + "fillStyle": "solid", + "strokeWidth": 1, + "strokeStyle": "solid", + "roughness": 1, + "opacity": 100, + "groupIds": [], + "frameId": null, + "roundness": null, + "seed": 1819942090, + "versionNonce": 1096932915, + "isDeleted": false, + "boundElements": [], + "updated": 1, + "link": null, + "locked": false, + "text": "this program's default", + "fontSize": 11, + "fontFamily": 1, + "textAlign": "left", + "verticalAlign": "top", + "containerId": null, + "originalText": "this program's default", + "lineHeight": 1.25, + "baseline": 11 + }, + { + "id": "a894278518", + "type": "arrow", + "x": 520, + "y": 310, + "width": 170, + "height": 30, + "angle": 0, + "strokeColor": "#555555", + "backgroundColor": "transparent", + "fillStyle": "solid", + "strokeWidth": 2, + "strokeStyle": "dashed", + "roughness": 1, + "opacity": 100, + "groupIds": [], + "frameId": null, + "roundness": { + "type": 2 + }, + "seed": 1941511701, + "versionNonce": 1333096436, + "isDeleted": false, + "boundElements": [], + "updated": 1, + "link": null, + "locked": false, + "points": [ + [ + 0, + 0 + ], + [ + 170, + 30 + ] + ], + "lastCommittedPoint": null, + "startBinding": null, + "endBinding": null, + "startArrowhead": null, + "endArrowhead": "arrow" + }, + { + "id": "t1991458349", + "type": "text", + "x": 605.0, + "y": 311.0, + "width": 108.9, + "height": 13.75, + "angle": 0, + "strokeColor": "#555555", + "backgroundColor": "transparent", + "fillStyle": "solid", + "strokeWidth": 1, + "strokeStyle": "solid", + "roughness": 1, + "opacity": 100, + "groupIds": [], + "frameId": null, + "roundness": null, + "seed": 1970003592, + "versionNonce": 1098249765, + "isDeleted": false, + "boundElements": [], + "updated": 1, + "link": null, + "locked": false, + "text": "alternative action", + "fontSize": 11, + "fontFamily": 1, + "textAlign": "left", + "verticalAlign": "top", + "containerId": null, + "originalText": "alternative action", + "lineHeight": 1.25, + "baseline": 11 + }, + { + "id": "t1892250442", + "type": "text", + "x": 470, + "y": 10, + "width": 399.3, + "height": 13.75, + "angle": 0, + "strokeColor": "#1d1d1d", + "backgroundColor": "transparent", + "fillStyle": "solid", + "strokeWidth": 1, + "strokeStyle": "solid", + "roughness": 1, + "opacity": 100, + "groupIds": [], + "frameId": null, + "roundness": null, + "seed": 1355246149, + "versionNonce": 1727600140, + "isDeleted": false, + "boundElements": [], + "updated": 1, + "link": null, + "locked": false, + "text": "Figure 33.1 \u2014 a syscall's path through fwatch's seccomp-bpf filter", + "fontSize": 11, + "fontFamily": 1, + "textAlign": "left", + "verticalAlign": "top", + "containerId": null, + "originalText": "Figure 33.1 \u2014 a syscall's path through fwatch's seccomp-bpf filter", + "lineHeight": 1.25, + "baseline": 11 + }, + { + "id": "t881985589", + "type": "text", + "x": 30, + "y": 410, + "width": 641.3000000000001, + "height": 13.75, + "angle": 0, + "strokeColor": "#1d1d1d", + "backgroundColor": "transparent", + "fillStyle": "solid", + "strokeWidth": 1, + "strokeStyle": "solid", + "roughness": 1, + "opacity": 100, + "groupIds": [], + "frameId": null, + "roundness": null, + "seed": 994109340, + "versionNonce": 151199675, + "isDeleted": false, + "boundElements": [], + "updated": 1, + "link": null, + "locked": false, + "text": "Real: cpp/rust filters admit 26 syscalls, go's 39 (goroutine scheduler needs futex/clone/sigaltstack too).", + "fontSize": 11, + "fontFamily": 1, + "textAlign": "left", + "verticalAlign": "top", + "containerId": null, + "originalText": "Real: cpp/rust filters admit 26 syscalls, go's 39 (goroutine scheduler needs futex/clone/sigaltstack too).", + "lineHeight": 1.25, + "baseline": 11 + }, + { + "id": "t508170261", + "type": "text", + "x": 30, + "y": 430, + "width": 1022.45, + "height": 13.75, + "angle": 0, + "strokeColor": "#1d1d1d", + "backgroundColor": "transparent", + "fillStyle": "solid", + "strokeWidth": 1, + "strokeStyle": "solid", + "roughness": 1, + "opacity": 100, + "groupIds": [], + "frameId": null, + "roundness": null, + "seed": 1323691748, + "versionNonce": 1636877331, + "isDeleted": false, + "boundElements": [], + "updated": 1, + "link": null, + "locked": false, + "text": "Guest (systems-target, kernel 6.19.10): socket(AF_INET,...) = -1 EPERM under fwatch's filter; a separate ACT_KILL_PROCESS probe on the same guest hit SIGSYS on getpid().", + "fontSize": 11, + "fontFamily": 1, + "textAlign": "left", + "verticalAlign": "top", + "containerId": null, + "originalText": "Guest (systems-target, kernel 6.19.10): socket(AF_INET,...) = -1 EPERM under fwatch's filter; a separate ACT_KILL_PROCESS probe on the same guest hit SIGSYS on getpid().", + "lineHeight": 1.25, + "baseline": 11 + } + ], + "appState": { + "viewBackgroundColor": "#ffffff", + "gridSize": null + }, + "files": {} +} \ No newline at end of file diff --git a/assets/diagrams/33-seccomp-filter-flow.svg b/assets/diagrams/33-seccomp-filter-flow.svg new file mode 100644 index 0000000..6cafb84 --- /dev/null +++ b/assets/diagrams/33-seccomp-filter-flow.svg @@ -0,0 +1,53 @@ + + + +userspace: fwatch watch loop + +kernel: seccomp-bpf filter — one cBPF program, RUN AGAIN on every syscall entry + +syscall(2) issued +e.g. epoll_wait, read, socket + +this IS classic BPF (cBPF) +you write the filter — Part 8's bpftrace observes; this decides + +BPF_LD|BPF_W|BPF_ABS +load nr + arch offset + +jump table: nr == read? epoll_wait? … +26 entries (cpp/rust) · 39 (go) + +match → SECCOMP_RET_ALLOW +syscall actually runs + +no match → default action +(seccomp_init's action arg) + +SCMP_ACT_ERRNO(EPERM) — fwatch's choice +socket(2) = -1 EPERM; process keeps running + +SCMP_ACT_KILL_PROCESS — not used by fwatch +getpid() denied → +++ killed by SIGSYS +++ + +traps into kernel +traps into kernel + + +match +match + +return to caller +return to caller + +no match +no match + +this program's default +this program's default + +alternative action +alternative action +Figure 33.1 — a syscall's path through fwatch's seccomp-bpf filter +Real: cpp/rust filters admit 26 syscalls, go's 39 (goroutine scheduler needs futex/clone/sigaltstack too). +Guest (systems-target, kernel 6.19.10): socket(AF_INET,...) = -1 EPERM under fwatch's filter; a separate ACT_KILL_PROCESS probe on the same guest hit SIGSYS on getpid(). + \ No newline at end of file diff --git a/assets/diagrams/34-image-layering.excalidraw b/assets/diagrams/34-image-layering.excalidraw new file mode 100644 index 0000000..93f0fe2 --- /dev/null +++ b/assets/diagrams/34-image-layering.excalidraw @@ -0,0 +1,846 @@ +{ + "type": "excalidraw", + "version": 2, + "source": "https://excalidraw.com", + "elements": [ + { + "id": "r1621800435", + "type": "rectangle", + "x": 20, + "y": 30, + "width": 380, + "height": 240, + "angle": 0, + "strokeColor": "#dddddd", + "backgroundColor": "#f4f4f4", + "fillStyle": "solid", + "strokeWidth": 2, + "strokeStyle": "solid", + "roughness": 1, + "opacity": 100, + "groupIds": [], + "frameId": null, + "roundness": { + "type": 3 + }, + "seed": 984833536, + "versionNonce": 690419675, + "isDeleted": false, + "boundElements": [], + "updated": 1, + "link": null, + "locked": false + }, + { + "id": "t1595953958", + "type": "text", + "x": 32, + "y": 36, + "width": 165.0, + "height": 15.0, + "angle": 0, + "strokeColor": "#555555", + "backgroundColor": "transparent", + "fillStyle": "solid", + "strokeWidth": 1, + "strokeStyle": "solid", + "roughness": 1, + "opacity": 100, + "groupIds": [], + "frameId": null, + "roundness": null, + "seed": 619239148, + "versionNonce": 538901792, + "isDeleted": false, + "boundElements": [], + "updated": 1, + "link": null, + "locked": false, + "text": "builder stage (discarded)", + "fontSize": 12, + "fontFamily": 1, + "textAlign": "left", + "verticalAlign": "top", + "containerId": null, + "originalText": "builder stage (discarded)", + "lineHeight": 1.25, + "baseline": 12 + }, + { + "id": "r613445054", + "type": "rectangle", + "x": 460, + "y": 30, + "width": 400, + "height": 240, + "angle": 0, + "strokeColor": "#dddddd", + "backgroundColor": "#fff8ef", + "fillStyle": "solid", + "strokeWidth": 2, + "strokeStyle": "solid", + "roughness": 1, + "opacity": 100, + "groupIds": [], + "frameId": null, + "roundness": { + "type": 3 + }, + "seed": 1059568730, + "versionNonce": 554110188, + "isDeleted": false, + "boundElements": [], + "updated": 1, + "link": null, + "locked": false + }, + { + "id": "t531546782", + "type": "text", + "x": 472, + "y": 36, + "width": 191.4, + "height": 15.0, + "angle": 0, + "strokeColor": "#555555", + "backgroundColor": "transparent", + "fillStyle": "solid", + "strokeWidth": 1, + "strokeStyle": "solid", + "roughness": 1, + "opacity": 100, + "groupIds": [], + "frameId": null, + "roundness": null, + "seed": 1821925975, + "versionNonce": 497374190, + "isDeleted": false, + "boundElements": [], + "updated": 1, + "link": null, + "locked": false, + "text": "runtime stage (shipped image)", + "fontSize": 12, + "fontFamily": 1, + "textAlign": "left", + "verticalAlign": "top", + "containerId": null, + "originalText": "runtime stage (shipped image)", + "lineHeight": 1.25, + "baseline": 12 + }, + { + "id": "r1585488827", + "type": "rectangle", + "x": 44, + "y": 70, + "width": 340, + "height": 60, + "angle": 0, + "strokeColor": "#999999", + "backgroundColor": "#ffffff", + "fillStyle": "solid", + "strokeWidth": 2, + "strokeStyle": "solid", + "roughness": 1, + "opacity": 100, + "groupIds": [], + "frameId": null, + "roundness": { + "type": 3 + }, + "seed": 283466691, + "versionNonce": 1114156423, + "isDeleted": false, + "boundElements": [], + "updated": 1, + "link": null, + "locked": false + }, + { + "id": "t73738148", + "type": "text", + "x": 54, + "y": 78, + "width": 315.70000000000005, + "height": 17.5, + "angle": 0, + "strokeColor": "#111111", + "backgroundColor": "transparent", + "fillStyle": "solid", + "strokeWidth": 1, + "strokeStyle": "solid", + "roughness": 1, + "opacity": 100, + "groupIds": [], + "frameId": null, + "roundness": null, + "seed": 1514244070, + "versionNonce": 561156743, + "isDeleted": false, + "boundElements": [], + "updated": 1, + "link": null, + "locked": false, + "text": "registry.access.redhat.com/ubi10/ubi:10.2", + "fontSize": 14, + "fontFamily": 1, + "textAlign": "left", + "verticalAlign": "top", + "containerId": null, + "originalText": "registry.access.redhat.com/ubi10/ubi:10.2", + "lineHeight": 1.25, + "baseline": 14 + }, + { + "id": "t1142747838", + "type": "text", + "x": 54, + "y": 98, + "width": 90.75000000000001, + "height": 13.75, + "angle": 0, + "strokeColor": "#555555", + "backgroundColor": "transparent", + "fillStyle": "solid", + "strokeWidth": 1, + "strokeStyle": "solid", + "roughness": 1, + "opacity": 100, + "groupIds": [], + "frameId": null, + "roundness": null, + "seed": 1325831284, + "versionNonce": 429921011, + "isDeleted": false, + "boundElements": [], + "updated": 1, + "link": null, + "locked": false, + "text": "220 MB measured", + "fontSize": 11, + "fontFamily": 1, + "textAlign": "left", + "verticalAlign": "top", + "containerId": null, + "originalText": "220 MB measured", + "lineHeight": 1.25, + "baseline": 11 + }, + { + "id": "r887653069", + "type": "rectangle", + "x": 44, + "y": 150, + "width": 340, + "height": 90, + "angle": 0, + "strokeColor": "#111111", + "backgroundColor": "#ffffff", + "fillStyle": "solid", + "strokeWidth": 2, + "strokeStyle": "solid", + "roughness": 1, + "opacity": 100, + "groupIds": [], + "frameId": null, + "roundness": { + "type": 3 + }, + "seed": 1471815477, + "versionNonce": 52447765, + "isDeleted": false, + "boundElements": [], + "updated": 1, + "link": null, + "locked": false + }, + { + "id": "t492321432", + "type": "text", + "x": 54, + "y": 158, + "width": 254.10000000000002, + "height": 17.5, + "angle": 0, + "strokeColor": "#111111", + "backgroundColor": "transparent", + "fillStyle": "solid", + "strokeWidth": 1, + "strokeStyle": "solid", + "roughness": 1, + "opacity": 100, + "groupIds": [], + "frameId": null, + "roundness": null, + "seed": 1608472045, + "versionNonce": 967939289, + "isDeleted": false, + "boundElements": [], + "updated": 1, + "link": null, + "locked": false, + "text": "dnf install gcc-c++/golang/rustup", + "fontSize": 14, + "fontFamily": 1, + "textAlign": "left", + "verticalAlign": "top", + "containerId": null, + "originalText": "dnf install gcc-c++/golang/rustup", + "lineHeight": 1.25, + "baseline": 14 + }, + { + "id": "t1202058217", + "type": "text", + "x": 54, + "y": 178, + "width": 133.10000000000002, + "height": 13.75, + "angle": 0, + "strokeColor": "#555555", + "backgroundColor": "transparent", + "fillStyle": "solid", + "strokeWidth": 1, + "strokeStyle": "solid", + "roughness": 1, + "opacity": 100, + "groupIds": [], + "frameId": null, + "roundness": null, + "seed": 1325324526, + "versionNonce": 1722227580, + "isDeleted": false, + "boundElements": [], + "updated": 1, + "link": null, + "locked": false, + "text": "compile app statically", + "fontSize": 11, + "fontFamily": 1, + "textAlign": "left", + "verticalAlign": "top", + "containerId": null, + "originalText": "compile app statically", + "lineHeight": 1.25, + "baseline": 11 + }, + { + "id": "t790595909", + "type": "text", + "x": 54, + "y": 193, + "width": 223.85000000000002, + "height": 13.75, + "angle": 0, + "strokeColor": "#555555", + "backgroundColor": "transparent", + "fillStyle": "solid", + "strokeWidth": 1, + "strokeStyle": "solid", + "roughness": 1, + "opacity": 100, + "groupIds": [], + "frameId": null, + "roundness": null, + "seed": 1136825043, + "versionNonce": 1158719935, + "isDeleted": false, + "boundElements": [], + "updated": 1, + "link": null, + "locked": false, + "text": "cpp: -static-libstdc++ -static-libgcc", + "fontSize": 11, + "fontFamily": 1, + "textAlign": "left", + "verticalAlign": "top", + "containerId": null, + "originalText": "cpp: -static-libstdc++ -static-libgcc", + "lineHeight": 1.25, + "baseline": 11 + }, + { + "id": "t387759355", + "type": "text", + "x": 54, + "y": 208, + "width": 235.95000000000002, + "height": 13.75, + "angle": 0, + "strokeColor": "#555555", + "backgroundColor": "transparent", + "fillStyle": "solid", + "strokeWidth": 1, + "strokeStyle": "solid", + "roughness": 1, + "opacity": 100, + "groupIds": [], + "frameId": null, + "roundness": null, + "seed": 1976585418, + "versionNonce": 987841413, + "isDeleted": false, + "boundElements": [], + "updated": 1, + "link": null, + "locked": false, + "text": "go: CGO_ENABLED=0 rust: rustup 1.97.1", + "fontSize": 11, + "fontFamily": 1, + "textAlign": "left", + "verticalAlign": "top", + "containerId": null, + "originalText": "go: CGO_ENABLED=0 rust: rustup 1.97.1", + "lineHeight": 1.25, + "baseline": 11 + }, + { + "id": "r686249107", + "type": "rectangle", + "x": 484, + "y": 70, + "width": 352, + "height": 60, + "angle": 0, + "strokeColor": "#e8870c", + "backgroundColor": "#fff8ef", + "fillStyle": "solid", + "strokeWidth": 2, + "strokeStyle": "solid", + "roughness": 1, + "opacity": 100, + "groupIds": [], + "frameId": null, + "roundness": { + "type": 3 + }, + "seed": 414425652, + "versionNonce": 79624146, + "isDeleted": false, + "boundElements": [], + "updated": 1, + "link": null, + "locked": false + }, + { + "id": "t1524303659", + "type": "text", + "x": 494, + "y": 78, + "width": 331.1, + "height": 17.5, + "angle": 0, + "strokeColor": "#111111", + "backgroundColor": "transparent", + "fillStyle": "solid", + "strokeWidth": 1, + "strokeStyle": "solid", + "roughness": 1, + "opacity": 100, + "groupIds": [], + "frameId": null, + "roundness": null, + "seed": 1092626186, + "versionNonce": 286582059, + "isDeleted": false, + "boundElements": [], + "updated": 1, + "link": null, + "locked": false, + "text": "registry.access.redhat.com/ubi10-micro:10.2", + "fontSize": 14, + "fontFamily": 1, + "textAlign": "left", + "verticalAlign": "top", + "containerId": null, + "originalText": "registry.access.redhat.com/ubi10-micro:10.2", + "lineHeight": 1.25, + "baseline": 14 + }, + { + "id": "t1193605168", + "type": "text", + "x": 494, + "y": 98, + "width": 96.80000000000001, + "height": 13.75, + "angle": 0, + "strokeColor": "#555555", + "backgroundColor": "transparent", + "fillStyle": "solid", + "strokeWidth": 1, + "strokeStyle": "solid", + "roughness": 1, + "opacity": 100, + "groupIds": [], + "frameId": null, + "roundness": null, + "seed": 1602253299, + "versionNonce": 1173688695, + "isDeleted": false, + "boundElements": [], + "updated": 1, + "link": null, + "locked": false, + "text": "24.3 MB measured", + "fontSize": 11, + "fontFamily": 1, + "textAlign": "left", + "verticalAlign": "top", + "containerId": null, + "originalText": "24.3 MB measured", + "lineHeight": 1.25, + "baseline": 11 + }, + { + "id": "r1934406755", + "type": "rectangle", + "x": 484, + "y": 150, + "width": 352, + "height": 40, + "angle": 0, + "strokeColor": "#111111", + "backgroundColor": "#111111", + "fillStyle": "solid", + "strokeWidth": 2, + "strokeStyle": "solid", + "roughness": 1, + "opacity": 100, + "groupIds": [], + "frameId": null, + "roundness": { + "type": 3 + }, + "seed": 1908722830, + "versionNonce": 902308263, + "isDeleted": false, + "boundElements": [], + "updated": 1, + "link": null, + "locked": false + }, + { + "id": "t1731796848", + "type": "text", + "x": 494, + "y": 158, + "width": 323.40000000000003, + "height": 17.5, + "angle": 0, + "strokeColor": "#ffffff", + "backgroundColor": "transparent", + "fillStyle": "solid", + "strokeWidth": 1, + "strokeStyle": "solid", + "roughness": 1, + "opacity": 100, + "groupIds": [], + "frameId": null, + "roundness": null, + "seed": 964054447, + "versionNonce": 973119072, + "isDeleted": false, + "boundElements": [], + "updated": 1, + "link": null, + "locked": false, + "text": "COPY --from=builder app /usr/local/bin/app", + "fontSize": 14, + "fontFamily": 1, + "textAlign": "left", + "verticalAlign": "top", + "containerId": null, + "originalText": "COPY --from=builder app /usr/local/bin/app", + "lineHeight": 1.25, + "baseline": 14 + }, + { + "id": "r606456512", + "type": "rectangle", + "x": 484, + "y": 210, + "width": 352, + "height": 40, + "angle": 0, + "strokeColor": "#111111", + "backgroundColor": "#ffffff", + "fillStyle": "solid", + "strokeWidth": 2, + "strokeStyle": "solid", + "roughness": 1, + "opacity": 100, + "groupIds": [], + "frameId": null, + "roundness": { + "type": 3 + }, + "seed": 1088140993, + "versionNonce": 571830926, + "isDeleted": false, + "boundElements": [], + "updated": 1, + "link": null, + "locked": false + }, + { + "id": "t1500075463", + "type": "text", + "x": 494, + "y": 218, + "width": 377.3, + "height": 17.5, + "angle": 0, + "strokeColor": "#111111", + "backgroundColor": "transparent", + "fillStyle": "solid", + "strokeWidth": 1, + "strokeStyle": "solid", + "roughness": 1, + "opacity": 100, + "groupIds": [], + "frameId": null, + "roundness": null, + "seed": 1567373787, + "versionNonce": 1960820685, + "isDeleted": false, + "boundElements": [], + "updated": 1, + "link": null, + "locked": false, + "text": "ENTRYPOINT [\"/usr/local/bin/app\"] CMD [\"serve\"]", + "fontSize": 14, + "fontFamily": 1, + "textAlign": "left", + "verticalAlign": "top", + "containerId": null, + "originalText": "ENTRYPOINT [\"/usr/local/bin/app\"] CMD [\"serve\"]", + "lineHeight": 1.25, + "baseline": 14 + }, + { + "id": "a1689176029", + "type": "arrow", + "x": 384, + "y": 195, + "width": 100, + "height": 0, + "angle": 0, + "strokeColor": "#b8650a", + "backgroundColor": "transparent", + "fillStyle": "solid", + "strokeWidth": 2, + "strokeStyle": "solid", + "roughness": 1, + "opacity": 100, + "groupIds": [], + "frameId": null, + "roundness": { + "type": 2 + }, + "seed": 285803148, + "versionNonce": 1370178456, + "isDeleted": false, + "boundElements": [], + "updated": 1, + "link": null, + "locked": false, + "points": [ + [ + 0, + 0 + ], + [ + 100, + 0 + ] + ], + "lastCommittedPoint": null, + "startBinding": null, + "endBinding": null, + "startArrowhead": null, + "endArrowhead": "arrow" + }, + { + "id": "t1285508864", + "type": "text", + "x": 434.0, + "y": 181.0, + "width": 114.95, + "height": 13.75, + "angle": 0, + "strokeColor": "#b8650a", + "backgroundColor": "transparent", + "fillStyle": "solid", + "strokeWidth": 1, + "strokeStyle": "solid", + "roughness": 1, + "opacity": 100, + "groupIds": [], + "frameId": null, + "roundness": null, + "seed": 865591394, + "versionNonce": 1602426191, + "isDeleted": false, + "boundElements": [], + "updated": 1, + "link": null, + "locked": false, + "text": "COPY --from=builder", + "fontSize": 11, + "fontFamily": 1, + "textAlign": "left", + "verticalAlign": "top", + "containerId": null, + "originalText": "COPY --from=builder", + "lineHeight": 1.25, + "baseline": 11 + }, + { + "id": "t142807649", + "type": "text", + "x": 44, + "y": 290, + "width": 602.25, + "height": 18.75, + "angle": 0, + "strokeColor": "#1d1d1d", + "backgroundColor": "transparent", + "fillStyle": "solid", + "strokeWidth": 1, + "strokeStyle": "solid", + "roughness": 1, + "opacity": 100, + "groupIds": [], + "frameId": null, + "roundness": null, + "seed": 234335808, + "versionNonce": 1031832319, + "isDeleted": false, + "boundElements": [], + "updated": 1, + "link": null, + "locked": false, + "text": "Final images measured on this host: cpp 27.2 MB, go 27.2 MB, rust 24.9 MB", + "fontSize": 15, + "fontFamily": 1, + "textAlign": "left", + "verticalAlign": "top", + "containerId": null, + "originalText": "Final images measured on this host: cpp 27.2 MB, go 27.2 MB, rust 24.9 MB", + "lineHeight": 1.25, + "baseline": 15 + }, + { + "id": "t417245571", + "type": "text", + "x": 44, + "y": 315, + "width": 639.1, + "height": 17.5, + "angle": 0, + "strokeColor": "#555555", + "backgroundColor": "transparent", + "fillStyle": "solid", + "strokeWidth": 1, + "strokeStyle": "solid", + "roughness": 1, + "opacity": 100, + "groupIds": [], + "frameId": null, + "roundness": null, + "seed": 1028240103, + "versionNonce": 985262388, + "isDeleted": false, + "boundElements": [], + "updated": 1, + "link": null, + "locked": false, + "text": "-- barely above the bare ubi10-micro base (24.3 MB); the 220 MB builder never ships", + "fontSize": 14, + "fontFamily": 1, + "textAlign": "left", + "verticalAlign": "top", + "containerId": null, + "originalText": "-- barely above the bare ubi10-micro base (24.3 MB); the 220 MB builder never ships", + "lineHeight": 1.25, + "baseline": 14 + }, + { + "id": "t748779884", + "type": "text", + "x": 44, + "y": 345, + "width": 729.3000000000001, + "height": 16.25, + "angle": 0, + "strokeColor": "#555555", + "backgroundColor": "transparent", + "fillStyle": "solid", + "strokeWidth": 1, + "strokeStyle": "solid", + "roughness": 1, + "opacity": 100, + "groupIds": [], + "frameId": null, + "roundness": null, + "seed": 985786868, + "versionNonce": 97636662, + "isDeleted": false, + "boundElements": [], + "updated": 1, + "link": null, + "locked": false, + "text": "Static linking (cpp -static-libstdc++, go CGO_ENABLED=0) is what makes 'copy one binary' sufficient --", + "fontSize": 13, + "fontFamily": 1, + "textAlign": "left", + "verticalAlign": "top", + "containerId": null, + "originalText": "Static linking (cpp -static-libstdc++, go CGO_ENABLED=0) is what makes 'copy one binary' sufficient --", + "lineHeight": 1.25, + "baseline": 13 + }, + { + "id": "t140334131", + "type": "text", + "x": 44, + "y": 365, + "width": 593.45, + "height": 16.25, + "angle": 0, + "strokeColor": "#555555", + "backgroundColor": "transparent", + "fillStyle": "solid", + "strokeWidth": 1, + "strokeStyle": "solid", + "roughness": 1, + "opacity": 100, + "groupIds": [], + "frameId": null, + "roundness": null, + "seed": 1548967242, + "versionNonce": 1716554502, + "isDeleted": false, + "boundElements": [], + "updated": 1, + "link": null, + "locked": false, + "text": "a dynamically-linked binary would need its .so's copied into the runtime stage too.", + "fontSize": 13, + "fontFamily": 1, + "textAlign": "left", + "verticalAlign": "top", + "containerId": null, + "originalText": "a dynamically-linked binary would need its .so's copied into the runtime stage too.", + "lineHeight": 1.25, + "baseline": 13 + } + ], + "appState": { + "viewBackgroundColor": "#ffffff", + "gridSize": null + }, + "files": {} +} \ No newline at end of file diff --git a/assets/diagrams/34-image-layering.svg b/assets/diagrams/34-image-layering.svg new file mode 100644 index 0000000..460fd1c --- /dev/null +++ b/assets/diagrams/34-image-layering.svg @@ -0,0 +1,29 @@ + + + +builder stage (discarded) + +runtime stage (shipped image) + +registry.access.redhat.com/ubi10/ubi:10.2 +220 MB measured + +dnf install gcc-c++/golang/rustup +compile app statically +cpp: -static-libstdc++ -static-libgcc +go: CGO_ENABLED=0 rust: rustup 1.97.1 + +registry.access.redhat.com/ubi10-micro:10.2 +24.3 MB measured + +COPY --from=builder app /usr/local/bin/app + +ENTRYPOINT ["/usr/local/bin/app"] CMD ["serve"] + +COPY --from=builder +COPY --from=builder +Final images measured on this host: cpp 27.2 MB, go 27.2 MB, rust 24.9 MB +-- barely above the bare ubi10-micro base (24.3 MB); the 220 MB builder never ships +Static linking (cpp -static-libstdc++, go CGO_ENABLED=0) is what makes 'copy one binary' sufficient -- +a dynamically-linked binary would need its .so's copied into the runtime stage too. + \ No newline at end of file diff --git a/assets/diagrams/34-ns-debugging-map.excalidraw b/assets/diagrams/34-ns-debugging-map.excalidraw new file mode 100644 index 0000000..a35e708 --- /dev/null +++ b/assets/diagrams/34-ns-debugging-map.excalidraw @@ -0,0 +1,1109 @@ +{ + "type": "excalidraw", + "version": 2, + "source": "https://excalidraw.com", + "elements": [ + { + "id": "r851808859", + "type": "rectangle", + "x": 20, + "y": 30, + "width": 880, + "height": 130, + "angle": 0, + "strokeColor": "#dddddd", + "backgroundColor": "#f4f4f4", + "fillStyle": "solid", + "strokeWidth": 2, + "strokeStyle": "solid", + "roughness": 1, + "opacity": 100, + "groupIds": [], + "frameId": null, + "roundness": { + "type": 3 + }, + "seed": 928233603, + "versionNonce": 685123035, + "isDeleted": false, + "boundElements": [], + "updated": 1, + "link": null, + "locked": false + }, + { + "id": "t365366183", + "type": "text", + "x": 32, + "y": 36, + "width": 244.20000000000002, + "height": 15.0, + "angle": 0, + "strokeColor": "#555555", + "backgroundColor": "transparent", + "fillStyle": "solid", + "strokeWidth": 1, + "strokeStyle": "solid", + "roughness": 1, + "opacity": 100, + "groupIds": [], + "frameId": null, + "roundness": null, + "seed": 1748730940, + "versionNonce": 1337227903, + "isDeleted": false, + "boundElements": [], + "updated": 1, + "link": null, + "locked": false, + "text": "host (rootless user rsedor, uid 1000)", + "fontSize": 12, + "fontFamily": 1, + "textAlign": "left", + "verticalAlign": "top", + "containerId": null, + "originalText": "host (rootless user rsedor, uid 1000)", + "lineHeight": 1.25, + "baseline": 12 + }, + { + "id": "r1384913294", + "type": "rectangle", + "x": 20, + "y": 190, + "width": 880, + "height": 200, + "angle": 0, + "strokeColor": "#dddddd", + "backgroundColor": "#fff8ef", + "fillStyle": "solid", + "strokeWidth": 2, + "strokeStyle": "solid", + "roughness": 1, + "opacity": 100, + "groupIds": [], + "frameId": null, + "roundness": { + "type": 3 + }, + "seed": 225790868, + "versionNonce": 1591494656, + "isDeleted": false, + "boundElements": [], + "updated": 1, + "link": null, + "locked": false + }, + { + "id": "t514860128", + "type": "text", + "x": 32, + "y": 196, + "width": 422.40000000000003, + "height": 15.0, + "angle": 0, + "strokeColor": "#555555", + "backgroundColor": "transparent", + "fillStyle": "solid", + "strokeWidth": 1, + "strokeStyle": "solid", + "roughness": 1, + "opacity": 100, + "groupIds": [], + "frameId": null, + "roundness": null, + "seed": 816900920, + "versionNonce": 646162393, + "isDeleted": false, + "boundElements": [], + "updated": 1, + "link": null, + "locked": false, + "text": "podman's user namespace (uid 0 in container == uid 1000 on host)", + "fontSize": 12, + "fontFamily": 1, + "textAlign": "left", + "verticalAlign": "top", + "containerId": null, + "originalText": "podman's user namespace (uid 0 in container == uid 1000 on host)", + "lineHeight": 1.25, + "baseline": 12 + }, + { + "id": "r1978916558", + "type": "rectangle", + "x": 44, + "y": 66, + "width": 260, + "height": 70, + "angle": 0, + "strokeColor": "#111111", + "backgroundColor": "#ffffff", + "fillStyle": "solid", + "strokeWidth": 2, + "strokeStyle": "solid", + "roughness": 1, + "opacity": 100, + "groupIds": [], + "frameId": null, + "roundness": { + "type": 3 + }, + "seed": 1982205519, + "versionNonce": 824309888, + "isDeleted": false, + "boundElements": [], + "updated": 1, + "link": null, + "locked": false + }, + { + "id": "t1420911871", + "type": "text", + "x": 54, + "y": 74, + "width": 154.0, + "height": 17.5, + "angle": 0, + "strokeColor": "#111111", + "backgroundColor": "transparent", + "fillStyle": "solid", + "strokeWidth": 1, + "strokeStyle": "solid", + "roughness": 1, + "opacity": 100, + "groupIds": [], + "frameId": null, + "roundness": null, + "seed": 1980130933, + "versionNonce": 358034792, + "isDeleted": false, + "boundElements": [], + "updated": 1, + "link": null, + "locked": false, + "text": "podman exec demo34go", + "fontSize": 14, + "fontFamily": 1, + "textAlign": "left", + "verticalAlign": "top", + "containerId": null, + "originalText": "podman exec demo34go", + "lineHeight": 1.25, + "baseline": 14 + }, + { + "id": "t1993901833", + "type": "text", + "x": 54, + "y": 94, + "width": 108.9, + "height": 13.75, + "angle": 0, + "strokeColor": "#555555", + "backgroundColor": "transparent", + "fillStyle": "solid", + "strokeWidth": 1, + "strokeStyle": "solid", + "roughness": 1, + "opacity": 100, + "groupIds": [], + "frameId": null, + "roundness": null, + "seed": 521987619, + "versionNonce": 1116217913, + "isDeleted": false, + "boundElements": [], + "updated": 1, + "link": null, + "locked": false, + "text": "cat /proc/1/status", + "fontSize": 11, + "fontFamily": 1, + "textAlign": "left", + "verticalAlign": "top", + "containerId": null, + "originalText": "cat /proc/1/status", + "lineHeight": 1.25, + "baseline": 11 + }, + { + "id": "r1378178516", + "type": "rectangle", + "x": 340, + "y": 66, + "width": 260, + "height": 70, + "angle": 0, + "strokeColor": "#111111", + "backgroundColor": "#ffffff", + "fillStyle": "solid", + "strokeWidth": 2, + "strokeStyle": "solid", + "roughness": 1, + "opacity": 100, + "groupIds": [], + "frameId": null, + "roundness": { + "type": 3 + }, + "seed": 1641816402, + "versionNonce": 1822143295, + "isDeleted": false, + "boundElements": [], + "updated": 1, + "link": null, + "locked": false + }, + { + "id": "t926167838", + "type": "text", + "x": 350, + "y": 74, + "width": 169.4, + "height": 17.5, + "angle": 0, + "strokeColor": "#111111", + "backgroundColor": "transparent", + "fillStyle": "solid", + "strokeWidth": 1, + "strokeStyle": "solid", + "roughness": 1, + "opacity": 100, + "groupIds": [], + "frameId": null, + "roundness": null, + "seed": 1251454809, + "versionNonce": 310886477, + "isDeleted": false, + "boundElements": [], + "updated": 1, + "link": null, + "locked": false, + "text": "podman unshare nsenter", + "fontSize": 14, + "fontFamily": 1, + "textAlign": "left", + "verticalAlign": "top", + "containerId": null, + "originalText": "podman unshare nsenter", + "lineHeight": 1.25, + "baseline": 14 + }, + { + "id": "t1725121586", + "type": "text", + "x": 350, + "y": 94, + "width": 193.60000000000002, + "height": 13.75, + "angle": 0, + "strokeColor": "#555555", + "backgroundColor": "transparent", + "fillStyle": "solid", + "strokeWidth": 1, + "strokeStyle": "solid", + "roughness": 1, + "opacity": 100, + "groupIds": [], + "frameId": null, + "roundness": null, + "seed": 1397074756, + "versionNonce": 1299753351, + "isDeleted": false, + "boundElements": [], + "updated": 1, + "link": null, + "locked": false, + "text": "--target --pid --mount", + "fontSize": 11, + "fontFamily": 1, + "textAlign": "left", + "verticalAlign": "top", + "containerId": null, + "originalText": "--target --pid --mount", + "lineHeight": 1.25, + "baseline": 11 + }, + { + "id": "r1510242733", + "type": "rectangle", + "x": 636, + "y": 66, + "width": 240, + "height": 70, + "angle": 0, + "strokeColor": "#999999", + "backgroundColor": "#ffffff", + "fillStyle": "solid", + "strokeWidth": 2, + "strokeStyle": "dashed", + "roughness": 1, + "opacity": 100, + "groupIds": [], + "frameId": null, + "roundness": { + "type": 3 + }, + "seed": 1591720123, + "versionNonce": 1924206979, + "isDeleted": false, + "boundElements": [], + "updated": 1, + "link": null, + "locked": false + }, + { + "id": "t809686007", + "type": "text", + "x": 646, + "y": 74, + "width": 200.20000000000002, + "height": 17.5, + "angle": 0, + "strokeColor": "#111111", + "backgroundColor": "transparent", + "fillStyle": "solid", + "strokeWidth": 1, + "strokeStyle": "solid", + "roughness": 1, + "opacity": 100, + "groupIds": [], + "frameId": null, + "roundness": null, + "seed": 1818753576, + "versionNonce": 185728640, + "isDeleted": false, + "boundElements": [], + "updated": 1, + "link": null, + "locked": false, + "text": "plain nsenter (no unshare)", + "fontSize": 14, + "fontFamily": 1, + "textAlign": "left", + "verticalAlign": "top", + "containerId": null, + "originalText": "plain nsenter (no unshare)", + "lineHeight": 1.25, + "baseline": 14 + }, + { + "id": "t879980112", + "type": "text", + "x": 646, + "y": 94, + "width": 151.25, + "height": 13.75, + "angle": 0, + "strokeColor": "#555555", + "backgroundColor": "transparent", + "fillStyle": "solid", + "strokeWidth": 1, + "strokeStyle": "solid", + "roughness": 1, + "opacity": 100, + "groupIds": [], + "frameId": null, + "roundness": null, + "seed": 1469187837, + "versionNonce": 50036903, + "isDeleted": false, + "boundElements": [], + "updated": 1, + "link": null, + "locked": false, + "text": "EPERM: reassociate failed", + "fontSize": 11, + "fontFamily": 1, + "textAlign": "left", + "verticalAlign": "top", + "containerId": null, + "originalText": "EPERM: reassociate failed", + "lineHeight": 1.25, + "baseline": 11 + }, + { + "id": "r747577458", + "type": "rectangle", + "x": 44, + "y": 230, + "width": 300, + "height": 50, + "angle": 0, + "strokeColor": "#e8870c", + "backgroundColor": "#fff8ef", + "fillStyle": "solid", + "strokeWidth": 2, + "strokeStyle": "solid", + "roughness": 1, + "opacity": 100, + "groupIds": [], + "frameId": null, + "roundness": { + "type": 3 + }, + "seed": 1219646401, + "versionNonce": 627599541, + "isDeleted": false, + "boundElements": [], + "updated": 1, + "link": null, + "locked": false + }, + { + "id": "t746248742", + "type": "text", + "x": 54, + "y": 238, + "width": 269.5, + "height": 17.5, + "angle": 0, + "strokeColor": "#111111", + "backgroundColor": "transparent", + "fillStyle": "solid", + "strokeWidth": 1, + "strokeStyle": "solid", + "roughness": 1, + "opacity": 100, + "groupIds": [], + "frameId": null, + "roundness": null, + "seed": 1255087381, + "versionNonce": 1663484594, + "isDeleted": false, + "boundElements": [], + "updated": 1, + "link": null, + "locked": false, + "text": "container PID 1 == host pid 2998602", + "fontSize": 14, + "fontFamily": 1, + "textAlign": "left", + "verticalAlign": "top", + "containerId": null, + "originalText": "container PID 1 == host pid 2998602", + "lineHeight": 1.25, + "baseline": 14 + }, + { + "id": "t1673484217", + "type": "text", + "x": 54, + "y": 258, + "width": 163.35000000000002, + "height": 13.75, + "angle": 0, + "strokeColor": "#555555", + "backgroundColor": "transparent", + "fillStyle": "solid", + "strokeWidth": 1, + "strokeStyle": "solid", + "roughness": 1, + "opacity": 100, + "groupIds": [], + "frameId": null, + "roundness": null, + "seed": 858019945, + "versionNonce": 629602551, + "isDeleted": false, + "boundElements": [], + "updated": 1, + "link": null, + "locked": false, + "text": "(podman inspect .State.Pid)", + "fontSize": 11, + "fontFamily": 1, + "textAlign": "left", + "verticalAlign": "top", + "containerId": null, + "originalText": "(podman inspect .State.Pid)", + "lineHeight": 1.25, + "baseline": 11 + }, + { + "id": "r648040562", + "type": "rectangle", + "x": 380, + "y": 230, + "width": 500, + "height": 50, + "angle": 0, + "strokeColor": "#111111", + "backgroundColor": "#111111", + "fillStyle": "solid", + "strokeWidth": 2, + "strokeStyle": "solid", + "roughness": 1, + "opacity": 100, + "groupIds": [], + "frameId": null, + "roundness": { + "type": 3 + }, + "seed": 30284613, + "versionNonce": 967259050, + "isDeleted": false, + "boundElements": [], + "updated": 1, + "link": null, + "locked": false + }, + { + "id": "t1070218664", + "type": "text", + "x": 390, + "y": 238, + "width": 585.2, + "height": 17.5, + "angle": 0, + "strokeColor": "#ffffff", + "backgroundColor": "transparent", + "fillStyle": "solid", + "strokeWidth": 1, + "strokeStyle": "solid", + "roughness": 1, + "opacity": 100, + "groupIds": [], + "frameId": null, + "roundness": null, + "seed": 719479071, + "versionNonce": 615785387, + "isDeleted": false, + "boundElements": [], + "updated": 1, + "link": null, + "locked": false, + "text": "Name: app State: S (sleeping) readlink /proc/1/exe -> /usr/local/bin/app", + "fontSize": 14, + "fontFamily": 1, + "textAlign": "left", + "verticalAlign": "top", + "containerId": null, + "originalText": "Name: app State: S (sleeping) readlink /proc/1/exe -> /usr/local/bin/app", + "lineHeight": 1.25, + "baseline": 14 + }, + { + "id": "r1121938148", + "type": "rectangle", + "x": 44, + "y": 300, + "width": 400, + "height": 60, + "angle": 0, + "strokeColor": "#111111", + "backgroundColor": "#ffffff", + "fillStyle": "solid", + "strokeWidth": 2, + "strokeStyle": "solid", + "roughness": 1, + "opacity": 100, + "groupIds": [], + "frameId": null, + "roundness": { + "type": 3 + }, + "seed": 1753251212, + "versionNonce": 871602564, + "isDeleted": false, + "boundElements": [], + "updated": 1, + "link": null, + "locked": false + }, + { + "id": "t526384492", + "type": "text", + "x": 54, + "y": 308, + "width": 200.20000000000002, + "height": 17.5, + "angle": 0, + "strokeColor": "#111111", + "backgroundColor": "transparent", + "fillStyle": "solid", + "strokeWidth": 1, + "strokeStyle": "solid", + "roughness": 1, + "opacity": 100, + "groupIds": [], + "frameId": null, + "roundness": null, + "seed": 378400194, + "versionNonce": 1056995331, + "isDeleted": false, + "boundElements": [], + "updated": 1, + "link": null, + "locked": false, + "text": "subuid rsedor:524288:65536", + "fontSize": 14, + "fontFamily": 1, + "textAlign": "left", + "verticalAlign": "top", + "containerId": null, + "originalText": "subuid rsedor:524288:65536", + "lineHeight": 1.25, + "baseline": 14 + }, + { + "id": "t905819966", + "type": "text", + "x": 54, + "y": 328, + "width": 235.95000000000002, + "height": 13.75, + "angle": 0, + "strokeColor": "#555555", + "backgroundColor": "transparent", + "fillStyle": "solid", + "strokeWidth": 1, + "strokeStyle": "solid", + "roughness": 1, + "opacity": 100, + "groupIds": [], + "frameId": null, + "roundness": null, + "seed": 185848706, + "versionNonce": 641007186, + "isDeleted": false, + "boundElements": [], + "updated": 1, + "link": null, + "locked": false, + "text": "uid_map: 0->1000 (1), 1->524288 (65536)", + "fontSize": 11, + "fontFamily": 1, + "textAlign": "left", + "verticalAlign": "top", + "containerId": null, + "originalText": "uid_map: 0->1000 (1), 1->524288 (65536)", + "lineHeight": 1.25, + "baseline": 11 + }, + { + "id": "r1677728677", + "type": "rectangle", + "x": 480, + "y": 300, + "width": 400, + "height": 60, + "angle": 0, + "strokeColor": "#999999", + "backgroundColor": "#ffffff", + "fillStyle": "solid", + "strokeWidth": 2, + "strokeStyle": "solid", + "roughness": 1, + "opacity": 100, + "groupIds": [], + "frameId": null, + "roundness": { + "type": 3 + }, + "seed": 1140645975, + "versionNonce": 8850942, + "isDeleted": false, + "boundElements": [], + "updated": 1, + "link": null, + "locked": false + }, + { + "id": "t1033813197", + "type": "text", + "x": 490, + "y": 308, + "width": 200.20000000000002, + "height": 17.5, + "angle": 0, + "strokeColor": "#111111", + "backgroundColor": "transparent", + "fillStyle": "solid", + "strokeWidth": 1, + "strokeStyle": "solid", + "roughness": 1, + "opacity": 100, + "groupIds": [], + "frameId": null, + "roundness": null, + "seed": 697629110, + "versionNonce": 79399132, + "isDeleted": false, + "boundElements": [], + "updated": 1, + "link": null, + "locked": false, + "text": "zombie scan (podman exec):", + "fontSize": 14, + "fontFamily": 1, + "textAlign": "left", + "verticalAlign": "top", + "containerId": null, + "originalText": "zombie scan (podman exec):", + "lineHeight": 1.25, + "baseline": 14 + }, + { + "id": "t73573208", + "type": "text", + "x": 490, + "y": 328, + "width": 211.75000000000003, + "height": 13.75, + "angle": 0, + "strokeColor": "#555555", + "backgroundColor": "transparent", + "fillStyle": "solid", + "strokeWidth": 1, + "strokeStyle": "solid", + "roughness": 1, + "opacity": 100, + "groupIds": [], + "frameId": null, + "roundness": null, + "seed": 784510758, + "versionNonce": 794686413, + "isDeleted": false, + "boundElements": [], + "updated": 1, + "link": null, + "locked": false, + "text": "/proc/[0-9]*/stat state==Z, count=0", + "fontSize": 11, + "fontFamily": 1, + "textAlign": "left", + "verticalAlign": "top", + "containerId": null, + "originalText": "/proc/[0-9]*/stat state==Z, count=0", + "lineHeight": 1.25, + "baseline": 11 + }, + { + "id": "a957661166", + "type": "arrow", + "x": 174, + "y": 136, + "width": 0, + "height": 94, + "angle": 0, + "strokeColor": "#b8650a", + "backgroundColor": "transparent", + "fillStyle": "solid", + "strokeWidth": 2, + "strokeStyle": "solid", + "roughness": 1, + "opacity": 100, + "groupIds": [], + "frameId": null, + "roundness": { + "type": 2 + }, + "seed": 1884038424, + "versionNonce": 1494285026, + "isDeleted": false, + "boundElements": [], + "updated": 1, + "link": null, + "locked": false, + "points": [ + [ + 0, + 0 + ], + [ + 0, + 94 + ] + ], + "lastCommittedPoint": null, + "startBinding": null, + "endBinding": null, + "startArrowhead": null, + "endArrowhead": "arrow" + }, + { + "id": "t1680353426", + "type": "text", + "x": 174.0, + "y": 169.0, + "width": 169.4, + "height": 13.75, + "angle": 0, + "strokeColor": "#b8650a", + "backgroundColor": "transparent", + "fillStyle": "solid", + "strokeWidth": 1, + "strokeStyle": "solid", + "roughness": 1, + "opacity": 100, + "groupIds": [], + "frameId": null, + "roundness": null, + "seed": 1631809376, + "versionNonce": 1059717701, + "isDeleted": false, + "boundElements": [], + "updated": 1, + "link": null, + "locked": false, + "text": "enters via container runtime", + "fontSize": 11, + "fontFamily": 1, + "textAlign": "left", + "verticalAlign": "top", + "containerId": null, + "originalText": "enters via container runtime", + "lineHeight": 1.25, + "baseline": 11 + }, + { + "id": "a931447477", + "type": "arrow", + "x": 470, + "y": 136, + "width": 0, + "height": 94, + "angle": 0, + "strokeColor": "#b8650a", + "backgroundColor": "transparent", + "fillStyle": "solid", + "strokeWidth": 2, + "strokeStyle": "solid", + "roughness": 1, + "opacity": 100, + "groupIds": [], + "frameId": null, + "roundness": { + "type": 2 + }, + "seed": 383318077, + "versionNonce": 946420661, + "isDeleted": false, + "boundElements": [], + "updated": 1, + "link": null, + "locked": false, + "points": [ + [ + 0, + 0 + ], + [ + 0, + 94 + ] + ], + "lastCommittedPoint": null, + "startBinding": null, + "endBinding": null, + "startArrowhead": null, + "endArrowhead": "arrow" + }, + { + "id": "t1638767495", + "type": "text", + "x": 470.0, + "y": 169.0, + "width": 199.65, + "height": 13.75, + "angle": 0, + "strokeColor": "#b8650a", + "backgroundColor": "transparent", + "fillStyle": "solid", + "strokeWidth": 1, + "strokeStyle": "solid", + "roughness": 1, + "opacity": 100, + "groupIds": [], + "frameId": null, + "roundness": null, + "seed": 778221152, + "versionNonce": 934531792, + "isDeleted": false, + "boundElements": [], + "updated": 1, + "link": null, + "locked": false, + "text": "podman unshare joins userns first", + "fontSize": 11, + "fontFamily": 1, + "textAlign": "left", + "verticalAlign": "top", + "containerId": null, + "originalText": "podman unshare joins userns first", + "lineHeight": 1.25, + "baseline": 11 + }, + { + "id": "a627444143", + "type": "arrow", + "x": 636, + "y": 101, + "width": 36, + "height": 0, + "angle": 0, + "strokeColor": "#555555", + "backgroundColor": "transparent", + "fillStyle": "solid", + "strokeWidth": 2, + "strokeStyle": "dashed", + "roughness": 1, + "opacity": 100, + "groupIds": [], + "frameId": null, + "roundness": { + "type": 2 + }, + "seed": 1730303836, + "versionNonce": 566611761, + "isDeleted": false, + "boundElements": [], + "updated": 1, + "link": null, + "locked": false, + "points": [ + [ + 0, + 0 + ], + [ + -36, + 0 + ] + ], + "lastCommittedPoint": null, + "startBinding": null, + "endBinding": null, + "startArrowhead": null, + "endArrowhead": "arrow" + }, + { + "id": "a383196384", + "type": "arrow", + "x": 244, + "y": 280, + "width": 136, + "height": 25, + "angle": 0, + "strokeColor": "#555555", + "backgroundColor": "transparent", + "fillStyle": "solid", + "strokeWidth": 2, + "strokeStyle": "solid", + "roughness": 1, + "opacity": 100, + "groupIds": [], + "frameId": null, + "roundness": { + "type": 2 + }, + "seed": 809265471, + "versionNonce": 1505654928, + "isDeleted": false, + "boundElements": [], + "updated": 1, + "link": null, + "locked": false, + "points": [ + [ + 0, + 0 + ], + [ + 136, + -25 + ] + ], + "lastCommittedPoint": null, + "startBinding": null, + "endBinding": null, + "startArrowhead": null, + "endArrowhead": "arrow" + }, + { + "id": "a1011797615", + "type": "arrow", + "x": 630, + "y": 280, + "width": 0, + "height": 20, + "angle": 0, + "strokeColor": "#555555", + "backgroundColor": "transparent", + "fillStyle": "solid", + "strokeWidth": 2, + "strokeStyle": "dashed", + "roughness": 1, + "opacity": 100, + "groupIds": [], + "frameId": null, + "roundness": { + "type": 2 + }, + "seed": 311128184, + "versionNonce": 1982202249, + "isDeleted": false, + "boundElements": [], + "updated": 1, + "link": null, + "locked": false, + "points": [ + [ + 0, + 0 + ], + [ + 0, + 20 + ] + ], + "lastCommittedPoint": null, + "startBinding": null, + "endBinding": null, + "startArrowhead": null, + "endArrowhead": "arrow" + }, + { + "id": "t1149595863", + "type": "text", + "x": 44, + "y": 390, + "width": 958.1, + "height": 16.25, + "angle": 0, + "strokeColor": "#555555", + "backgroundColor": "transparent", + "fillStyle": "solid", + "strokeWidth": 1, + "strokeStyle": "solid", + "roughness": 1, + "opacity": 100, + "groupIds": [], + "frameId": null, + "roundness": null, + "seed": 1524482370, + "versionNonce": 1014940981, + "isDeleted": false, + "boundElements": [], + "updated": 1, + "link": null, + "locked": false, + "text": "Both paths independently confirm the same PID 1 identity -- podman exec from inside the container runtime, nsenter from the bare host.", + "fontSize": 13, + "fontFamily": 1, + "textAlign": "left", + "verticalAlign": "top", + "containerId": null, + "originalText": "Both paths independently confirm the same PID 1 identity -- podman exec from inside the container runtime, nsenter from the bare host.", + "lineHeight": 1.25, + "baseline": 13 + } + ], + "appState": { + "viewBackgroundColor": "#ffffff", + "gridSize": null + }, + "files": {} +} \ No newline at end of file diff --git a/assets/diagrams/34-ns-debugging-map.svg b/assets/diagrams/34-ns-debugging-map.svg new file mode 100644 index 0000000..f364e4e --- /dev/null +++ b/assets/diagrams/34-ns-debugging-map.svg @@ -0,0 +1,37 @@ + + + +host (rootless user rsedor, uid 1000) + +podman's user namespace (uid 0 in container == uid 1000 on host) + +podman exec demo34go +cat /proc/1/status + +podman unshare nsenter +--target <hostpid> --pid --mount + +plain nsenter (no unshare) +EPERM: reassociate failed + +container PID 1 == host pid 2998602 +(podman inspect .State.Pid) + +Name: app State: S (sleeping) readlink /proc/1/exe -> /usr/local/bin/app + +subuid rsedor:524288:65536 +uid_map: 0->1000 (1), 1->524288 (65536) + +zombie scan (podman exec): +/proc/[0-9]*/stat state==Z, count=0 + +enters via container runtime +enters via container runtime + +podman unshare joins userns first +podman unshare joins userns first + + + +Both paths independently confirm the same PID 1 identity -- podman exec from inside the container runtime, nsenter from the bare host. + \ No newline at end of file diff --git a/assets/diagrams/34-pid1-signal-flow.excalidraw b/assets/diagrams/34-pid1-signal-flow.excalidraw new file mode 100644 index 0000000..79711ac --- /dev/null +++ b/assets/diagrams/34-pid1-signal-flow.excalidraw @@ -0,0 +1,1357 @@ +{ + "type": "excalidraw", + "version": 2, + "source": "https://excalidraw.com", + "elements": [ + { + "id": "r1520545838", + "type": "rectangle", + "x": 20, + "y": 30, + "width": 430, + "height": 380, + "angle": 0, + "strokeColor": "#dddddd", + "backgroundColor": "#fff8ef", + "fillStyle": "solid", + "strokeWidth": 2, + "strokeStyle": "solid", + "roughness": 1, + "opacity": 100, + "groupIds": [], + "frameId": null, + "roundness": { + "type": 3 + }, + "seed": 1613957608, + "versionNonce": 1027639434, + "isDeleted": false, + "boundElements": [], + "updated": 1, + "link": null, + "locked": false + }, + { + "id": "t1145928858", + "type": "text", + "x": 32, + "y": 36, + "width": 165.0, + "height": 15.0, + "angle": 0, + "strokeColor": "#555555", + "backgroundColor": "transparent", + "fillStyle": "solid", + "strokeWidth": 1, + "strokeStyle": "solid", + "roughness": 1, + "opacity": 100, + "groupIds": [], + "frameId": null, + "roundness": null, + "seed": 1311549483, + "versionNonce": 608772552, + "isDeleted": false, + "boundElements": [], + "updated": 1, + "link": null, + "locked": false, + "text": "serve (installs handlers)", + "fontSize": 12, + "fontFamily": 1, + "textAlign": "left", + "verticalAlign": "top", + "containerId": null, + "originalText": "serve (installs handlers)", + "lineHeight": 1.25, + "baseline": 12 + }, + { + "id": "r305201436", + "type": "rectangle", + "x": 490, + "y": 30, + "width": 430, + "height": 380, + "angle": 0, + "strokeColor": "#dddddd", + "backgroundColor": "#f4f4f4", + "fillStyle": "solid", + "strokeWidth": 2, + "strokeStyle": "solid", + "roughness": 1, + "opacity": 100, + "groupIds": [], + "frameId": null, + "roundness": { + "type": 3 + }, + "seed": 1088040156, + "versionNonce": 146560674, + "isDeleted": false, + "boundElements": [], + "updated": 1, + "link": null, + "locked": false + }, + { + "id": "t860673614", + "type": "text", + "x": 502, + "y": 36, + "width": 138.60000000000002, + "height": 15.0, + "angle": 0, + "strokeColor": "#555555", + "backgroundColor": "transparent", + "fillStyle": "solid", + "strokeWidth": 1, + "strokeStyle": "solid", + "roughness": 1, + "opacity": 100, + "groupIds": [], + "frameId": null, + "roundness": null, + "seed": 1194067905, + "versionNonce": 52353770, + "isDeleted": false, + "boundElements": [], + "updated": 1, + "link": null, + "locked": false, + "text": "naive (installs none)", + "fontSize": 12, + "fontFamily": 1, + "textAlign": "left", + "verticalAlign": "top", + "containerId": null, + "originalText": "naive (installs none)", + "lineHeight": 1.25, + "baseline": 12 + }, + { + "id": "r8966384", + "type": "rectangle", + "x": 44, + "y": 66, + "width": 380, + "height": 40, + "angle": 0, + "strokeColor": "#111111", + "backgroundColor": "#111111", + "fillStyle": "solid", + "strokeWidth": 2, + "strokeStyle": "solid", + "roughness": 1, + "opacity": 100, + "groupIds": [], + "frameId": null, + "roundness": { + "type": 3 + }, + "seed": 22887737, + "versionNonce": 265039205, + "isDeleted": false, + "boundElements": [], + "updated": 1, + "link": null, + "locked": false + }, + { + "id": "t1819599735", + "type": "text", + "x": 54, + "y": 74, + "width": 292.6, + "height": 17.5, + "angle": 0, + "strokeColor": "#ffffff", + "backgroundColor": "transparent", + "fillStyle": "solid", + "strokeWidth": 1, + "strokeStyle": "solid", + "roughness": 1, + "opacity": 100, + "groupIds": [], + "frameId": null, + "roundness": null, + "seed": 432892946, + "versionNonce": 1729870130, + "isDeleted": false, + "boundElements": [], + "updated": 1, + "link": null, + "locked": false, + "text": "podman stop -t 3 -> SIGTERM to PID 1", + "fontSize": 14, + "fontFamily": 1, + "textAlign": "left", + "verticalAlign": "top", + "containerId": null, + "originalText": "podman stop -t 3 -> SIGTERM to PID 1", + "lineHeight": 1.25, + "baseline": 14 + }, + { + "id": "r20511555", + "type": "rectangle", + "x": 44, + "y": 122, + "width": 380, + "height": 50, + "angle": 0, + "strokeColor": "#111111", + "backgroundColor": "#ffffff", + "fillStyle": "solid", + "strokeWidth": 2, + "strokeStyle": "solid", + "roughness": 1, + "opacity": 100, + "groupIds": [], + "frameId": null, + "roundness": { + "type": 3 + }, + "seed": 862423176, + "versionNonce": 745151017, + "isDeleted": false, + "boundElements": [], + "updated": 1, + "link": null, + "locked": false + }, + { + "id": "t1974530789", + "type": "text", + "x": 54, + "y": 130, + "width": 223.3, + "height": 17.5, + "angle": 0, + "strokeColor": "#111111", + "backgroundColor": "transparent", + "fillStyle": "solid", + "strokeWidth": 1, + "strokeStyle": "solid", + "roughness": 1, + "opacity": 100, + "groupIds": [], + "frameId": null, + "roundness": null, + "seed": 1985428442, + "versionNonce": 64316786, + "isDeleted": false, + "boundElements": [], + "updated": 1, + "link": null, + "locked": false, + "text": "signalfd/Notify/Signals wakes", + "fontSize": 14, + "fontFamily": 1, + "textAlign": "left", + "verticalAlign": "top", + "containerId": null, + "originalText": "signalfd/Notify/Signals wakes", + "lineHeight": 1.25, + "baseline": 14 + }, + { + "id": "t1586585119", + "type": "text", + "x": 54, + "y": 150, + "width": 157.3, + "height": 13.75, + "angle": 0, + "strokeColor": "#555555", + "backgroundColor": "transparent", + "fillStyle": "solid", + "strokeWidth": 1, + "strokeStyle": "solid", + "roughness": 1, + "opacity": 100, + "groupIds": [], + "frameId": null, + "roundness": null, + "seed": 1809216265, + "versionNonce": 606204818, + "isDeleted": false, + "boundElements": [], + "updated": 1, + "link": null, + "locked": false, + "text": "job_spawner.request_stop()", + "fontSize": 11, + "fontFamily": 1, + "textAlign": "left", + "verticalAlign": "top", + "containerId": null, + "originalText": "job_spawner.request_stop()", + "lineHeight": 1.25, + "baseline": 11 + }, + { + "id": "r821120127", + "type": "rectangle", + "x": 44, + "y": 188, + "width": 380, + "height": 50, + "angle": 0, + "strokeColor": "#111111", + "backgroundColor": "#ffffff", + "fillStyle": "solid", + "strokeWidth": 2, + "strokeStyle": "solid", + "roughness": 1, + "opacity": 100, + "groupIds": [], + "frameId": null, + "roundness": { + "type": 3 + }, + "seed": 1050232528, + "versionNonce": 734400353, + "isDeleted": false, + "boundElements": [], + "updated": 1, + "link": null, + "locked": false + }, + { + "id": "t1878927192", + "type": "text", + "x": 54, + "y": 196, + "width": 284.90000000000003, + "height": 17.5, + "angle": 0, + "strokeColor": "#111111", + "backgroundColor": "transparent", + "fillStyle": "solid", + "strokeWidth": 1, + "strokeStyle": "solid", + "roughness": 1, + "opacity": 100, + "groupIds": [], + "frameId": null, + "roundness": null, + "seed": 278686536, + "versionNonce": 1266264421, + "isDeleted": false, + "boundElements": [], + "updated": 1, + "link": null, + "locked": false, + "text": "kill(worker, SIGTERM); waitpid/Wait()", + "fontSize": 14, + "fontFamily": 1, + "textAlign": "left", + "verticalAlign": "top", + "containerId": null, + "originalText": "kill(worker, SIGTERM); waitpid/Wait()", + "lineHeight": 1.25, + "baseline": 14 + }, + { + "id": "t1958562336", + "type": "text", + "x": 54, + "y": 216, + "width": 78.65, + "height": 13.75, + "angle": 0, + "strokeColor": "#555555", + "backgroundColor": "transparent", + "fillStyle": "solid", + "strokeWidth": 1, + "strokeStyle": "solid", + "roughness": 1, + "opacity": 100, + "groupIds": [], + "frameId": null, + "roundness": null, + "seed": 455932739, + "versionNonce": 787409879, + "isDeleted": false, + "boundElements": [], + "updated": 1, + "link": null, + "locked": false, + "text": "worker reaped", + "fontSize": 11, + "fontFamily": 1, + "textAlign": "left", + "verticalAlign": "top", + "containerId": null, + "originalText": "worker reaped", + "lineHeight": 1.25, + "baseline": 11 + }, + { + "id": "r1227831983", + "type": "rectangle", + "x": 44, + "y": 254, + "width": 380, + "height": 40, + "angle": 0, + "strokeColor": "#e8870c", + "backgroundColor": "#fff8ef", + "fillStyle": "solid", + "strokeWidth": 2, + "strokeStyle": "solid", + "roughness": 1, + "opacity": 100, + "groupIds": [], + "frameId": null, + "roundness": { + "type": 3 + }, + "seed": 325216464, + "versionNonce": 120948648, + "isDeleted": false, + "boundElements": [], + "updated": 1, + "link": null, + "locked": false + }, + { + "id": "t630890672", + "type": "text", + "x": 54, + "y": 262, + "width": 277.20000000000005, + "height": 17.5, + "angle": 0, + "strokeColor": "#111111", + "backgroundColor": "transparent", + "fillStyle": "solid", + "strokeWidth": 1, + "strokeStyle": "solid", + "roughness": 1, + "opacity": 100, + "groupIds": [], + "frameId": null, + "roundness": null, + "seed": 1379778097, + "versionNonce": 41210891, + "isDeleted": false, + "boundElements": [], + "updated": 1, + "link": null, + "locked": false, + "text": "print \"app: shutting down (SIGTERM)\"", + "fontSize": 14, + "fontFamily": 1, + "textAlign": "left", + "verticalAlign": "top", + "containerId": null, + "originalText": "print \"app: shutting down (SIGTERM)\"", + "lineHeight": 1.25, + "baseline": 14 + }, + { + "id": "r72303485", + "type": "rectangle", + "x": 44, + "y": 310, + "width": 380, + "height": 60, + "angle": 0, + "strokeColor": "#e8870c", + "backgroundColor": "#fff8ef", + "fillStyle": "solid", + "strokeWidth": 2, + "strokeStyle": "solid", + "roughness": 1, + "opacity": 100, + "groupIds": [], + "frameId": null, + "roundness": { + "type": 3 + }, + "seed": 1692750386, + "versionNonce": 1460807906, + "isDeleted": false, + "boundElements": [], + "updated": 1, + "link": null, + "locked": false + }, + { + "id": "t1158048643", + "type": "text", + "x": 54, + "y": 318, + "width": 385.00000000000006, + "height": 17.5, + "angle": 0, + "strokeColor": "#111111", + "backgroundColor": "transparent", + "fillStyle": "solid", + "strokeWidth": 1, + "strokeStyle": "solid", + "roughness": 1, + "opacity": 100, + "groupIds": [], + "frameId": null, + "roundness": null, + "seed": 1427601340, + "versionNonce": 1939676152, + "isDeleted": false, + "boundElements": [], + "updated": 1, + "link": null, + "locked": false, + "text": "exit 0 -- measured: cpp 192ms, go 60ms, rust 101ms", + "fontSize": 14, + "fontFamily": 1, + "textAlign": "left", + "verticalAlign": "top", + "containerId": null, + "originalText": "exit 0 -- measured: cpp 192ms, go 60ms, rust 101ms", + "lineHeight": 1.25, + "baseline": 14 + }, + { + "id": "t939037511", + "type": "text", + "x": 54, + "y": 338, + "width": 127.05000000000001, + "height": 13.75, + "angle": 0, + "strokeColor": "#555555", + "backgroundColor": "transparent", + "fillStyle": "solid", + "strokeWidth": 1, + "strokeStyle": "solid", + "roughness": 1, + "opacity": 100, + "groupIds": [], + "frameId": null, + "roundness": null, + "seed": 1475237294, + "versionNonce": 1986263276, + "isDeleted": false, + "boundElements": [], + "updated": 1, + "link": null, + "locked": false, + "text": "(3000ms grace budget)", + "fontSize": 11, + "fontFamily": 1, + "textAlign": "left", + "verticalAlign": "top", + "containerId": null, + "originalText": "(3000ms grace budget)", + "lineHeight": 1.25, + "baseline": 11 + }, + { + "id": "r1449055490", + "type": "rectangle", + "x": 514, + "y": 66, + "width": 380, + "height": 40, + "angle": 0, + "strokeColor": "#111111", + "backgroundColor": "#111111", + "fillStyle": "solid", + "strokeWidth": 2, + "strokeStyle": "solid", + "roughness": 1, + "opacity": 100, + "groupIds": [], + "frameId": null, + "roundness": { + "type": 3 + }, + "seed": 896838793, + "versionNonce": 1537003995, + "isDeleted": false, + "boundElements": [], + "updated": 1, + "link": null, + "locked": false + }, + { + "id": "t344963807", + "type": "text", + "x": 524, + "y": 74, + "width": 292.6, + "height": 17.5, + "angle": 0, + "strokeColor": "#ffffff", + "backgroundColor": "transparent", + "fillStyle": "solid", + "strokeWidth": 1, + "strokeStyle": "solid", + "roughness": 1, + "opacity": 100, + "groupIds": [], + "frameId": null, + "roundness": null, + "seed": 1928735533, + "versionNonce": 946110963, + "isDeleted": false, + "boundElements": [], + "updated": 1, + "link": null, + "locked": false, + "text": "podman stop -t 2 -> SIGTERM to PID 1", + "fontSize": 14, + "fontFamily": 1, + "textAlign": "left", + "verticalAlign": "top", + "containerId": null, + "originalText": "podman stop -t 2 -> SIGTERM to PID 1", + "lineHeight": 1.25, + "baseline": 14 + }, + { + "id": "r1162817797", + "type": "rectangle", + "x": 514, + "y": 122, + "width": 380, + "height": 70, + "angle": 0, + "strokeColor": "#999999", + "backgroundColor": "#ffffff", + "fillStyle": "solid", + "strokeWidth": 2, + "strokeStyle": "dashed", + "roughness": 1, + "opacity": 100, + "groupIds": [], + "frameId": null, + "roundness": { + "type": 3 + }, + "seed": 725718221, + "versionNonce": 656644678, + "isDeleted": false, + "boundElements": [], + "updated": 1, + "link": null, + "locked": false + }, + { + "id": "t1867458436", + "type": "text", + "x": 524, + "y": 130, + "width": 346.5, + "height": 17.5, + "angle": 0, + "strokeColor": "#111111", + "backgroundColor": "transparent", + "fillStyle": "solid", + "strokeWidth": 1, + "strokeStyle": "solid", + "roughness": 1, + "opacity": 100, + "groupIds": [], + "frameId": null, + "roundness": null, + "seed": 1895659493, + "versionNonce": 1451263192, + "isDeleted": false, + "boundElements": [], + "updated": 1, + "link": null, + "locked": false, + "text": "kernel: PID 1 has no explicit SIGTERM handler", + "fontSize": 14, + "fontFamily": 1, + "textAlign": "left", + "verticalAlign": "top", + "containerId": null, + "originalText": "kernel: PID 1 has no explicit SIGTERM handler", + "lineHeight": 1.25, + "baseline": 14 + }, + { + "id": "t1725667596", + "type": "text", + "x": 524, + "y": 150, + "width": 290.40000000000003, + "height": 13.75, + "angle": 0, + "strokeColor": "#555555", + "backgroundColor": "transparent", + "fillStyle": "solid", + "strokeWidth": 1, + "strokeStyle": "solid", + "roughness": 1, + "opacity": 100, + "groupIds": [], + "frameId": null, + "roundness": null, + "seed": 1376764895, + "versionNonce": 1596838843, + "isDeleted": false, + "boundElements": [], + "updated": 1, + "link": null, + "locked": false, + "text": "-> default action (terminate) is masked, not run", + "fontSize": 11, + "fontFamily": 1, + "textAlign": "left", + "verticalAlign": "top", + "containerId": null, + "originalText": "-> default action (terminate) is masked, not run", + "lineHeight": 1.25, + "baseline": 11 + }, + { + "id": "t1061283216", + "type": "text", + "x": 524, + "y": 165, + "width": 205.70000000000002, + "height": 13.75, + "angle": 0, + "strokeColor": "#555555", + "backgroundColor": "transparent", + "fillStyle": "solid", + "strokeWidth": 1, + "strokeStyle": "solid", + "roughness": 1, + "opacity": 100, + "groupIds": [], + "frameId": null, + "roundness": null, + "seed": 299792649, + "versionNonce": 124244701, + "isDeleted": false, + "boundElements": [], + "updated": 1, + "link": null, + "locked": false, + "text": "cpp/rust: signal is simply dropped", + "fontSize": 11, + "fontFamily": 1, + "textAlign": "left", + "verticalAlign": "top", + "containerId": null, + "originalText": "cpp/rust: signal is simply dropped", + "lineHeight": 1.25, + "baseline": 11 + }, + { + "id": "r427506165", + "type": "rectangle", + "x": 514, + "y": 210, + "width": 380, + "height": 40, + "angle": 0, + "strokeColor": "#999999", + "backgroundColor": "#ffffff", + "fillStyle": "solid", + "strokeWidth": 2, + "strokeStyle": "solid", + "roughness": 1, + "opacity": 100, + "groupIds": [], + "frameId": null, + "roundness": { + "type": 3 + }, + "seed": 934935555, + "versionNonce": 1639359946, + "isDeleted": false, + "boundElements": [], + "updated": 1, + "link": null, + "locked": false + }, + { + "id": "t1455076607", + "type": "text", + "x": 524, + "y": 218, + "width": 392.70000000000005, + "height": 17.5, + "angle": 0, + "strokeColor": "#111111", + "backgroundColor": "transparent", + "fillStyle": "solid", + "strokeWidth": 1, + "strokeStyle": "solid", + "roughness": 1, + "opacity": 100, + "groupIds": [], + "frameId": null, + "roundness": null, + "seed": 577879941, + "versionNonce": 1648268907, + "isDeleted": false, + "boundElements": [], + "updated": 1, + "link": null, + "locked": false, + "text": "Go exception: runtime installs its own fatal-signal", + "fontSize": 14, + "fontFamily": 1, + "textAlign": "left", + "verticalAlign": "top", + "containerId": null, + "originalText": "Go exception: runtime installs its own fatal-signal", + "lineHeight": 1.25, + "baseline": 14 + }, + { + "id": "r1829071013", + "type": "rectangle", + "x": 514, + "y": 256, + "width": 380, + "height": 40, + "angle": 0, + "strokeColor": "#999999", + "backgroundColor": "#ffffff", + "fillStyle": "solid", + "strokeWidth": 2, + "strokeStyle": "solid", + "roughness": 1, + "opacity": 100, + "groupIds": [], + "frameId": null, + "roundness": { + "type": 3 + }, + "seed": 626982873, + "versionNonce": 1838884700, + "isDeleted": false, + "boundElements": [], + "updated": 1, + "link": null, + "locked": false + }, + { + "id": "t1127930094", + "type": "text", + "x": 524, + "y": 264, + "width": 354.20000000000005, + "height": 17.5, + "angle": 0, + "strokeColor": "#111111", + "backgroundColor": "transparent", + "fillStyle": "solid", + "strokeWidth": 1, + "strokeStyle": "solid", + "roughness": 1, + "opacity": 100, + "groupIds": [], + "frameId": null, + "roundness": null, + "seed": 1300257956, + "versionNonce": 1113492787, + "isDeleted": false, + "boundElements": [], + "updated": 1, + "link": null, + "locked": false, + "text": "handler at startup regardless of signal.Notify", + "fontSize": 14, + "fontFamily": 1, + "textAlign": "left", + "verticalAlign": "top", + "containerId": null, + "originalText": "handler at startup regardless of signal.Notify", + "lineHeight": 1.25, + "baseline": 14 + }, + { + "id": "r763674718", + "type": "rectangle", + "x": 514, + "y": 312, + "width": 380, + "height": 60, + "angle": 0, + "strokeColor": "#111111", + "backgroundColor": "#ffffff", + "fillStyle": "solid", + "strokeWidth": 2, + "strokeStyle": "solid", + "roughness": 1, + "opacity": 100, + "groupIds": [], + "frameId": null, + "roundness": { + "type": 3 + }, + "seed": 1537736281, + "versionNonce": 16458482, + "isDeleted": false, + "boundElements": [], + "updated": 1, + "link": null, + "locked": false + }, + { + "id": "t158309405", + "type": "text", + "x": 524, + "y": 320, + "width": 438.90000000000003, + "height": 17.5, + "angle": 0, + "strokeColor": "#111111", + "backgroundColor": "transparent", + "fillStyle": "solid", + "strokeWidth": 1, + "strokeStyle": "solid", + "roughness": 1, + "opacity": 100, + "groupIds": [], + "frameId": null, + "roundness": null, + "seed": 1239500419, + "versionNonce": 828916654, + "isDeleted": false, + "boundElements": [], + "updated": 1, + "link": null, + "locked": false, + "text": "measured: cpp 2108ms, rust 2074ms (full grace, SIGKILLed)", + "fontSize": 14, + "fontFamily": 1, + "textAlign": "left", + "verticalAlign": "top", + "containerId": null, + "originalText": "measured: cpp 2108ms, rust 2074ms (full grace, SIGKILLed)", + "lineHeight": 1.25, + "baseline": 14 + }, + { + "id": "t1126778842", + "type": "text", + "x": 524, + "y": 340, + "width": 344.85, + "height": 13.75, + "angle": 0, + "strokeColor": "#555555", + "backgroundColor": "transparent", + "fillStyle": "solid", + "strokeWidth": 1, + "strokeStyle": "solid", + "roughness": 1, + "opacity": 100, + "groupIds": [], + "frameId": null, + "roundness": null, + "seed": 779073315, + "versionNonce": 1370597291, + "isDeleted": false, + "boundElements": [], + "updated": 1, + "link": null, + "locked": false, + "text": "go 60ms (dies anyway -- but no shutdown line, no cleanup)", + "fontSize": 11, + "fontFamily": 1, + "textAlign": "left", + "verticalAlign": "top", + "containerId": null, + "originalText": "go 60ms (dies anyway -- but no shutdown line, no cleanup)", + "lineHeight": 1.25, + "baseline": 11 + }, + { + "id": "a1745795989", + "type": "arrow", + "x": 234, + "y": 106, + "width": 0, + "height": 16, + "angle": 0, + "strokeColor": "#b8650a", + "backgroundColor": "transparent", + "fillStyle": "solid", + "strokeWidth": 2, + "strokeStyle": "solid", + "roughness": 1, + "opacity": 100, + "groupIds": [], + "frameId": null, + "roundness": { + "type": 2 + }, + "seed": 1290034759, + "versionNonce": 1953041790, + "isDeleted": false, + "boundElements": [], + "updated": 1, + "link": null, + "locked": false, + "points": [ + [ + 0, + 0 + ], + [ + 0, + 16 + ] + ], + "lastCommittedPoint": null, + "startBinding": null, + "endBinding": null, + "startArrowhead": null, + "endArrowhead": "arrow" + }, + { + "id": "a1872636671", + "type": "arrow", + "x": 234, + "y": 172, + "width": 0, + "height": 16, + "angle": 0, + "strokeColor": "#b8650a", + "backgroundColor": "transparent", + "fillStyle": "solid", + "strokeWidth": 2, + "strokeStyle": "solid", + "roughness": 1, + "opacity": 100, + "groupIds": [], + "frameId": null, + "roundness": { + "type": 2 + }, + "seed": 1584047340, + "versionNonce": 1774763880, + "isDeleted": false, + "boundElements": [], + "updated": 1, + "link": null, + "locked": false, + "points": [ + [ + 0, + 0 + ], + [ + 0, + 16 + ] + ], + "lastCommittedPoint": null, + "startBinding": null, + "endBinding": null, + "startArrowhead": null, + "endArrowhead": "arrow" + }, + { + "id": "a47723997", + "type": "arrow", + "x": 234, + "y": 238, + "width": 0, + "height": 16, + "angle": 0, + "strokeColor": "#b8650a", + "backgroundColor": "transparent", + "fillStyle": "solid", + "strokeWidth": 2, + "strokeStyle": "solid", + "roughness": 1, + "opacity": 100, + "groupIds": [], + "frameId": null, + "roundness": { + "type": 2 + }, + "seed": 42634620, + "versionNonce": 278870020, + "isDeleted": false, + "boundElements": [], + "updated": 1, + "link": null, + "locked": false, + "points": [ + [ + 0, + 0 + ], + [ + 0, + 16 + ] + ], + "lastCommittedPoint": null, + "startBinding": null, + "endBinding": null, + "startArrowhead": null, + "endArrowhead": "arrow" + }, + { + "id": "a541193695", + "type": "arrow", + "x": 234, + "y": 294, + "width": 0, + "height": 16, + "angle": 0, + "strokeColor": "#b8650a", + "backgroundColor": "transparent", + "fillStyle": "solid", + "strokeWidth": 2, + "strokeStyle": "solid", + "roughness": 1, + "opacity": 100, + "groupIds": [], + "frameId": null, + "roundness": { + "type": 2 + }, + "seed": 1033456206, + "versionNonce": 475659152, + "isDeleted": false, + "boundElements": [], + "updated": 1, + "link": null, + "locked": false, + "points": [ + [ + 0, + 0 + ], + [ + 0, + 16 + ] + ], + "lastCommittedPoint": null, + "startBinding": null, + "endBinding": null, + "startArrowhead": null, + "endArrowhead": "arrow" + }, + { + "id": "a172896794", + "type": "arrow", + "x": 704, + "y": 106, + "width": 0, + "height": 16, + "angle": 0, + "strokeColor": "#555555", + "backgroundColor": "transparent", + "fillStyle": "solid", + "strokeWidth": 2, + "strokeStyle": "solid", + "roughness": 1, + "opacity": 100, + "groupIds": [], + "frameId": null, + "roundness": { + "type": 2 + }, + "seed": 655215017, + "versionNonce": 957388647, + "isDeleted": false, + "boundElements": [], + "updated": 1, + "link": null, + "locked": false, + "points": [ + [ + 0, + 0 + ], + [ + 0, + 16 + ] + ], + "lastCommittedPoint": null, + "startBinding": null, + "endBinding": null, + "startArrowhead": null, + "endArrowhead": "arrow" + }, + { + "id": "a1166974718", + "type": "arrow", + "x": 704, + "y": 192, + "width": 0, + "height": 18, + "angle": 0, + "strokeColor": "#555555", + "backgroundColor": "transparent", + "fillStyle": "solid", + "strokeWidth": 2, + "strokeStyle": "solid", + "roughness": 1, + "opacity": 100, + "groupIds": [], + "frameId": null, + "roundness": { + "type": 2 + }, + "seed": 937430114, + "versionNonce": 440555076, + "isDeleted": false, + "boundElements": [], + "updated": 1, + "link": null, + "locked": false, + "points": [ + [ + 0, + 0 + ], + [ + 0, + 18 + ] + ], + "lastCommittedPoint": null, + "startBinding": null, + "endBinding": null, + "startArrowhead": null, + "endArrowhead": "arrow" + }, + { + "id": "a1096011032", + "type": "arrow", + "x": 704, + "y": 250, + "width": 0, + "height": 6, + "angle": 0, + "strokeColor": "#555555", + "backgroundColor": "transparent", + "fillStyle": "solid", + "strokeWidth": 2, + "strokeStyle": "solid", + "roughness": 1, + "opacity": 100, + "groupIds": [], + "frameId": null, + "roundness": { + "type": 2 + }, + "seed": 1798255687, + "versionNonce": 639222810, + "isDeleted": false, + "boundElements": [], + "updated": 1, + "link": null, + "locked": false, + "points": [ + [ + 0, + 0 + ], + [ + 0, + 6 + ] + ], + "lastCommittedPoint": null, + "startBinding": null, + "endBinding": null, + "startArrowhead": null, + "endArrowhead": "arrow" + }, + { + "id": "a1874746224", + "type": "arrow", + "x": 704, + "y": 296, + "width": 0, + "height": 16, + "angle": 0, + "strokeColor": "#555555", + "backgroundColor": "transparent", + "fillStyle": "solid", + "strokeWidth": 2, + "strokeStyle": "solid", + "roughness": 1, + "opacity": 100, + "groupIds": [], + "frameId": null, + "roundness": { + "type": 2 + }, + "seed": 32237367, + "versionNonce": 385077731, + "isDeleted": false, + "boundElements": [], + "updated": 1, + "link": null, + "locked": false, + "points": [ + [ + 0, + 0 + ], + [ + 0, + 16 + ] + ], + "lastCommittedPoint": null, + "startBinding": null, + "endBinding": null, + "startArrowhead": null, + "endArrowhead": "arrow" + }, + { + "id": "t631219102", + "type": "text", + "x": 470, + "y": 20, + "width": 40, + "height": 20.0, + "angle": 0, + "strokeColor": "#1d1d1d", + "backgroundColor": "transparent", + "fillStyle": "solid", + "strokeWidth": 1, + "strokeStyle": "solid", + "roughness": 1, + "opacity": 100, + "groupIds": [], + "frameId": null, + "roundness": null, + "seed": 1033883458, + "versionNonce": 790789862, + "isDeleted": false, + "boundElements": [], + "updated": 1, + "link": null, + "locked": false, + "text": "vs", + "fontSize": 16, + "fontFamily": 1, + "textAlign": "left", + "verticalAlign": "top", + "containerId": null, + "originalText": "vs", + "lineHeight": 1.25, + "baseline": 16 + } + ], + "appState": { + "viewBackgroundColor": "#ffffff", + "gridSize": null + }, + "files": {} +} \ No newline at end of file diff --git a/assets/diagrams/34-pid1-signal-flow.svg b/assets/diagrams/34-pid1-signal-flow.svg new file mode 100644 index 0000000..7b688af --- /dev/null +++ b/assets/diagrams/34-pid1-signal-flow.svg @@ -0,0 +1,42 @@ + + + +serve (installs handlers) + +naive (installs none) + +podman stop -t 3 -> SIGTERM to PID 1 + +signalfd/Notify/Signals wakes +job_spawner.request_stop() + +kill(worker, SIGTERM); waitpid/Wait() +worker reaped + +print "app: shutting down (SIGTERM)" + +exit 0 -- measured: cpp 192ms, go 60ms, rust 101ms +(3000ms grace budget) + +podman stop -t 2 -> SIGTERM to PID 1 + +kernel: PID 1 has no explicit SIGTERM handler +-> default action (terminate) is masked, not run +cpp/rust: signal is simply dropped + +Go exception: runtime installs its own fatal-signal + +handler at startup regardless of signal.Notify + +measured: cpp 2108ms, rust 2074ms (full grace, SIGKILLed) +go 60ms (dies anyway -- but no shutdown line, no cleanup) + + + + + + + + +vs + \ No newline at end of file diff --git a/assets/diagrams/35-isolation-spectrum.excalidraw b/assets/diagrams/35-isolation-spectrum.excalidraw new file mode 100644 index 0000000..c0a4d5a --- /dev/null +++ b/assets/diagrams/35-isolation-spectrum.excalidraw @@ -0,0 +1,1408 @@ +{ + "type": "excalidraw", + "version": 2, + "source": "https://excalidraw.com", + "elements": [ + { + "id": "r522285977", + "type": "rectangle", + "x": 20, + "y": 20, + "width": 940, + "height": 340, + "angle": 0, + "strokeColor": "#dddddd", + "backgroundColor": "#fafafa", + "fillStyle": "solid", + "strokeWidth": 2, + "strokeStyle": "solid", + "roughness": 1, + "opacity": 100, + "groupIds": [], + "frameId": null, + "roundness": { + "type": 3 + }, + "seed": 1621762637, + "versionNonce": 330794221, + "isDeleted": false, + "boundElements": [], + "updated": 1, + "link": null, + "locked": false + }, + { + "id": "t1863286141", + "type": "text", + "x": 32, + "y": 26, + "width": 567.6, + "height": 15.0, + "angle": 0, + "strokeColor": "#555555", + "backgroundColor": "transparent", + "fillStyle": "solid", + "strokeWidth": 1, + "strokeStyle": "solid", + "roughness": 1, + "opacity": 100, + "groupIds": [], + "frameId": null, + "roundness": null, + "seed": 1175832704, + "versionNonce": 1752174566, + "isDeleted": false, + "boundElements": [], + "updated": 1, + "link": null, + "locked": false, + "text": "Shared host kernel <-----------------------------------------> separate guest kernel", + "fontSize": 12, + "fontFamily": 1, + "textAlign": "left", + "verticalAlign": "top", + "containerId": null, + "originalText": "Shared host kernel <-----------------------------------------> separate guest kernel", + "lineHeight": 1.25, + "baseline": 12 + }, + { + "id": "r1338801076", + "type": "rectangle", + "x": 45, + "y": 70, + "width": 200, + "height": 130, + "angle": 0, + "strokeColor": "#111111", + "backgroundColor": "#ffffff", + "fillStyle": "solid", + "strokeWidth": 2, + "strokeStyle": "solid", + "roughness": 1, + "opacity": 100, + "groupIds": [], + "frameId": null, + "roundness": { + "type": 3 + }, + "seed": 120442386, + "versionNonce": 880276850, + "isDeleted": false, + "boundElements": [], + "updated": 1, + "link": null, + "locked": false + }, + { + "id": "t1828511476", + "type": "text", + "x": 55, + "y": 78, + "width": 92.4, + "height": 17.5, + "angle": 0, + "strokeColor": "#111111", + "backgroundColor": "transparent", + "fillStyle": "solid", + "strokeWidth": 1, + "strokeStyle": "solid", + "roughness": 1, + "opacity": 100, + "groupIds": [], + "frameId": null, + "roundness": null, + "seed": 382190692, + "versionNonce": 1211165721, + "isDeleted": false, + "boundElements": [], + "updated": 1, + "link": null, + "locked": false, + "text": "bare process", + "fontSize": 14, + "fontFamily": 1, + "textAlign": "left", + "verticalAlign": "top", + "containerId": null, + "originalText": "bare process", + "lineHeight": 1.25, + "baseline": 14 + }, + { + "id": "t57364187", + "type": "text", + "x": 55, + "y": 98, + "width": 127.05000000000001, + "height": 13.75, + "angle": 0, + "strokeColor": "#555555", + "backgroundColor": "transparent", + "fillStyle": "solid", + "strokeWidth": 1, + "strokeStyle": "solid", + "roughness": 1, + "opacity": 100, + "groupIds": [], + "frameId": null, + "roundness": null, + "seed": 577206153, + "versionNonce": 544577420, + "isDeleted": false, + "boundElements": [], + "updated": 1, + "link": null, + "locked": false, + "text": "no isolation boundary", + "fontSize": 11, + "fontFamily": 1, + "textAlign": "left", + "verticalAlign": "top", + "containerId": null, + "originalText": "no isolation boundary", + "lineHeight": 1.25, + "baseline": 11 + }, + { + "id": "t1380351039", + "type": "text", + "x": 55, + "y": 113, + "width": 133.10000000000002, + "height": 13.75, + "angle": 0, + "strokeColor": "#555555", + "backgroundColor": "transparent", + "fillStyle": "solid", + "strokeWidth": 1, + "strokeStyle": "solid", + "roughness": 1, + "opacity": 100, + "groupIds": [], + "frameId": null, + "roundness": null, + "seed": 927553182, + "versionNonce": 1287812240, + "isDeleted": false, + "boundElements": [], + "updated": 1, + "link": null, + "locked": false, + "text": "syscall = host syscall", + "fontSize": 11, + "fontFamily": 1, + "textAlign": "left", + "verticalAlign": "top", + "containerId": null, + "originalText": "syscall = host syscall", + "lineHeight": 1.25, + "baseline": 11 + }, + { + "id": "t1564713737", + "type": "text", + "x": 55, + "y": 128, + "width": 157.3, + "height": 13.75, + "angle": 0, + "strokeColor": "#555555", + "backgroundColor": "transparent", + "fillStyle": "solid", + "strokeWidth": 1, + "strokeStyle": "solid", + "roughness": 1, + "opacity": 100, + "groupIds": [], + "frameId": null, + "roundness": null, + "seed": 434031023, + "versionNonce": 859747059, + "isDeleted": false, + "boundElements": [], + "updated": 1, + "link": null, + "locked": false, + "text": "overheadbench host numbers", + "fontSize": 11, + "fontFamily": 1, + "textAlign": "left", + "verticalAlign": "top", + "containerId": null, + "originalText": "overheadbench host numbers", + "lineHeight": 1.25, + "baseline": 11 + }, + { + "id": "r1191546148", + "type": "rectangle", + "x": 285, + "y": 70, + "width": 210, + "height": 130, + "angle": 0, + "strokeColor": "#2f6db5", + "backgroundColor": "#eef4fb", + "fillStyle": "solid", + "strokeWidth": 2, + "strokeStyle": "solid", + "roughness": 1, + "opacity": 100, + "groupIds": [], + "frameId": null, + "roundness": { + "type": 3 + }, + "seed": 1198148082, + "versionNonce": 1633795133, + "isDeleted": false, + "boundElements": [], + "updated": 1, + "link": null, + "locked": false + }, + { + "id": "t1721406365", + "type": "text", + "x": 295, + "y": 78, + "width": 123.20000000000002, + "height": 17.5, + "angle": 0, + "strokeColor": "#111111", + "backgroundColor": "transparent", + "fillStyle": "solid", + "strokeWidth": 1, + "strokeStyle": "solid", + "roughness": 1, + "opacity": 100, + "groupIds": [], + "frameId": null, + "roundness": null, + "seed": 906239473, + "versionNonce": 1720062878, + "isDeleted": false, + "boundElements": [], + "updated": 1, + "link": null, + "locked": false, + "text": "podman container", + "fontSize": 14, + "fontFamily": 1, + "textAlign": "left", + "verticalAlign": "top", + "containerId": null, + "originalText": "podman container", + "lineHeight": 1.25, + "baseline": 14 + }, + { + "id": "t611032724", + "type": "text", + "x": 295, + "y": 98, + "width": 193.60000000000002, + "height": 13.75, + "angle": 0, + "strokeColor": "#555555", + "backgroundColor": "transparent", + "fillStyle": "solid", + "strokeWidth": 1, + "strokeStyle": "solid", + "roughness": 1, + "opacity": 100, + "groupIds": [], + "frameId": null, + "roundness": null, + "seed": 334084102, + "versionNonce": 1490526544, + "isDeleted": false, + "boundElements": [], + "updated": 1, + "link": null, + "locked": false, + "text": "namespaces + cgroup v2 + seccomp", + "fontSize": 11, + "fontFamily": 1, + "textAlign": "left", + "verticalAlign": "top", + "containerId": null, + "originalText": "namespaces + cgroup v2 + seccomp", + "lineHeight": 1.25, + "baseline": 11 + }, + { + "id": "t1081566992", + "type": "text", + "x": 295, + "y": 113, + "width": 205.70000000000002, + "height": 13.75, + "angle": 0, + "strokeColor": "#555555", + "backgroundColor": "transparent", + "fillStyle": "solid", + "strokeWidth": 1, + "strokeStyle": "solid", + "roughness": 1, + "opacity": 100, + "groupIds": [], + "frameId": null, + "roundness": null, + "seed": 1757352758, + "versionNonce": 1958294306, + "isDeleted": false, + "boundElements": [], + "updated": 1, + "link": null, + "locked": false, + "text": "one host kernel, one syscall table", + "fontSize": 11, + "fontFamily": 1, + "textAlign": "left", + "verticalAlign": "top", + "containerId": null, + "originalText": "one host kernel, one syscall table", + "lineHeight": 1.25, + "baseline": 11 + }, + { + "id": "t200870530", + "type": "text", + "x": 295, + "y": 128, + "width": 284.35, + "height": 13.75, + "angle": 0, + "strokeColor": "#555555", + "backgroundColor": "transparent", + "fillStyle": "solid", + "strokeWidth": 1, + "strokeStyle": "solid", + "roughness": 1, + "opacity": 100, + "groupIds": [], + "frameId": null, + "roundness": null, + "seed": 170737540, + "versionNonce": 91029427, + "isDeleted": false, + "boundElements": [], + "updated": 1, + "link": null, + "locked": false, + "text": "syscall/mem ~= host; io ~= host on a bind mount", + "fontSize": 11, + "fontFamily": 1, + "textAlign": "left", + "verticalAlign": "top", + "containerId": null, + "originalText": "syscall/mem ~= host; io ~= host on a bind mount", + "lineHeight": 1.25, + "baseline": 11 + }, + { + "id": "r884753610", + "type": "rectangle", + "x": 535, + "y": 70, + "width": 210, + "height": 130, + "angle": 0, + "strokeColor": "#e8870c", + "backgroundColor": "#fff8ef", + "fillStyle": "solid", + "strokeWidth": 2, + "strokeStyle": "solid", + "roughness": 1, + "opacity": 100, + "groupIds": [], + "frameId": null, + "roundness": { + "type": 3 + }, + "seed": 1380217321, + "versionNonce": 494290358, + "isDeleted": false, + "boundElements": [], + "updated": 1, + "link": null, + "locked": false + }, + { + "id": "t1398400379", + "type": "text", + "x": 545, + "y": 78, + "width": 161.70000000000002, + "height": 17.5, + "angle": 0, + "strokeColor": "#111111", + "backgroundColor": "transparent", + "fillStyle": "solid", + "strokeWidth": 1, + "strokeStyle": "solid", + "roughness": 1, + "opacity": 100, + "groupIds": [], + "frameId": null, + "roundness": null, + "seed": 1496805918, + "versionNonce": 1112587739, + "isDeleted": false, + "boundElements": [], + "updated": 1, + "link": null, + "locked": false, + "text": "microVM (Firecracker,", + "fontSize": 14, + "fontFamily": 1, + "textAlign": "left", + "verticalAlign": "top", + "containerId": null, + "originalText": "microVM (Firecracker,", + "lineHeight": 1.25, + "baseline": 14 + }, + { + "id": "t70781213", + "type": "text", + "x": 545, + "y": 98, + "width": 169.4, + "height": 13.75, + "angle": 0, + "strokeColor": "#555555", + "backgroundColor": "transparent", + "fillStyle": "solid", + "strokeWidth": 1, + "strokeStyle": "solid", + "roughness": 1, + "opacity": 100, + "groupIds": [], + "frameId": null, + "roundness": null, + "seed": 667188722, + "versionNonce": 1288527590, + "isDeleted": false, + "boundElements": [], + "updated": 1, + "link": null, + "locked": false, + "text": "cloud-hypervisor / rust-vmm)", + "fontSize": 11, + "fontFamily": 1, + "textAlign": "left", + "verticalAlign": "top", + "containerId": null, + "originalText": "cloud-hypervisor / rust-vmm)", + "lineHeight": 1.25, + "baseline": 11 + }, + { + "id": "t1799043530", + "type": "text", + "x": 545, + "y": 113, + "width": 242.00000000000003, + "height": 13.75, + "angle": 0, + "strokeColor": "#555555", + "backgroundColor": "transparent", + "fillStyle": "solid", + "strokeWidth": 1, + "strokeStyle": "solid", + "roughness": 1, + "opacity": 100, + "groupIds": [], + "frameId": null, + "roundness": null, + "seed": 932954587, + "versionNonce": 853082285, + "isDeleted": false, + "boundElements": [], + "updated": 1, + "link": null, + "locked": false, + "text": "own guest kernel, minimal virtio devices", + "fontSize": 11, + "fontFamily": 1, + "textAlign": "left", + "verticalAlign": "top", + "containerId": null, + "originalText": "own guest kernel, minimal virtio devices", + "lineHeight": 1.25, + "baseline": 11 + }, + { + "id": "t1180967315", + "type": "text", + "x": 545, + "y": 128, + "width": 254.10000000000002, + "height": 13.75, + "angle": 0, + "strokeColor": "#555555", + "backgroundColor": "transparent", + "fillStyle": "solid", + "strokeWidth": 1, + "strokeStyle": "solid", + "roughness": 1, + "opacity": 100, + "groupIds": [], + "frameId": null, + "roundness": null, + "seed": 595301265, + "versionNonce": 1980229149, + "isDeleted": false, + "boundElements": [], + "updated": 1, + "link": null, + "locked": false, + "text": "~125ms boot, <5 MiB overhead (Firecracker)", + "fontSize": 11, + "fontFamily": 1, + "textAlign": "left", + "verticalAlign": "top", + "containerId": null, + "originalText": "~125ms boot, <5 MiB overhead (Firecracker)", + "lineHeight": 1.25, + "baseline": 11 + }, + { + "id": "r6239396", + "type": "rectangle", + "x": 785, + "y": 70, + "width": 175, + "height": 130, + "angle": 0, + "strokeColor": "#888888", + "backgroundColor": "#f4f4f4", + "fillStyle": "solid", + "strokeWidth": 2, + "strokeStyle": "solid", + "roughness": 1, + "opacity": 100, + "groupIds": [], + "frameId": null, + "roundness": { + "type": 3 + }, + "seed": 315581771, + "versionNonce": 407653268, + "isDeleted": false, + "boundElements": [], + "updated": 1, + "link": null, + "locked": false + }, + { + "id": "t824493717", + "type": "text", + "x": 795, + "y": 78, + "width": 123.20000000000002, + "height": 17.5, + "angle": 0, + "strokeColor": "#111111", + "backgroundColor": "transparent", + "fillStyle": "solid", + "strokeWidth": 1, + "strokeStyle": "solid", + "roughness": 1, + "opacity": 100, + "groupIds": [], + "frameId": null, + "roundness": null, + "seed": 1614200511, + "versionNonce": 1446825042, + "isDeleted": false, + "boundElements": [], + "updated": 1, + "link": null, + "locked": false, + "text": "full KVM/QEMU VM", + "fontSize": 14, + "fontFamily": 1, + "textAlign": "left", + "verticalAlign": "top", + "containerId": null, + "originalText": "full KVM/QEMU VM", + "lineHeight": 1.25, + "baseline": 14 + }, + { + "id": "t1723685372", + "type": "text", + "x": 795, + "y": 98, + "width": 211.75000000000003, + "height": 13.75, + "angle": 0, + "strokeColor": "#555555", + "backgroundColor": "transparent", + "fillStyle": "solid", + "strokeWidth": 1, + "strokeStyle": "solid", + "roughness": 1, + "opacity": 100, + "groupIds": [], + "frameId": null, + "roundness": null, + "seed": 1021610234, + "versionNonce": 265390862, + "isDeleted": false, + "boundElements": [], + "updated": 1, + "link": null, + "locked": false, + "text": "own guest kernel, full device model", + "fontSize": 11, + "fontFamily": 1, + "textAlign": "left", + "verticalAlign": "top", + "containerId": null, + "originalText": "own guest kernel, full device model", + "lineHeight": 1.25, + "baseline": 11 + }, + { + "id": "t1575258052", + "type": "text", + "x": 795, + "y": 113, + "width": 157.3, + "height": 13.75, + "angle": 0, + "strokeColor": "#555555", + "backgroundColor": "transparent", + "fillStyle": "solid", + "strokeWidth": 1, + "strokeStyle": "solid", + "roughness": 1, + "opacity": 100, + "groupIds": [], + "frameId": null, + "roundness": null, + "seed": 1656590838, + "versionNonce": 932320447, + "isDeleted": false, + "boundElements": [], + "updated": 1, + "link": null, + "locked": false, + "text": "systems-target in this lab", + "fontSize": 11, + "fontFamily": 1, + "textAlign": "left", + "verticalAlign": "top", + "containerId": null, + "originalText": "systems-target in this lab", + "lineHeight": 1.25, + "baseline": 11 + }, + { + "id": "t543337325", + "type": "text", + "x": 795, + "y": 128, + "width": 260.15000000000003, + "height": 13.75, + "angle": 0, + "strokeColor": "#555555", + "backgroundColor": "transparent", + "fillStyle": "solid", + "strokeWidth": 1, + "strokeStyle": "solid", + "roughness": 1, + "opacity": 100, + "groupIds": [], + "frameId": null, + "roundness": null, + "seed": 986616447, + "versionNonce": 1380031228, + "isDeleted": false, + "boundElements": [], + "updated": 1, + "link": null, + "locked": false, + "text": "syscall ~= host; io ~3x slower (virtio-blk)", + "fontSize": 11, + "fontFamily": 1, + "textAlign": "left", + "verticalAlign": "top", + "containerId": null, + "originalText": "syscall ~= host; io ~3x slower (virtio-blk)", + "lineHeight": 1.25, + "baseline": 11 + }, + { + "id": "r1815295698", + "type": "rectangle", + "x": 45, + "y": 230, + "width": 200, + "height": 100, + "angle": 0, + "strokeColor": "#999999", + "backgroundColor": "#ffffff", + "fillStyle": "solid", + "strokeWidth": 2, + "strokeStyle": "solid", + "roughness": 1, + "opacity": 100, + "groupIds": [], + "frameId": null, + "roundness": { + "type": 3 + }, + "seed": 1666416530, + "versionNonce": 526666951, + "isDeleted": false, + "boundElements": [], + "updated": 1, + "link": null, + "locked": false + }, + { + "id": "t51823441", + "type": "text", + "x": 55, + "y": 238, + "width": 161.70000000000002, + "height": 17.5, + "angle": 0, + "strokeColor": "#111111", + "backgroundColor": "transparent", + "fillStyle": "solid", + "strokeWidth": 1, + "strokeStyle": "solid", + "roughness": 1, + "opacity": 100, + "groupIds": [], + "frameId": null, + "roundness": null, + "seed": 432216803, + "versionNonce": 1195809908, + "isDeleted": false, + "boundElements": [], + "updated": 1, + "link": null, + "locked": false, + "text": "escape = root on host", + "fontSize": 14, + "fontFamily": 1, + "textAlign": "left", + "verticalAlign": "top", + "containerId": null, + "originalText": "escape = root on host", + "lineHeight": 1.25, + "baseline": 14 + }, + { + "id": "t957741965", + "type": "text", + "x": 55, + "y": 258, + "width": 169.4, + "height": 13.75, + "angle": 0, + "strokeColor": "#555555", + "backgroundColor": "transparent", + "fillStyle": "solid", + "strokeWidth": 1, + "strokeStyle": "solid", + "roughness": 1, + "opacity": 100, + "groupIds": [], + "frameId": null, + "roundness": null, + "seed": 5453976, + "versionNonce": 292634593, + "isDeleted": false, + "boundElements": [], + "updated": 1, + "link": null, + "locked": false, + "text": "kernel 0-days apply directly", + "fontSize": 11, + "fontFamily": 1, + "textAlign": "left", + "verticalAlign": "top", + "containerId": null, + "originalText": "kernel 0-days apply directly", + "lineHeight": 1.25, + "baseline": 11 + }, + { + "id": "r920774735", + "type": "rectangle", + "x": 285, + "y": 230, + "width": 210, + "height": 100, + "angle": 0, + "strokeColor": "#999999", + "backgroundColor": "#ffffff", + "fillStyle": "solid", + "strokeWidth": 2, + "strokeStyle": "solid", + "roughness": 1, + "opacity": 100, + "groupIds": [], + "frameId": null, + "roundness": { + "type": 3 + }, + "seed": 1692691003, + "versionNonce": 5885412, + "isDeleted": false, + "boundElements": [], + "updated": 1, + "link": null, + "locked": false + }, + { + "id": "t66632114", + "type": "text", + "x": 295, + "y": 238, + "width": 169.4, + "height": 17.5, + "angle": 0, + "strokeColor": "#111111", + "backgroundColor": "transparent", + "fillStyle": "solid", + "strokeWidth": 1, + "strokeStyle": "solid", + "roughness": 1, + "opacity": 100, + "groupIds": [], + "frameId": null, + "roundness": null, + "seed": 1959247992, + "versionNonce": 1347222261, + "isDeleted": false, + "boundElements": [], + "updated": 1, + "link": null, + "locked": false, + "text": "escape = kernel bug in", + "fontSize": 14, + "fontFamily": 1, + "textAlign": "left", + "verticalAlign": "top", + "containerId": null, + "originalText": "escape = kernel bug in", + "lineHeight": 1.25, + "baseline": 14 + }, + { + "id": "t682676679", + "type": "text", + "x": 295, + "y": 258, + "width": 127.05000000000001, + "height": 13.75, + "angle": 0, + "strokeColor": "#555555", + "backgroundColor": "transparent", + "fillStyle": "solid", + "strokeWidth": 1, + "strokeStyle": "solid", + "roughness": 1, + "opacity": 100, + "groupIds": [], + "frameId": null, + "roundness": null, + "seed": 1043622505, + "versionNonce": 445496106, + "isDeleted": false, + "boundElements": [], + "updated": 1, + "link": null, + "locked": false, + "text": "namespace/cgroup code", + "fontSize": 11, + "fontFamily": 1, + "textAlign": "left", + "verticalAlign": "top", + "containerId": null, + "originalText": "namespace/cgroup code", + "lineHeight": 1.25, + "baseline": 11 + }, + { + "id": "r106505383", + "type": "rectangle", + "x": 535, + "y": 230, + "width": 210, + "height": 100, + "angle": 0, + "strokeColor": "#999999", + "backgroundColor": "#ffffff", + "fillStyle": "solid", + "strokeWidth": 2, + "strokeStyle": "solid", + "roughness": 1, + "opacity": 100, + "groupIds": [], + "frameId": null, + "roundness": { + "type": 3 + }, + "seed": 1335118805, + "versionNonce": 1587806369, + "isDeleted": false, + "boundElements": [], + "updated": 1, + "link": null, + "locked": false + }, + { + "id": "t473224228", + "type": "text", + "x": 545, + "y": 238, + "width": 231.00000000000003, + "height": 17.5, + "angle": 0, + "strokeColor": "#111111", + "backgroundColor": "transparent", + "fillStyle": "solid", + "strokeWidth": 1, + "strokeStyle": "solid", + "roughness": 1, + "opacity": 100, + "groupIds": [], + "frameId": null, + "roundness": null, + "seed": 641016584, + "versionNonce": 270624933, + "isDeleted": false, + "boundElements": [], + "updated": 1, + "link": null, + "locked": false, + "text": "escape = bug in the ~5 devices", + "fontSize": 14, + "fontFamily": 1, + "textAlign": "left", + "verticalAlign": "top", + "containerId": null, + "originalText": "escape = bug in the ~5 devices", + "lineHeight": 1.25, + "baseline": 14 + }, + { + "id": "t1963651267", + "type": "text", + "x": 545, + "y": 258, + "width": 181.50000000000003, + "height": 13.75, + "angle": 0, + "strokeColor": "#555555", + "backgroundColor": "transparent", + "fillStyle": "solid", + "strokeWidth": 1, + "strokeStyle": "solid", + "roughness": 1, + "opacity": 100, + "groupIds": [], + "frameId": null, + "roundness": null, + "seed": 1232443034, + "versionNonce": 1894932183, + "isDeleted": false, + "boundElements": [], + "updated": 1, + "link": null, + "locked": false, + "text": "the VMM exposes, or KVM itself", + "fontSize": 11, + "fontFamily": 1, + "textAlign": "left", + "verticalAlign": "top", + "containerId": null, + "originalText": "the VMM exposes, or KVM itself", + "lineHeight": 1.25, + "baseline": 11 + }, + { + "id": "r1377389134", + "type": "rectangle", + "x": 785, + "y": 230, + "width": 175, + "height": 100, + "angle": 0, + "strokeColor": "#999999", + "backgroundColor": "#ffffff", + "fillStyle": "solid", + "strokeWidth": 2, + "strokeStyle": "solid", + "roughness": 1, + "opacity": 100, + "groupIds": [], + "frameId": null, + "roundness": { + "type": 3 + }, + "seed": 939192363, + "versionNonce": 1531698353, + "isDeleted": false, + "boundElements": [], + "updated": 1, + "link": null, + "locked": false + }, + { + "id": "t1001582500", + "type": "text", + "x": 795, + "y": 238, + "width": 184.8, + "height": 17.5, + "angle": 0, + "strokeColor": "#111111", + "backgroundColor": "transparent", + "fillStyle": "solid", + "strokeWidth": 1, + "strokeStyle": "solid", + "roughness": 1, + "opacity": 100, + "groupIds": [], + "frameId": null, + "roundness": null, + "seed": 1591675000, + "versionNonce": 504848771, + "isDeleted": false, + "boundElements": [], + "updated": 1, + "link": null, + "locked": false, + "text": "escape = bug in the full", + "fontSize": 14, + "fontFamily": 1, + "textAlign": "left", + "verticalAlign": "top", + "containerId": null, + "originalText": "escape = bug in the full", + "lineHeight": 1.25, + "baseline": 14 + }, + { + "id": "t1644790930", + "type": "text", + "x": 795, + "y": 258, + "width": 169.4, + "height": 13.75, + "angle": 0, + "strokeColor": "#555555", + "backgroundColor": "transparent", + "fillStyle": "solid", + "strokeWidth": 1, + "strokeStyle": "solid", + "roughness": 1, + "opacity": 100, + "groupIds": [], + "frameId": null, + "roundness": null, + "seed": 326331592, + "versionNonce": 1866905623, + "isDeleted": false, + "boundElements": [], + "updated": 1, + "link": null, + "locked": false, + "text": "emulated device model or KVM", + "fontSize": 11, + "fontFamily": 1, + "textAlign": "left", + "verticalAlign": "top", + "containerId": null, + "originalText": "emulated device model or KVM", + "lineHeight": 1.25, + "baseline": 11 + }, + { + "id": "a1470956053", + "type": "arrow", + "x": 245, + "y": 135, + "width": 40, + "height": 0, + "angle": 0, + "strokeColor": "#555555", + "backgroundColor": "transparent", + "fillStyle": "solid", + "strokeWidth": 2, + "strokeStyle": "solid", + "roughness": 1, + "opacity": 100, + "groupIds": [], + "frameId": null, + "roundness": { + "type": 2 + }, + "seed": 1293721532, + "versionNonce": 1304914390, + "isDeleted": false, + "boundElements": [], + "updated": 1, + "link": null, + "locked": false, + "points": [ + [ + 0, + 0 + ], + [ + 40, + 0 + ] + ], + "lastCommittedPoint": null, + "startBinding": null, + "endBinding": null, + "startArrowhead": null, + "endArrowhead": "arrow" + }, + { + "id": "t798743836", + "type": "text", + "x": 265.0, + "y": 121.0, + "width": 102.85000000000001, + "height": 13.75, + "angle": 0, + "strokeColor": "#555555", + "backgroundColor": "transparent", + "fillStyle": "solid", + "strokeWidth": 1, + "strokeStyle": "solid", + "roughness": 1, + "opacity": 100, + "groupIds": [], + "frameId": null, + "roundness": null, + "seed": 79492110, + "versionNonce": 268365488, + "isDeleted": false, + "boundElements": [], + "updated": 1, + "link": null, + "locked": false, + "text": "less trust needed", + "fontSize": 11, + "fontFamily": 1, + "textAlign": "left", + "verticalAlign": "top", + "containerId": null, + "originalText": "less trust needed", + "lineHeight": 1.25, + "baseline": 11 + }, + { + "id": "a293701183", + "type": "arrow", + "x": 495, + "y": 135, + "width": 40, + "height": 0, + "angle": 0, + "strokeColor": "#555555", + "backgroundColor": "transparent", + "fillStyle": "solid", + "strokeWidth": 2, + "strokeStyle": "solid", + "roughness": 1, + "opacity": 100, + "groupIds": [], + "frameId": null, + "roundness": { + "type": 2 + }, + "seed": 398236831, + "versionNonce": 1092723922, + "isDeleted": false, + "boundElements": [], + "updated": 1, + "link": null, + "locked": false, + "points": [ + [ + 0, + 0 + ], + [ + 40, + 0 + ] + ], + "lastCommittedPoint": null, + "startBinding": null, + "endBinding": null, + "startArrowhead": null, + "endArrowhead": "arrow" + }, + { + "id": "t1913625368", + "type": "text", + "x": 515.0, + "y": 121.0, + "width": 60.50000000000001, + "height": 13.75, + "angle": 0, + "strokeColor": "#555555", + "backgroundColor": "transparent", + "fillStyle": "solid", + "strokeWidth": 1, + "strokeStyle": "solid", + "roughness": 1, + "opacity": 100, + "groupIds": [], + "frameId": null, + "roundness": null, + "seed": 1605791917, + "versionNonce": 971305480, + "isDeleted": false, + "boundElements": [], + "updated": 1, + "link": null, + "locked": false, + "text": "own kernel", + "fontSize": 11, + "fontFamily": 1, + "textAlign": "left", + "verticalAlign": "top", + "containerId": null, + "originalText": "own kernel", + "lineHeight": 1.25, + "baseline": 11 + }, + { + "id": "a1448427391", + "type": "arrow", + "x": 745, + "y": 135, + "width": 40, + "height": 0, + "angle": 0, + "strokeColor": "#555555", + "backgroundColor": "transparent", + "fillStyle": "solid", + "strokeWidth": 2, + "strokeStyle": "solid", + "roughness": 1, + "opacity": 100, + "groupIds": [], + "frameId": null, + "roundness": { + "type": 2 + }, + "seed": 7765048, + "versionNonce": 494588137, + "isDeleted": false, + "boundElements": [], + "updated": 1, + "link": null, + "locked": false, + "points": [ + [ + 0, + 0 + ], + [ + 40, + 0 + ] + ], + "lastCommittedPoint": null, + "startBinding": null, + "endBinding": null, + "startArrowhead": null, + "endArrowhead": "arrow" + }, + { + "id": "t209872789", + "type": "text", + "x": 765.0, + "y": 121.0, + "width": 102.85000000000001, + "height": 13.75, + "angle": 0, + "strokeColor": "#555555", + "backgroundColor": "transparent", + "fillStyle": "solid", + "strokeWidth": 1, + "strokeStyle": "solid", + "roughness": 1, + "opacity": 100, + "groupIds": [], + "frameId": null, + "roundness": null, + "seed": 973299140, + "versionNonce": 225594605, + "isDeleted": false, + "boundElements": [], + "updated": 1, + "link": null, + "locked": false, + "text": "full device model", + "fontSize": 11, + "fontFamily": 1, + "textAlign": "left", + "verticalAlign": "top", + "containerId": null, + "originalText": "full device model", + "lineHeight": 1.25, + "baseline": 11 + }, + { + "id": "t894980437", + "type": "text", + "x": 500, + "y": 390, + "width": 689.7, + "height": 13.75, + "angle": 0, + "strokeColor": "#1d1d1d", + "backgroundColor": "transparent", + "fillStyle": "solid", + "strokeWidth": 1, + "strokeStyle": "solid", + "roughness": 1, + "opacity": 100, + "groupIds": [], + "frameId": null, + "roundness": null, + "seed": 277159316, + "versionNonce": 369677631, + "isDeleted": false, + "boundElements": [], + "updated": 1, + "link": null, + "locked": false, + "text": "rust-vmm crates (kvm-ioctls, virtio-queue, vm-memory) underpin both Firecracker v1.16.1 and Cloud Hypervisor v53.0", + "fontSize": 11, + "fontFamily": 1, + "textAlign": "left", + "verticalAlign": "top", + "containerId": null, + "originalText": "rust-vmm crates (kvm-ioctls, virtio-queue, vm-memory) underpin both Firecracker v1.16.1 and Cloud Hypervisor v53.0", + "lineHeight": 1.25, + "baseline": 11 + } + ], + "appState": { + "viewBackgroundColor": "#ffffff", + "gridSize": null + }, + "files": {} +} \ No newline at end of file diff --git a/assets/diagrams/35-isolation-spectrum.svg b/assets/diagrams/35-isolation-spectrum.svg new file mode 100644 index 0000000..d943cad --- /dev/null +++ b/assets/diagrams/35-isolation-spectrum.svg @@ -0,0 +1,47 @@ + + + +Shared host kernel <-----------------------------------------> separate guest kernel + +bare process +no isolation boundary +syscall = host syscall +overheadbench host numbers + +podman container +namespaces + cgroup v2 + seccomp +one host kernel, one syscall table +syscall/mem ~= host; io ~= host on a bind mount + +microVM (Firecracker, +cloud-hypervisor / rust-vmm) +own guest kernel, minimal virtio devices +~125ms boot, <5 MiB overhead (Firecracker) + +full KVM/QEMU VM +own guest kernel, full device model +systems-target in this lab +syscall ~= host; io ~3x slower (virtio-blk) + +escape = root on host +kernel 0-days apply directly + +escape = kernel bug in +namespace/cgroup code + +escape = bug in the ~5 devices +the VMM exposes, or KVM itself + +escape = bug in the full +emulated device model or KVM + +less trust needed +less trust needed + +own kernel +own kernel + +full device model +full device model +rust-vmm crates (kvm-ioctls, virtio-queue, vm-memory) underpin both Firecracker v1.16.1 and Cloud Hypervisor v53.0 + \ No newline at end of file diff --git a/assets/diagrams/35-kvm-virtio-stack.excalidraw b/assets/diagrams/35-kvm-virtio-stack.excalidraw new file mode 100644 index 0000000..539d2b0 --- /dev/null +++ b/assets/diagrams/35-kvm-virtio-stack.excalidraw @@ -0,0 +1,2330 @@ +{ + "type": "excalidraw", + "version": 2, + "source": "https://excalidraw.com", + "elements": [ + { + "id": "r616816452", + "type": "rectangle", + "x": 20, + "y": 20, + "width": 920, + "height": 130, + "angle": 0, + "strokeColor": "#dddddd", + "backgroundColor": "#f4f4f4", + "fillStyle": "solid", + "strokeWidth": 2, + "strokeStyle": "solid", + "roughness": 1, + "opacity": 100, + "groupIds": [], + "frameId": null, + "roundness": { + "type": 3 + }, + "seed": 1925318893, + "versionNonce": 1990841614, + "isDeleted": false, + "boundElements": [], + "updated": 1, + "link": null, + "locked": false + }, + { + "id": "t1897602274", + "type": "text", + "x": 32, + "y": 26, + "width": 264.0, + "height": 15.0, + "angle": 0, + "strokeColor": "#555555", + "backgroundColor": "transparent", + "fillStyle": "solid", + "strokeWidth": 1, + "strokeStyle": "solid", + "roughness": 1, + "opacity": 100, + "groupIds": [], + "frameId": null, + "roundness": null, + "seed": 809766187, + "versionNonce": 1152277146, + "isDeleted": false, + "boundElements": [], + "updated": 1, + "link": null, + "locked": false, + "text": "Host, VMX root mode (ring 0 host kernel)", + "fontSize": 12, + "fontFamily": 1, + "textAlign": "left", + "verticalAlign": "top", + "containerId": null, + "originalText": "Host, VMX root mode (ring 0 host kernel)", + "lineHeight": 1.25, + "baseline": 12 + }, + { + "id": "r1570438349", + "type": "rectangle", + "x": 20, + "y": 170, + "width": 920, + "height": 150, + "angle": 0, + "strokeColor": "#dddddd", + "backgroundColor": "#eef4fb", + "fillStyle": "solid", + "strokeWidth": 2, + "strokeStyle": "solid", + "roughness": 1, + "opacity": 100, + "groupIds": [], + "frameId": null, + "roundness": { + "type": 3 + }, + "seed": 292912127, + "versionNonce": 228987000, + "isDeleted": false, + "boundElements": [], + "updated": 1, + "link": null, + "locked": false + }, + { + "id": "t1006862364", + "type": "text", + "x": 32, + "y": 176, + "width": 448.8, + "height": 15.0, + "angle": 0, + "strokeColor": "#555555", + "backgroundColor": "transparent", + "fillStyle": "solid", + "strokeWidth": 1, + "strokeStyle": "solid", + "roughness": 1, + "opacity": 100, + "groupIds": [], + "frameId": null, + "roundness": null, + "seed": 1312109153, + "versionNonce": 1332376793, + "isDeleted": false, + "boundElements": [], + "updated": 1, + "link": null, + "locked": false, + "text": "Guest, VMX non-root mode (ring 0 guest kernel, ring 3 guest process)", + "fontSize": 12, + "fontFamily": 1, + "textAlign": "left", + "verticalAlign": "top", + "containerId": null, + "originalText": "Guest, VMX non-root mode (ring 0 guest kernel, ring 3 guest process)", + "lineHeight": 1.25, + "baseline": 12 + }, + { + "id": "r97444046", + "type": "rectangle", + "x": 20, + "y": 340, + "width": 920, + "height": 130, + "angle": 0, + "strokeColor": "#dddddd", + "backgroundColor": "#fff8ef", + "fillStyle": "solid", + "strokeWidth": 2, + "strokeStyle": "solid", + "roughness": 1, + "opacity": 100, + "groupIds": [], + "frameId": null, + "roundness": { + "type": 3 + }, + "seed": 1917787981, + "versionNonce": 1658944199, + "isDeleted": false, + "boundElements": [], + "updated": 1, + "link": null, + "locked": false + }, + { + "id": "t34388662", + "type": "text", + "x": 32, + "y": 346, + "width": 356.40000000000003, + "height": 15.0, + "angle": 0, + "strokeColor": "#555555", + "backgroundColor": "transparent", + "fillStyle": "solid", + "strokeWidth": 1, + "strokeStyle": "solid", + "roughness": 1, + "opacity": 100, + "groupIds": [], + "frameId": null, + "roundness": null, + "seed": 1177317519, + "versionNonce": 774173864, + "isDeleted": false, + "boundElements": [], + "updated": 1, + "link": null, + "locked": false, + "text": "Trapped path: virtio-blk write + fsync (a real vmexit)", + "fontSize": 12, + "fontFamily": 1, + "textAlign": "left", + "verticalAlign": "top", + "containerId": null, + "originalText": "Trapped path: virtio-blk write + fsync (a real vmexit)", + "lineHeight": 1.25, + "baseline": 12 + }, + { + "id": "r1394106624", + "type": "rectangle", + "x": 20, + "y": 490, + "width": 920, + "height": 110, + "angle": 0, + "strokeColor": "#dddddd", + "backgroundColor": "#fafafa", + "fillStyle": "solid", + "strokeWidth": 2, + "strokeStyle": "solid", + "roughness": 1, + "opacity": 100, + "groupIds": [], + "frameId": null, + "roundness": { + "type": 3 + }, + "seed": 333258995, + "versionNonce": 1365458870, + "isDeleted": false, + "boundElements": [], + "updated": 1, + "link": null, + "locked": false + }, + { + "id": "t1594211623", + "type": "text", + "x": 32, + "y": 496, + "width": 349.8, + "height": 15.0, + "angle": 0, + "strokeColor": "#555555", + "backgroundColor": "transparent", + "fillStyle": "solid", + "strokeWidth": 1, + "strokeStyle": "solid", + "roughness": 1, + "opacity": 100, + "groupIds": [], + "frameId": null, + "roundness": null, + "seed": 276603995, + "versionNonce": 797917984, + "isDeleted": false, + "boundElements": [], + "updated": 1, + "link": null, + "locked": false, + "text": "overheadbench measured deltas (this host, this guest)", + "fontSize": 12, + "fontFamily": 1, + "textAlign": "left", + "verticalAlign": "top", + "containerId": null, + "originalText": "overheadbench measured deltas (this host, this guest)", + "lineHeight": 1.25, + "baseline": 12 + }, + { + "id": "r65980721", + "type": "rectangle", + "x": 40, + "y": 55, + "width": 200, + "height": 70, + "angle": 0, + "strokeColor": "#888888", + "backgroundColor": "#f4f4f4", + "fillStyle": "solid", + "strokeWidth": 2, + "strokeStyle": "solid", + "roughness": 1, + "opacity": 100, + "groupIds": [], + "frameId": null, + "roundness": { + "type": 3 + }, + "seed": 300745757, + "versionNonce": 727824424, + "isDeleted": false, + "boundElements": [], + "updated": 1, + "link": null, + "locked": false + }, + { + "id": "t1113477136", + "type": "text", + "x": 50, + "y": 63, + "width": 161.70000000000002, + "height": 17.5, + "angle": 0, + "strokeColor": "#111111", + "backgroundColor": "transparent", + "fillStyle": "solid", + "strokeWidth": 1, + "strokeStyle": "solid", + "roughness": 1, + "opacity": 100, + "groupIds": [], + "frameId": null, + "roundness": null, + "seed": 1220350254, + "versionNonce": 956937771, + "isDeleted": false, + "boundElements": [], + "updated": 1, + "link": null, + "locked": false, + "text": "kvm.ko + kvm_intel.ko", + "fontSize": 14, + "fontFamily": 1, + "textAlign": "left", + "verticalAlign": "top", + "containerId": null, + "originalText": "kvm.ko + kvm_intel.ko", + "lineHeight": 1.25, + "baseline": 14 + }, + { + "id": "t1350545563", + "type": "text", + "x": 50, + "y": 83, + "width": 121.00000000000001, + "height": 13.75, + "angle": 0, + "strokeColor": "#555555", + "backgroundColor": "transparent", + "fillStyle": "solid", + "strokeWidth": 1, + "strokeStyle": "solid", + "roughness": 1, + "opacity": 100, + "groupIds": [], + "frameId": null, + "roundness": null, + "seed": 1536588244, + "versionNonce": 1806981628, + "isDeleted": false, + "boundElements": [], + "updated": 1, + "link": null, + "locked": false, + "text": "/dev/kvm, EPT tables", + "fontSize": 11, + "fontFamily": 1, + "textAlign": "left", + "verticalAlign": "top", + "containerId": null, + "originalText": "/dev/kvm, EPT tables", + "lineHeight": 1.25, + "baseline": 11 + }, + { + "id": "r1849650273", + "type": "rectangle", + "x": 270, + "y": 55, + "width": 200, + "height": 70, + "angle": 0, + "strokeColor": "#2f6db5", + "backgroundColor": "#eef4fb", + "fillStyle": "solid", + "strokeWidth": 2, + "strokeStyle": "solid", + "roughness": 1, + "opacity": 100, + "groupIds": [], + "frameId": null, + "roundness": { + "type": 3 + }, + "seed": 1174963773, + "versionNonce": 454915088, + "isDeleted": false, + "boundElements": [], + "updated": 1, + "link": null, + "locked": false + }, + { + "id": "t1096198985", + "type": "text", + "x": 280, + "y": 63, + "width": 138.60000000000002, + "height": 17.5, + "angle": 0, + "strokeColor": "#111111", + "backgroundColor": "transparent", + "fillStyle": "solid", + "strokeWidth": 1, + "strokeStyle": "solid", + "roughness": 1, + "opacity": 100, + "groupIds": [], + "frameId": null, + "roundness": null, + "seed": 332218422, + "versionNonce": 1139293102, + "isDeleted": false, + "boundElements": [], + "updated": 1, + "link": null, + "locked": false, + "text": "QEMU/VMM userspace", + "fontSize": 14, + "fontFamily": 1, + "textAlign": "left", + "verticalAlign": "top", + "containerId": null, + "originalText": "QEMU/VMM userspace", + "lineHeight": 1.25, + "baseline": 14 + }, + { + "id": "t306558575", + "type": "text", + "x": 280, + "y": 83, + "width": 187.55, + "height": 13.75, + "angle": 0, + "strokeColor": "#555555", + "backgroundColor": "transparent", + "fillStyle": "solid", + "strokeWidth": 1, + "strokeStyle": "solid", + "roughness": 1, + "opacity": 100, + "groupIds": [], + "frameId": null, + "roundness": null, + "seed": 1416860418, + "versionNonce": 1155392771, + "isDeleted": false, + "boundElements": [], + "updated": 1, + "link": null, + "locked": false, + "text": "vCPU thread, virtio-blk backend", + "fontSize": 11, + "fontFamily": 1, + "textAlign": "left", + "verticalAlign": "top", + "containerId": null, + "originalText": "vCPU thread, virtio-blk backend", + "lineHeight": 1.25, + "baseline": 11 + }, + { + "id": "r1310956167", + "type": "rectangle", + "x": 730, + "y": 55, + "width": 190, + "height": 70, + "angle": 0, + "strokeColor": "#111111", + "backgroundColor": "#ffffff", + "fillStyle": "solid", + "strokeWidth": 2, + "strokeStyle": "solid", + "roughness": 1, + "opacity": 100, + "groupIds": [], + "frameId": null, + "roundness": { + "type": 3 + }, + "seed": 1651468000, + "versionNonce": 13583511, + "isDeleted": false, + "boundElements": [], + "updated": 1, + "link": null, + "locked": false + }, + { + "id": "t448309055", + "type": "text", + "x": 740, + "y": 63, + "width": 115.50000000000001, + "height": 17.5, + "angle": 0, + "strokeColor": "#111111", + "backgroundColor": "transparent", + "fillStyle": "solid", + "strokeWidth": 1, + "strokeStyle": "solid", + "roughness": 1, + "opacity": 100, + "groupIds": [], + "frameId": null, + "roundness": null, + "seed": 1619856415, + "versionNonce": 1434727469, + "isDeleted": false, + "boundElements": [], + "updated": 1, + "link": null, + "locked": false, + "text": "Host NVMe/btrfs", + "fontSize": 14, + "fontFamily": 1, + "textAlign": "left", + "verticalAlign": "top", + "containerId": null, + "originalText": "Host NVMe/btrfs", + "lineHeight": 1.25, + "baseline": 14 + }, + { + "id": "t1640415291", + "type": "text", + "x": 740, + "y": 83, + "width": 145.20000000000002, + "height": 13.75, + "angle": 0, + "strokeColor": "#555555", + "backgroundColor": "transparent", + "fillStyle": "solid", + "strokeWidth": 1, + "strokeStyle": "solid", + "roughness": 1, + "opacity": 100, + "groupIds": [], + "frameId": null, + "roundness": null, + "seed": 52641870, + "versionNonce": 1364632400, + "isDeleted": false, + "boundElements": [], + "updated": 1, + "link": null, + "locked": false, + "text": "real fsync(2) lands here", + "fontSize": 11, + "fontFamily": 1, + "textAlign": "left", + "verticalAlign": "top", + "containerId": null, + "originalText": "real fsync(2) lands here", + "lineHeight": 1.25, + "baseline": 11 + }, + { + "id": "r1701680226", + "type": "rectangle", + "x": 40, + "y": 200, + "width": 220, + "height": 90, + "angle": 0, + "strokeColor": "#2f6db5", + "backgroundColor": "#eef4fb", + "fillStyle": "solid", + "strokeWidth": 2, + "strokeStyle": "solid", + "roughness": 1, + "opacity": 100, + "groupIds": [], + "frameId": null, + "roundness": { + "type": 3 + }, + "seed": 335735647, + "versionNonce": 1256260957, + "isDeleted": false, + "boundElements": [], + "updated": 1, + "link": null, + "locked": false + }, + { + "id": "t1365872152", + "type": "text", + "x": 50, + "y": 208, + "width": 161.70000000000002, + "height": 17.5, + "angle": 0, + "strokeColor": "#111111", + "backgroundColor": "transparent", + "fillStyle": "solid", + "strokeWidth": 1, + "strokeStyle": "solid", + "roughness": 1, + "opacity": 100, + "groupIds": [], + "frameId": null, + "roundness": null, + "seed": 1073168838, + "versionNonce": 554455780, + "isDeleted": false, + "boundElements": [], + "updated": 1, + "link": null, + "locked": false, + "text": "guest kernel (ring 0)", + "fontSize": 14, + "fontFamily": 1, + "textAlign": "left", + "verticalAlign": "top", + "containerId": null, + "originalText": "guest kernel (ring 0)", + "lineHeight": 1.25, + "baseline": 14 + }, + { + "id": "t1655515916", + "type": "text", + "x": 50, + "y": 228, + "width": 151.25, + "height": 13.75, + "angle": 0, + "strokeColor": "#555555", + "backgroundColor": "transparent", + "fillStyle": "solid", + "strokeWidth": 1, + "strokeStyle": "solid", + "roughness": 1, + "opacity": 100, + "groupIds": [], + "frameId": null, + "roundness": null, + "seed": 1816386919, + "versionNonce": 529765972, + "isDeleted": false, + "boundElements": [], + "updated": 1, + "link": null, + "locked": false, + "text": "getpid(2) served in-guest", + "fontSize": 11, + "fontFamily": 1, + "textAlign": "left", + "verticalAlign": "top", + "containerId": null, + "originalText": "getpid(2) served in-guest", + "lineHeight": 1.25, + "baseline": 11 + }, + { + "id": "t58228395", + "type": "text", + "x": 50, + "y": 243, + "width": 114.95, + "height": 13.75, + "angle": 0, + "strokeColor": "#555555", + "backgroundColor": "transparent", + "fillStyle": "solid", + "strokeWidth": 1, + "strokeStyle": "solid", + "roughness": 1, + "opacity": 100, + "groupIds": [], + "frameId": null, + "roundness": null, + "seed": 310166029, + "versionNonce": 253839898, + "isDeleted": false, + "boundElements": [], + "updated": 1, + "link": null, + "locked": false, + "text": "no host trap needed", + "fontSize": 11, + "fontFamily": 1, + "textAlign": "left", + "verticalAlign": "top", + "containerId": null, + "originalText": "no host trap needed", + "lineHeight": 1.25, + "baseline": 11 + }, + { + "id": "r165757960", + "type": "rectangle", + "x": 290, + "y": 200, + "width": 220, + "height": 90, + "angle": 0, + "strokeColor": "#111111", + "backgroundColor": "#ffffff", + "fillStyle": "solid", + "strokeWidth": 2, + "strokeStyle": "solid", + "roughness": 1, + "opacity": 100, + "groupIds": [], + "frameId": null, + "roundness": { + "type": 3 + }, + "seed": 1353827092, + "versionNonce": 896746255, + "isDeleted": false, + "boundElements": [], + "updated": 1, + "link": null, + "locked": false + }, + { + "id": "t324321559", + "type": "text", + "x": 300, + "y": 208, + "width": 169.4, + "height": 17.5, + "angle": 0, + "strokeColor": "#111111", + "backgroundColor": "transparent", + "fillStyle": "solid", + "strokeWidth": 1, + "strokeStyle": "solid", + "roughness": 1, + "opacity": 100, + "groupIds": [], + "frameId": null, + "roundness": null, + "seed": 392944372, + "versionNonce": 1928755509, + "isDeleted": false, + "boundElements": [], + "updated": 1, + "link": null, + "locked": false, + "text": "overheadbench (ring 3)", + "fontSize": 14, + "fontFamily": 1, + "textAlign": "left", + "verticalAlign": "top", + "containerId": null, + "originalText": "overheadbench (ring 3)", + "lineHeight": 1.25, + "baseline": 14 + }, + { + "id": "t564081542", + "type": "text", + "x": 300, + "y": 228, + "width": 181.50000000000003, + "height": 13.75, + "angle": 0, + "strokeColor": "#555555", + "backgroundColor": "transparent", + "fillStyle": "solid", + "strokeWidth": 1, + "strokeStyle": "solid", + "roughness": 1, + "opacity": 100, + "groupIds": [], + "frameId": null, + "roundness": null, + "seed": 91705525, + "versionNonce": 394034361, + "isDeleted": false, + "boundElements": [], + "updated": 1, + "link": null, + "locked": false, + "text": "syscall bench: raw getpid loop", + "fontSize": 11, + "fontFamily": 1, + "textAlign": "left", + "verticalAlign": "top", + "containerId": null, + "originalText": "syscall bench: raw getpid loop", + "lineHeight": 1.25, + "baseline": 11 + }, + { + "id": "r1312057073", + "type": "rectangle", + "x": 550, + "y": 200, + "width": 200, + "height": 90, + "angle": 0, + "strokeColor": "#2f6db5", + "backgroundColor": "#eef4fb", + "fillStyle": "solid", + "strokeWidth": 2, + "strokeStyle": "solid", + "roughness": 1, + "opacity": 100, + "groupIds": [], + "frameId": null, + "roundness": { + "type": 3 + }, + "seed": 647023417, + "versionNonce": 1845441072, + "isDeleted": false, + "boundElements": [], + "updated": 1, + "link": null, + "locked": false + }, + { + "id": "t1996781456", + "type": "text", + "x": 560, + "y": 208, + "width": 207.9, + "height": 17.5, + "angle": 0, + "strokeColor": "#111111", + "backgroundColor": "transparent", + "fillStyle": "solid", + "strokeWidth": 1, + "strokeStyle": "solid", + "roughness": 1, + "opacity": 100, + "groupIds": [], + "frameId": null, + "roundness": null, + "seed": 798727771, + "versionNonce": 1634957555, + "isDeleted": false, + "boundElements": [], + "updated": 1, + "link": null, + "locked": false, + "text": "virtio-blk front-end driver", + "fontSize": 14, + "fontFamily": 1, + "textAlign": "left", + "verticalAlign": "top", + "containerId": null, + "originalText": "virtio-blk front-end driver", + "lineHeight": 1.25, + "baseline": 14 + }, + { + "id": "t914603669", + "type": "text", + "x": 560, + "y": 228, + "width": 157.3, + "height": 13.75, + "angle": 0, + "strokeColor": "#555555", + "backgroundColor": "transparent", + "fillStyle": "solid", + "strokeWidth": 1, + "strokeStyle": "solid", + "roughness": 1, + "opacity": 100, + "groupIds": [], + "frameId": null, + "roundness": null, + "seed": 51271079, + "versionNonce": 1729053486, + "isDeleted": false, + "boundElements": [], + "updated": 1, + "link": null, + "locked": false, + "text": "io bench: open/write/fsync", + "fontSize": 11, + "fontFamily": 1, + "textAlign": "left", + "verticalAlign": "top", + "containerId": null, + "originalText": "io bench: open/write/fsync", + "lineHeight": 1.25, + "baseline": 11 + }, + { + "id": "r133091402", + "type": "rectangle", + "x": 790, + "y": 200, + "width": 130, + "height": 90, + "angle": 0, + "strokeColor": "#999999", + "backgroundColor": "#ffffff", + "fillStyle": "solid", + "strokeWidth": 2, + "strokeStyle": "solid", + "roughness": 1, + "opacity": 100, + "groupIds": [], + "frameId": null, + "roundness": { + "type": 3 + }, + "seed": 41962314, + "versionNonce": 1024810400, + "isDeleted": false, + "boundElements": [], + "updated": 1, + "link": null, + "locked": false + }, + { + "id": "t195416674", + "type": "text", + "x": 800, + "y": 208, + "width": 69.30000000000001, + "height": 17.5, + "angle": 0, + "strokeColor": "#111111", + "backgroundColor": "transparent", + "fillStyle": "solid", + "strokeWidth": 1, + "strokeStyle": "solid", + "roughness": 1, + "opacity": 100, + "groupIds": [], + "frameId": null, + "roundness": null, + "seed": 628305078, + "versionNonce": 1977065242, + "isDeleted": false, + "boundElements": [], + "updated": 1, + "link": null, + "locked": false, + "text": "virtqueue", + "fontSize": 14, + "fontFamily": 1, + "textAlign": "left", + "verticalAlign": "top", + "containerId": null, + "originalText": "virtqueue", + "lineHeight": 1.25, + "baseline": 14 + }, + { + "id": "t744081371", + "type": "text", + "x": 800, + "y": 228, + "width": 90.75000000000001, + "height": 13.75, + "angle": 0, + "strokeColor": "#555555", + "backgroundColor": "transparent", + "fillStyle": "solid", + "strokeWidth": 1, + "strokeStyle": "solid", + "roughness": 1, + "opacity": 100, + "groupIds": [], + "frameId": null, + "roundness": null, + "seed": 1197271210, + "versionNonce": 1678547207, + "isDeleted": false, + "boundElements": [], + "updated": 1, + "link": null, + "locked": false, + "text": "descriptor ring", + "fontSize": 11, + "fontFamily": 1, + "textAlign": "left", + "verticalAlign": "top", + "containerId": null, + "originalText": "descriptor ring", + "lineHeight": 1.25, + "baseline": 11 + }, + { + "id": "r842479381", + "type": "rectangle", + "x": 40, + "y": 370, + "width": 210, + "height": 90, + "angle": 0, + "strokeColor": "#e8870c", + "backgroundColor": "#fff8ef", + "fillStyle": "solid", + "strokeWidth": 2, + "strokeStyle": "solid", + "roughness": 1, + "opacity": 100, + "groupIds": [], + "frameId": null, + "roundness": { + "type": 3 + }, + "seed": 1306297241, + "versionNonce": 259334440, + "isDeleted": false, + "boundElements": [], + "updated": 1, + "link": null, + "locked": false + }, + { + "id": "t247473806", + "type": "text", + "x": 50, + "y": 378, + "width": 130.9, + "height": 17.5, + "angle": 0, + "strokeColor": "#111111", + "backgroundColor": "transparent", + "fillStyle": "solid", + "strokeWidth": 1, + "strokeStyle": "solid", + "roughness": 1, + "opacity": 100, + "groupIds": [], + "frameId": null, + "roundness": null, + "seed": 948040342, + "versionNonce": 999321218, + "isDeleted": false, + "boundElements": [], + "updated": 1, + "link": null, + "locked": false, + "text": "fsync(2) in guest", + "fontSize": 14, + "fontFamily": 1, + "textAlign": "left", + "verticalAlign": "top", + "containerId": null, + "originalText": "fsync(2) in guest", + "lineHeight": 1.25, + "baseline": 14 + }, + { + "id": "t90729686", + "type": "text", + "x": 50, + "y": 398, + "width": 145.20000000000002, + "height": 13.75, + "angle": 0, + "strokeColor": "#555555", + "backgroundColor": "transparent", + "fillStyle": "solid", + "strokeWidth": 1, + "strokeStyle": "solid", + "roughness": 1, + "opacity": 100, + "groupIds": [], + "frameId": null, + "roundness": null, + "seed": 610965843, + "versionNonce": 1110950378, + "isDeleted": false, + "boundElements": [], + "updated": 1, + "link": null, + "locked": false, + "text": "-> VM exit (EPT/IO trap)", + "fontSize": 11, + "fontFamily": 1, + "textAlign": "left", + "verticalAlign": "top", + "containerId": null, + "originalText": "-> VM exit (EPT/IO trap)", + "lineHeight": 1.25, + "baseline": 11 + }, + { + "id": "r1919410251", + "type": "rectangle", + "x": 300, + "y": 370, + "width": 210, + "height": 90, + "angle": 0, + "strokeColor": "#e8870c", + "backgroundColor": "#fff8ef", + "fillStyle": "solid", + "strokeWidth": 2, + "strokeStyle": "solid", + "roughness": 1, + "opacity": 100, + "groupIds": [], + "frameId": null, + "roundness": { + "type": 3 + }, + "seed": 912863615, + "versionNonce": 952213405, + "isDeleted": false, + "boundElements": [], + "updated": 1, + "link": null, + "locked": false + }, + { + "id": "t741122302", + "type": "text", + "x": 310, + "y": 378, + "width": 123.20000000000002, + "height": 17.5, + "angle": 0, + "strokeColor": "#111111", + "backgroundColor": "transparent", + "fillStyle": "solid", + "strokeWidth": 1, + "strokeStyle": "solid", + "roughness": 1, + "opacity": 100, + "groupIds": [], + "frameId": null, + "roundness": null, + "seed": 1746141637, + "versionNonce": 1398018354, + "isDeleted": false, + "boundElements": [], + "updated": 1, + "link": null, + "locked": false, + "text": "KVM exit handler", + "fontSize": 14, + "fontFamily": 1, + "textAlign": "left", + "verticalAlign": "top", + "containerId": null, + "originalText": "KVM exit handler", + "lineHeight": 1.25, + "baseline": 14 + }, + { + "id": "t1705578092", + "type": "text", + "x": 310, + "y": 398, + "width": 175.45000000000002, + "height": 13.75, + "angle": 0, + "strokeColor": "#555555", + "backgroundColor": "transparent", + "fillStyle": "solid", + "strokeWidth": 1, + "strokeStyle": "solid", + "roughness": 1, + "opacity": 100, + "groupIds": [], + "frameId": null, + "roundness": null, + "seed": 982792183, + "versionNonce": 1039776681, + "isDeleted": false, + "boundElements": [], + "updated": 1, + "link": null, + "locked": false, + "text": "hands off to QEMU vCPU thread", + "fontSize": 11, + "fontFamily": 1, + "textAlign": "left", + "verticalAlign": "top", + "containerId": null, + "originalText": "hands off to QEMU vCPU thread", + "lineHeight": 1.25, + "baseline": 11 + }, + { + "id": "r270722878", + "type": "rectangle", + "x": 560, + "y": 370, + "width": 200, + "height": 90, + "angle": 0, + "strokeColor": "#e8870c", + "backgroundColor": "#fff8ef", + "fillStyle": "solid", + "strokeWidth": 2, + "strokeStyle": "solid", + "roughness": 1, + "opacity": 100, + "groupIds": [], + "frameId": null, + "roundness": { + "type": 3 + }, + "seed": 1806450011, + "versionNonce": 788833310, + "isDeleted": false, + "boundElements": [], + "updated": 1, + "link": null, + "locked": false + }, + { + "id": "t404278376", + "type": "text", + "x": 570, + "y": 378, + "width": 192.50000000000003, + "height": 17.5, + "angle": 0, + "strokeColor": "#111111", + "backgroundColor": "transparent", + "fillStyle": "solid", + "strokeWidth": 1, + "strokeStyle": "solid", + "roughness": 1, + "opacity": 100, + "groupIds": [], + "frameId": null, + "roundness": null, + "seed": 1074992400, + "versionNonce": 1768033434, + "isDeleted": false, + "boundElements": [], + "updated": 1, + "link": null, + "locked": false, + "text": "virtio-blk backend issues", + "fontSize": 14, + "fontFamily": 1, + "textAlign": "left", + "verticalAlign": "top", + "containerId": null, + "originalText": "virtio-blk backend issues", + "lineHeight": 1.25, + "baseline": 14 + }, + { + "id": "t1822082336", + "type": "text", + "x": 570, + "y": 398, + "width": 181.50000000000003, + "height": 13.75, + "angle": 0, + "strokeColor": "#555555", + "backgroundColor": "transparent", + "fillStyle": "solid", + "strokeWidth": 1, + "strokeStyle": "solid", + "roughness": 1, + "opacity": 100, + "groupIds": [], + "frameId": null, + "roundness": null, + "seed": 1342685959, + "versionNonce": 1389285571, + "isDeleted": false, + "boundElements": [], + "updated": 1, + "link": null, + "locked": false, + "text": "real host fsync, then VM enter", + "fontSize": 11, + "fontFamily": 1, + "textAlign": "left", + "verticalAlign": "top", + "containerId": null, + "originalText": "real host fsync, then VM enter", + "lineHeight": 1.25, + "baseline": 11 + }, + { + "id": "r1496285268", + "type": "rectangle", + "x": 800, + "y": 370, + "width": 120, + "height": 90, + "angle": 0, + "strokeColor": "#999999", + "backgroundColor": "#ffffff", + "fillStyle": "solid", + "strokeWidth": 2, + "strokeStyle": "dashed", + "roughness": 1, + "opacity": 100, + "groupIds": [], + "frameId": null, + "roundness": { + "type": 3 + }, + "seed": 1148649502, + "versionNonce": 1628048927, + "isDeleted": false, + "boundElements": [], + "updated": 1, + "link": null, + "locked": false + }, + { + "id": "t387817756", + "type": "text", + "x": 810, + "y": 378, + "width": 77.0, + "height": 17.5, + "angle": 0, + "strokeColor": "#111111", + "backgroundColor": "transparent", + "fillStyle": "solid", + "strokeWidth": 1, + "strokeStyle": "solid", + "roughness": 1, + "opacity": 100, + "groupIds": [], + "frameId": null, + "roundness": null, + "seed": 855044002, + "versionNonce": 1422564823, + "isDeleted": false, + "boundElements": [], + "updated": 1, + "link": null, + "locked": false, + "text": "round trip", + "fontSize": 14, + "fontFamily": 1, + "textAlign": "left", + "verticalAlign": "top", + "containerId": null, + "originalText": "round trip", + "lineHeight": 1.25, + "baseline": 14 + }, + { + "id": "t290803275", + "type": "text", + "x": 810, + "y": 398, + "width": 90.75000000000001, + "height": 13.75, + "angle": 0, + "strokeColor": "#555555", + "backgroundColor": "transparent", + "fillStyle": "solid", + "strokeWidth": 1, + "strokeStyle": "solid", + "roughness": 1, + "opacity": 100, + "groupIds": [], + "frameId": null, + "roundness": null, + "seed": 1163144690, + "versionNonce": 1175542804, + "isDeleted": false, + "boundElements": [], + "updated": 1, + "link": null, + "locked": false, + "text": "cost lands here", + "fontSize": 11, + "fontFamily": 1, + "textAlign": "left", + "verticalAlign": "top", + "containerId": null, + "originalText": "cost lands here", + "lineHeight": 1.25, + "baseline": 11 + }, + { + "id": "r1715490997", + "type": "rectangle", + "x": 40, + "y": 520, + "width": 280, + "height": 65, + "angle": 0, + "strokeColor": "#111111", + "backgroundColor": "#ffffff", + "fillStyle": "solid", + "strokeWidth": 2, + "strokeStyle": "solid", + "roughness": 1, + "opacity": 100, + "groupIds": [], + "frameId": null, + "roundness": { + "type": 3 + }, + "seed": 1555019079, + "versionNonce": 353791972, + "isDeleted": false, + "boundElements": [], + "updated": 1, + "link": null, + "locked": false + }, + { + "id": "t213901082", + "type": "text", + "x": 50, + "y": 528, + "width": 254.10000000000002, + "height": 17.5, + "angle": 0, + "strokeColor": "#111111", + "backgroundColor": "transparent", + "fillStyle": "solid", + "strokeWidth": 1, + "strokeStyle": "solid", + "roughness": 1, + "opacity": 100, + "groupIds": [], + "frameId": null, + "roundness": null, + "seed": 844232078, + "versionNonce": 909548404, + "isDeleted": false, + "boundElements": [], + "updated": 1, + "link": null, + "locked": false, + "text": "syscall (getpid): host ~88ns/call", + "fontSize": 14, + "fontFamily": 1, + "textAlign": "left", + "verticalAlign": "top", + "containerId": null, + "originalText": "syscall (getpid): host ~88ns/call", + "lineHeight": 1.25, + "baseline": 14 + }, + { + "id": "t1531154050", + "type": "text", + "x": 50, + "y": 548, + "width": 248.05, + "height": 13.75, + "angle": 0, + "strokeColor": "#555555", + "backgroundColor": "transparent", + "fillStyle": "solid", + "strokeWidth": 1, + "strokeStyle": "solid", + "roughness": 1, + "opacity": 100, + "groupIds": [], + "frameId": null, + "roundness": null, + "seed": 476056055, + "versionNonce": 340506745, + "isDeleted": false, + "boundElements": [], + "updated": 1, + "link": null, + "locked": false, + "text": "guest ~88ns/call \u2014 no vmexit on this path", + "fontSize": 11, + "fontFamily": 1, + "textAlign": "left", + "verticalAlign": "top", + "containerId": null, + "originalText": "guest ~88ns/call \u2014 no vmexit on this path", + "lineHeight": 1.25, + "baseline": 11 + }, + { + "id": "r372215246", + "type": "rectangle", + "x": 340, + "y": 520, + "width": 280, + "height": 65, + "angle": 0, + "strokeColor": "#111111", + "backgroundColor": "#ffffff", + "fillStyle": "solid", + "strokeWidth": 2, + "strokeStyle": "solid", + "roughness": 1, + "opacity": 100, + "groupIds": [], + "frameId": null, + "roundness": { + "type": 3 + }, + "seed": 1690273353, + "versionNonce": 1103763261, + "isDeleted": false, + "boundElements": [], + "updated": 1, + "link": null, + "locked": false + }, + { + "id": "t1364929840", + "type": "text", + "x": 350, + "y": 528, + "width": 238.70000000000002, + "height": 17.5, + "angle": 0, + "strokeColor": "#111111", + "backgroundColor": "transparent", + "fillStyle": "solid", + "strokeWidth": 1, + "strokeStyle": "solid", + "roughness": 1, + "opacity": 100, + "groupIds": [], + "frameId": null, + "roundness": null, + "seed": 139488595, + "versionNonce": 303923700, + "isDeleted": false, + "boundElements": [], + "updated": 1, + "link": null, + "locked": false, + "text": "mem (128MiB scan): host ~14GB/s", + "fontSize": 14, + "fontFamily": 1, + "textAlign": "left", + "verticalAlign": "top", + "containerId": null, + "originalText": "mem (128MiB scan): host ~14GB/s", + "lineHeight": 1.25, + "baseline": 14 + }, + { + "id": "t286562249", + "type": "text", + "x": 350, + "y": 548, + "width": 205.70000000000002, + "height": 13.75, + "angle": 0, + "strokeColor": "#555555", + "backgroundColor": "transparent", + "fillStyle": "solid", + "strokeWidth": 1, + "strokeStyle": "solid", + "roughness": 1, + "opacity": 100, + "groupIds": [], + "frameId": null, + "roundness": null, + "seed": 97365091, + "versionNonce": 819923240, + "isDeleted": false, + "boundElements": [], + "updated": 1, + "link": null, + "locked": false, + "text": "guest ~15GB/s \u2014 EPT cost amortized", + "fontSize": 11, + "fontFamily": 1, + "textAlign": "left", + "verticalAlign": "top", + "containerId": null, + "originalText": "guest ~15GB/s \u2014 EPT cost amortized", + "lineHeight": 1.25, + "baseline": 11 + }, + { + "id": "r1190424501", + "type": "rectangle", + "x": 640, + "y": 520, + "width": 280, + "height": 65, + "angle": 0, + "strokeColor": "#e8870c", + "backgroundColor": "#fff8ef", + "fillStyle": "solid", + "strokeWidth": 2, + "strokeStyle": "solid", + "roughness": 1, + "opacity": 100, + "groupIds": [], + "frameId": null, + "roundness": { + "type": 3 + }, + "seed": 353207448, + "versionNonce": 153623538, + "isDeleted": false, + "boundElements": [], + "updated": 1, + "link": null, + "locked": false + }, + { + "id": "t485770548", + "type": "text", + "x": 650, + "y": 528, + "width": 315.70000000000005, + "height": 17.5, + "angle": 0, + "strokeColor": "#111111", + "backgroundColor": "transparent", + "fillStyle": "solid", + "strokeWidth": 1, + "strokeStyle": "solid", + "roughness": 1, + "opacity": 100, + "groupIds": [], + "frameId": null, + "roundness": null, + "seed": 899727742, + "versionNonce": 1714669890, + "isDeleted": false, + "boundElements": [], + "updated": 1, + "link": null, + "locked": false, + "text": "io (create+write+fsync): host ~1.3k ops/s", + "fontSize": 14, + "fontFamily": 1, + "textAlign": "left", + "verticalAlign": "top", + "containerId": null, + "originalText": "io (create+write+fsync): host ~1.3k ops/s", + "lineHeight": 1.25, + "baseline": 14 + }, + { + "id": "t556268365", + "type": "text", + "x": 650, + "y": 548, + "width": 235.95000000000002, + "height": 13.75, + "angle": 0, + "strokeColor": "#555555", + "backgroundColor": "transparent", + "fillStyle": "solid", + "strokeWidth": 1, + "strokeStyle": "solid", + "roughness": 1, + "opacity": 100, + "groupIds": [], + "frameId": null, + "roundness": null, + "seed": 1493541370, + "versionNonce": 1250171363, + "isDeleted": false, + "boundElements": [], + "updated": 1, + "link": null, + "locked": false, + "text": "guest ~0.5k ops/s \u2014 the real vmexit tax", + "fontSize": 11, + "fontFamily": 1, + "textAlign": "left", + "verticalAlign": "top", + "containerId": null, + "originalText": "guest ~0.5k ops/s \u2014 the real vmexit tax", + "lineHeight": 1.25, + "baseline": 11 + }, + { + "id": "a527947253", + "type": "arrow", + "x": 240, + "y": 90, + "width": 30, + "height": 0, + "angle": 0, + "strokeColor": "#555555", + "backgroundColor": "transparent", + "fillStyle": "solid", + "strokeWidth": 2, + "strokeStyle": "solid", + "roughness": 1, + "opacity": 100, + "groupIds": [], + "frameId": null, + "roundness": { + "type": 2 + }, + "seed": 1186590527, + "versionNonce": 1037173539, + "isDeleted": false, + "boundElements": [], + "updated": 1, + "link": null, + "locked": false, + "points": [ + [ + 0, + 0 + ], + [ + 30, + 0 + ] + ], + "lastCommittedPoint": null, + "startBinding": null, + "endBinding": null, + "startArrowhead": null, + "endArrowhead": "arrow" + }, + { + "id": "t1936592310", + "type": "text", + "x": 255.0, + "y": 76.0, + "width": 84.7, + "height": 13.75, + "angle": 0, + "strokeColor": "#555555", + "backgroundColor": "transparent", + "fillStyle": "solid", + "strokeWidth": 1, + "strokeStyle": "solid", + "roughness": 1, + "opacity": 100, + "groupIds": [], + "frameId": null, + "roundness": null, + "seed": 419745572, + "versionNonce": 1775811597, + "isDeleted": false, + "boundElements": [], + "updated": 1, + "link": null, + "locked": false, + "text": "ioctl(KVM_RUN)", + "fontSize": 11, + "fontFamily": 1, + "textAlign": "left", + "verticalAlign": "top", + "containerId": null, + "originalText": "ioctl(KVM_RUN)", + "lineHeight": 1.25, + "baseline": 11 + }, + { + "id": "a1684017212", + "type": "arrow", + "x": 470, + "y": 90, + "width": 260, + "height": 0, + "angle": 0, + "strokeColor": "#555555", + "backgroundColor": "transparent", + "fillStyle": "solid", + "strokeWidth": 2, + "strokeStyle": "dashed", + "roughness": 1, + "opacity": 100, + "groupIds": [], + "frameId": null, + "roundness": { + "type": 2 + }, + "seed": 1544644163, + "versionNonce": 292697611, + "isDeleted": false, + "boundElements": [], + "updated": 1, + "link": null, + "locked": false, + "points": [ + [ + 0, + 0 + ], + [ + 260, + 0 + ] + ], + "lastCommittedPoint": null, + "startBinding": null, + "endBinding": null, + "startArrowhead": null, + "endArrowhead": "arrow" + }, + { + "id": "t190897266", + "type": "text", + "x": 600.0, + "y": 76.0, + "width": 90.75000000000001, + "height": 13.75, + "angle": 0, + "strokeColor": "#555555", + "backgroundColor": "transparent", + "fillStyle": "solid", + "strokeWidth": 1, + "strokeStyle": "solid", + "roughness": 1, + "opacity": 100, + "groupIds": [], + "frameId": null, + "roundness": null, + "seed": 1617558521, + "versionNonce": 299995312, + "isDeleted": false, + "boundElements": [], + "updated": 1, + "link": null, + "locked": false, + "text": "host write path", + "fontSize": 11, + "fontFamily": 1, + "textAlign": "left", + "verticalAlign": "top", + "containerId": null, + "originalText": "host write path", + "lineHeight": 1.25, + "baseline": 11 + }, + { + "id": "a61068238", + "type": "arrow", + "x": 260, + "y": 245, + "width": 30, + "height": 0, + "angle": 0, + "strokeColor": "#555555", + "backgroundColor": "transparent", + "fillStyle": "solid", + "strokeWidth": 2, + "strokeStyle": "solid", + "roughness": 1, + "opacity": 100, + "groupIds": [], + "frameId": null, + "roundness": { + "type": 2 + }, + "seed": 1596249662, + "versionNonce": 1514147875, + "isDeleted": false, + "boundElements": [], + "updated": 1, + "link": null, + "locked": false, + "points": [ + [ + 0, + 0 + ], + [ + 30, + 0 + ] + ], + "lastCommittedPoint": null, + "startBinding": null, + "endBinding": null, + "startArrowhead": null, + "endArrowhead": "arrow" + }, + { + "id": "t1679717166", + "type": "text", + "x": 275.0, + "y": 231.0, + "width": 66.55000000000001, + "height": 13.75, + "angle": 0, + "strokeColor": "#555555", + "backgroundColor": "transparent", + "fillStyle": "solid", + "strokeWidth": 1, + "strokeStyle": "solid", + "roughness": 1, + "opacity": 100, + "groupIds": [], + "frameId": null, + "roundness": null, + "seed": 659835904, + "versionNonce": 142574261, + "isDeleted": false, + "boundElements": [], + "updated": 1, + "link": null, + "locked": false, + "text": "raw syscall", + "fontSize": 11, + "fontFamily": 1, + "textAlign": "left", + "verticalAlign": "top", + "containerId": null, + "originalText": "raw syscall", + "lineHeight": 1.25, + "baseline": 11 + }, + { + "id": "a1190530532", + "type": "arrow", + "x": 510, + "y": 245, + "width": 40, + "height": 0, + "angle": 0, + "strokeColor": "#555555", + "backgroundColor": "transparent", + "fillStyle": "solid", + "strokeWidth": 2, + "strokeStyle": "solid", + "roughness": 1, + "opacity": 100, + "groupIds": [], + "frameId": null, + "roundness": { + "type": 2 + }, + "seed": 640669406, + "versionNonce": 151036904, + "isDeleted": false, + "boundElements": [], + "updated": 1, + "link": null, + "locked": false, + "points": [ + [ + 0, + 0 + ], + [ + 40, + 0 + ] + ], + "lastCommittedPoint": null, + "startBinding": null, + "endBinding": null, + "startArrowhead": null, + "endArrowhead": "arrow" + }, + { + "id": "t1115976963", + "type": "text", + "x": 530.0, + "y": 231.0, + "width": 54.45, + "height": 13.75, + "angle": 0, + "strokeColor": "#555555", + "backgroundColor": "transparent", + "fillStyle": "solid", + "strokeWidth": 1, + "strokeStyle": "solid", + "roughness": 1, + "opacity": 100, + "groupIds": [], + "frameId": null, + "roundness": null, + "seed": 1653220304, + "versionNonce": 251995447, + "isDeleted": false, + "boundElements": [], + "updated": 1, + "link": null, + "locked": false, + "text": "block req", + "fontSize": 11, + "fontFamily": 1, + "textAlign": "left", + "verticalAlign": "top", + "containerId": null, + "originalText": "block req", + "lineHeight": 1.25, + "baseline": 11 + }, + { + "id": "a223521380", + "type": "arrow", + "x": 750, + "y": 245, + "width": 40, + "height": 0, + "angle": 0, + "strokeColor": "#555555", + "backgroundColor": "transparent", + "fillStyle": "solid", + "strokeWidth": 2, + "strokeStyle": "solid", + "roughness": 1, + "opacity": 100, + "groupIds": [], + "frameId": null, + "roundness": { + "type": 2 + }, + "seed": 1806684397, + "versionNonce": 1195097067, + "isDeleted": false, + "boundElements": [], + "updated": 1, + "link": null, + "locked": false, + "points": [ + [ + 0, + 0 + ], + [ + 40, + 0 + ] + ], + "lastCommittedPoint": null, + "startBinding": null, + "endBinding": null, + "startArrowhead": null, + "endArrowhead": "arrow" + }, + { + "id": "t1320377333", + "type": "text", + "x": 770.0, + "y": 231.0, + "width": 42.35, + "height": 13.75, + "angle": 0, + "strokeColor": "#555555", + "backgroundColor": "transparent", + "fillStyle": "solid", + "strokeWidth": 1, + "strokeStyle": "solid", + "roughness": 1, + "opacity": 100, + "groupIds": [], + "frameId": null, + "roundness": null, + "seed": 1209331079, + "versionNonce": 717446272, + "isDeleted": false, + "boundElements": [], + "updated": 1, + "link": null, + "locked": false, + "text": "enqueue", + "fontSize": 11, + "fontFamily": 1, + "textAlign": "left", + "verticalAlign": "top", + "containerId": null, + "originalText": "enqueue", + "lineHeight": 1.25, + "baseline": 11 + }, + { + "id": "a856534223", + "type": "arrow", + "x": 250, + "y": 415, + "width": 50, + "height": 0, + "angle": 0, + "strokeColor": "#b8650a", + "backgroundColor": "transparent", + "fillStyle": "solid", + "strokeWidth": 2, + "strokeStyle": "solid", + "roughness": 1, + "opacity": 100, + "groupIds": [], + "frameId": null, + "roundness": { + "type": 2 + }, + "seed": 359177988, + "versionNonce": 587158177, + "isDeleted": false, + "boundElements": [], + "updated": 1, + "link": null, + "locked": false, + "points": [ + [ + 0, + 0 + ], + [ + 50, + 0 + ] + ], + "lastCommittedPoint": null, + "startBinding": null, + "endBinding": null, + "startArrowhead": null, + "endArrowhead": "arrow" + }, + { + "id": "t1630109431", + "type": "text", + "x": 275.0, + "y": 401.0, + "width": 40, + "height": 13.75, + "angle": 0, + "strokeColor": "#b8650a", + "backgroundColor": "transparent", + "fillStyle": "solid", + "strokeWidth": 1, + "strokeStyle": "solid", + "roughness": 1, + "opacity": 100, + "groupIds": [], + "frameId": null, + "roundness": null, + "seed": 478982267, + "versionNonce": 342421886, + "isDeleted": false, + "boundElements": [], + "updated": 1, + "link": null, + "locked": false, + "text": "vmexit", + "fontSize": 11, + "fontFamily": 1, + "textAlign": "left", + "verticalAlign": "top", + "containerId": null, + "originalText": "vmexit", + "lineHeight": 1.25, + "baseline": 11 + }, + { + "id": "a969131630", + "type": "arrow", + "x": 510, + "y": 415, + "width": 50, + "height": 0, + "angle": 0, + "strokeColor": "#b8650a", + "backgroundColor": "transparent", + "fillStyle": "solid", + "strokeWidth": 2, + "strokeStyle": "solid", + "roughness": 1, + "opacity": 100, + "groupIds": [], + "frameId": null, + "roundness": { + "type": 2 + }, + "seed": 504987868, + "versionNonce": 1324907384, + "isDeleted": false, + "boundElements": [], + "updated": 1, + "link": null, + "locked": false, + "points": [ + [ + 0, + 0 + ], + [ + 50, + 0 + ] + ], + "lastCommittedPoint": null, + "startBinding": null, + "endBinding": null, + "startArrowhead": null, + "endArrowhead": "arrow" + }, + { + "id": "t1915337312", + "type": "text", + "x": 535.0, + "y": 401.0, + "width": 40, + "height": 13.75, + "angle": 0, + "strokeColor": "#b8650a", + "backgroundColor": "transparent", + "fillStyle": "solid", + "strokeWidth": 1, + "strokeStyle": "solid", + "roughness": 1, + "opacity": 100, + "groupIds": [], + "frameId": null, + "roundness": null, + "seed": 442542795, + "versionNonce": 1622250762, + "isDeleted": false, + "boundElements": [], + "updated": 1, + "link": null, + "locked": false, + "text": "kick", + "fontSize": 11, + "fontFamily": 1, + "textAlign": "left", + "verticalAlign": "top", + "containerId": null, + "originalText": "kick", + "lineHeight": 1.25, + "baseline": 11 + }, + { + "id": "a50988272", + "type": "arrow", + "x": 760, + "y": 415, + "width": 40, + "height": 0, + "angle": 0, + "strokeColor": "#b8650a", + "backgroundColor": "transparent", + "fillStyle": "solid", + "strokeWidth": 2, + "strokeStyle": "solid", + "roughness": 1, + "opacity": 100, + "groupIds": [], + "frameId": null, + "roundness": { + "type": 2 + }, + "seed": 1426583834, + "versionNonce": 1639644564, + "isDeleted": false, + "boundElements": [], + "updated": 1, + "link": null, + "locked": false, + "points": [ + [ + 0, + 0 + ], + [ + 40, + 0 + ] + ], + "lastCommittedPoint": null, + "startBinding": null, + "endBinding": null, + "startArrowhead": null, + "endArrowhead": "arrow" + }, + { + "id": "t402526356", + "type": "text", + "x": 780.0, + "y": 401.0, + "width": 42.35, + "height": 13.75, + "angle": 0, + "strokeColor": "#b8650a", + "backgroundColor": "transparent", + "fillStyle": "solid", + "strokeWidth": 1, + "strokeStyle": "solid", + "roughness": 1, + "opacity": 100, + "groupIds": [], + "frameId": null, + "roundness": null, + "seed": 1700968889, + "versionNonce": 649235222, + "isDeleted": false, + "boundElements": [], + "updated": 1, + "link": null, + "locked": false, + "text": "vmenter", + "fontSize": 11, + "fontFamily": 1, + "textAlign": "left", + "verticalAlign": "top", + "containerId": null, + "originalText": "vmenter", + "lineHeight": 1.25, + "baseline": 11 + }, + { + "id": "t489152464", + "type": "text", + "x": 480, + "y": 155, + "width": 181.50000000000003, + "height": 13.75, + "angle": 0, + "strokeColor": "#b8650a", + "backgroundColor": "transparent", + "fillStyle": "solid", + "strokeWidth": 1, + "strokeStyle": "solid", + "roughness": 1, + "opacity": 100, + "groupIds": [], + "frameId": null, + "roundness": null, + "seed": 921052172, + "versionNonce": 1906826017, + "isDeleted": false, + "boundElements": [], + "updated": 1, + "link": null, + "locked": false, + "text": "VMX root <-> non-root boundary", + "fontSize": 11, + "fontFamily": 1, + "textAlign": "left", + "verticalAlign": "top", + "containerId": null, + "originalText": "VMX root <-> non-root boundary", + "lineHeight": 1.25, + "baseline": 11 + } + ], + "appState": { + "viewBackgroundColor": "#ffffff", + "gridSize": null + }, + "files": {} +} \ No newline at end of file diff --git a/assets/diagrams/35-kvm-virtio-stack.svg b/assets/diagrams/35-kvm-virtio-stack.svg new file mode 100644 index 0000000..f666920 --- /dev/null +++ b/assets/diagrams/35-kvm-virtio-stack.svg @@ -0,0 +1,79 @@ + + + +Host, VMX root mode (ring 0 host kernel) + +Guest, VMX non-root mode (ring 0 guest kernel, ring 3 guest process) + +Trapped path: virtio-blk write + fsync (a real vmexit) + +overheadbench measured deltas (this host, this guest) + +kvm.ko + kvm_intel.ko +/dev/kvm, EPT tables + +QEMU/VMM userspace +vCPU thread, virtio-blk backend + +Host NVMe/btrfs +real fsync(2) lands here + +guest kernel (ring 0) +getpid(2) served in-guest +no host trap needed + +overheadbench (ring 3) +syscall bench: raw getpid loop + +virtio-blk front-end driver +io bench: open/write/fsync + +virtqueue +descriptor ring + +fsync(2) in guest +-> VM exit (EPT/IO trap) + +KVM exit handler +hands off to QEMU vCPU thread + +virtio-blk backend issues +real host fsync, then VM enter + +round trip +cost lands here + +syscall (getpid): host ~88ns/call +guest ~88ns/call — no vmexit on this path + +mem (128MiB scan): host ~14GB/s +guest ~15GB/s — EPT cost amortized + +io (create+write+fsync): host ~1.3k ops/s +guest ~0.5k ops/s — the real vmexit tax + +ioctl(KVM_RUN) +ioctl(KVM_RUN) + +host write path +host write path + +raw syscall +raw syscall + +block req +block req + +enqueue +enqueue + +vmexit +vmexit + +kick +kick + +vmenter +vmenter +VMX root <-> non-root boundary + \ No newline at end of file diff --git a/assets/diagrams/README.md b/assets/diagrams/README.md index abc8e57..ed1edda 100644 --- a/assets/diagrams/README.md +++ b/assets/diagrams/README.md @@ -62,3 +62,13 @@ here as you add them: | `29-shadow-memory` | 29 | ASan's shadow-memory model — one shadow byte per 8 application bytes via `(app_addr >> 3) + offset` — with the real freed-block/redzone shadow bytes (`fa`/`fd`) this chapter's `uaf` run produced, and the measured 0.002s native vs 0.046s TSan vs 1.0s valgrind memcheck gap that the inline-check design explains | | `30-ebpf-tooling-stack` | 30 | the layered eBPF observation stack on systems-target — kprobe/uprobe/tracepoint/perf_event/BTF at the bottom, bcc-tools/bpftrace/bpftool/c++filt above it (every box exercised live this session), SystemTap's kernel-module vs dyninst backends beside it, and the three target binaries' uprobe arg0 outcome (correct for C++/Rust's SysV ABI, wrong for Go's ABIInternal) | | `30-uprobe-attachment` | 30 | one uprobe attach walked across all three binaries — C++'s and Rust's mangled `cmd_work` symbols demangled by `c++filt` (Itanium and Rust v0 both auto-detected) then attached by the mangled string, beside Go's plain unmangled `main.cmdWork` — with the real, measured RDI-vs-RAX argument-register mismatch that only the Go column hits | +| `35-kvm-virtio-stack` | 35 | VMX root (host kernel + QEMU/virtio-blk backend) versus non-root (guest kernel + overheadbench) modes, the getpid path that never leaves the guest beside the fsync path that really vmexits through the virtqueue back to host NVMe, annotated with the measured overheadbench syscall/mem/io deltas between systems-target and the host | +| `35-isolation-spectrum` | 35 | the four-point isolation spectrum — bare process, podman container (shared kernel), rust-vmm-based microVM (own minimal-device guest kernel), full KVM/QEMU VM (own full-device guest kernel) — each annotated with its escape surface and the measured overheadbench numbers that place container and full-VM on the chart | +| `33-seccomp-filter-flow` | 33 | one syscall's path through fwatch's seccomp-bpf filter — BPF_LD_ABS loading the syscall number, the compiled jump table (26 entries cpp/rust, 39 go), SECCOMP_RET_ALLOW on match, and the two possible default actions (SCMP_ACT_ERRNO, fwatch's own choice, beside the unused SCMP_ACT_KILL_PROCESS) — annotated with the real guest EPERM and SIGSYS transcripts | +| `33-landlock-ruleset` | 33 | the Landlock ruleset/rule model — landlock_create_ruleset probing the ABI (never hardcoded), landlock_add_rule binding READ_FILE\|READ_DIR to one directory fd, and landlock_restrict_self enforcing it irrevocably — with the real ABI split (host 9, systems-target guest 7) and the granted-vs-EACCES filesystem view from the chapter's probes | +| `32-namespace-overview` | 32 | host vs pmon containerize's PID-1 child across all eight namespace types, with the real `/proc//ns/*` inode numbers from systems-target — pid/mnt/uts/net differ (unshared), time/ipc/user/cgroup match the host (deliberately left alone) | +| `32-cgroup-hierarchy` | 32 | the cgroup v2 unified hierarchy from root (`cgroup.subtree_control` delegating +memory +cpu) down to the `pmon-oomdemo` leaf — real `memory.max`/`memory.swap.max`/`cpu.max`/PSI values — into the kernel's own `dmesg` OOM-kill line for pid 27013 this session produced | +| `32-clone3-flow` | 32 | C++/Rust's unshare-then-clone3(flags=0) shape beside Go's two attempts — the first (LockOSThread + raw unshare + os/exec re-exec) hitting a real `ENOMEM` on systems-target, and the shipped fix (Cloneflags on SysProcAttr, one clone(2) for everything) passing 13/13 across all three languages | +| `34-image-layering` | 34 | `app`'s multi-stage Containerfile — the discarded UBI10 builder (220 MB measured) installing a toolchain and statically linking the binary, `COPY --from=builder` into the UBI10-micro runtime stage (24.3 MB base) — with the real measured final image sizes (cpp/go 27.2 MB, rust 24.9 MB) barely above the bare micro base | +| `34-pid1-signal-flow` | 34 | `serve` vs `naive` as a container's PID 1 receiving SIGTERM — serve's signalfd/Notify/Signals wake, stop-spawning, forward-to-worker, reap, graceful exit sequence beside naive's kernel-masked default action that just drops the signal — annotated with the real measured `podman stop` timings (serve: cpp 192ms/go 60ms/rust 101ms; naive: cpp 2108ms/rust 2074ms SIGKILLed, go 60ms but silent) | +| `34-ns-debugging-map` | 34 | the two independent paths that reach the same container PID 1 — `podman exec` from inside the container runtime and a rootless `podman unshare nsenter` from the bare host (plain `nsenter` alone fails EPERM) — both confirming `Name: app` and `/proc/1/exe`, beside the real subuid/subgid mapping and a live zero-zombie scan | diff --git a/examples/32-namespaces-and-cgroups/README.md b/examples/32-namespaces-and-cgroups/README.md new file mode 100644 index 0000000..57ea525 --- /dev/null +++ b/examples/32-namespaces-and-cgroups/README.md @@ -0,0 +1,188 @@ +# 32 — Namespaces & cgroups (`pmon` v6) + +**VM example.** This chapter builds the two primitives every container +runtime is made of, by hand, with no runtime in between: **namespaces** to +give a process its own view of pid/mount/hostname/network, and a **cgroup +v2** subtree to cap what it's allowed to use. `pmon containerize` runs a +command inside both at once: + +``` +pmon containerize [--hostname NAME] [--mem-max BYTES|max] + [--cpu-max "QUOTA PERIOD"|max] [--cgroup NAME] + -- CMD [ARGS...] +``` + +It `unshare`s `CLONE_NEWNS|CLONE_NEWUTS|CLONE_NEWNET|CLONE_NEWPID`, writes +`memory.max`/`cpu.max` (and disables swap) on a dedicated cgroup *before* CMD +ever runs, forks the pid-1 child into that setup, and — as the child, before +handing off to CMD — reports the two things that are otherwise invisible from +the outside: + +``` +pmon: child sees pid 1 +pmon: hostname= +``` + +...then execs CMD. Once CMD exits, the parent reports the cgroup-side +evidence and CMD's fate: + +``` +pmon: cgroup mem.pressure some= +pmon: child exited status= # or: pmon: child killed signal= () +``` + +## Why this is the real test, not an inference + +- **pid 1** isn't a claim you can `strace` your way to from outside — a + process either sees itself as pid 1 or it doesn't, and only a genuine + `CLONE_NEWPID` child does. +- **The hostname** only proves the UTS namespace is real if it's compared + against the *actual* host hostname, fetched independently — otherwise + you're just checking that `sethostname(2)` didn't return an error. +- **The memory cap** only proves anything with a negative control: a hog + given far less `memory.max` than it tries to allocate must be + **OOM-killed** (`SIGKILL`, exit 137) before it finishes — not merely run + slower. A cgroup that let the hog complete would be decorative, not + enforcing. + +## `pmon` lineage + +Each chapter that grows `pmon` adds one thing under a subcommand of its own +(the CLI doesn't retroactively merge unrelated features into one command): + +| Version | Chapter | Subcommand(s) added | +|---|---|---| +| v0 | 11 — process lifecycle | `run -- CMD` | +| v1 | 12 — signals | `supervise` (sigchld restart loop) | +| v2 | 13 — pidfd | `supervise --engine pidfd\|sigchld` | +| v3 | 14 — identity & privilege | `drop --user`, `bindprobe` | +| v4 | 18 — pipes/FIFOs/splice | `supervise --log`, `tail --fifo` | +| v5 | 19 — UNIX sockets | `supervise --ctl`, `pmctl status\|stop\|logfd` | +| **v6** | **32 — namespaces & cgroups** | **`containerize`** | + +## The three implementations + +Same two syscalls-worth of work, three ways to reach them: + +| Language | Mechanism | +|---|---| +| **C++23** | Raw `unshare(2)` (mount/uts/net namespaces land on the caller immediately; the pid namespace is deferred to the next child), then raw `clone3(2)` via `` — `flags=0`, `stack=0` gives fork(2) semantics, `exit_signal=SIGCHLD` makes it `waitpid`-able. The child prints its proof lines, flushes, and `execvp`s CMD. `std::expected` carries every syscall failure; RAII (`Fd`) owns every descriptor; single-threaded throughout, on purpose — see the Go entry for what goes wrong when a runtime isn't. | +| **Rust (edition 2024)** | `nix::sched::unshare` + `nix::unistd::fork` (the same "no code runs between clone and exec except what we write" shape as C++'s `clone3(flags=0)`), `nix::mount::mount`, `nix::unistd::{sethostname, execvp}`. Errors flow through `Result`/`?`; cgroup file I/O is plain `std::fs`. | +| **Go 1.26** | The idiomatic inversion, one step further than chapter 14's: Go can't safely `fork()` and keep running Go code in the child (the runtime assumes a live multi-threaded process on both sides), so `unshare(2)` happens on a `runtime.LockOSThread()`-pinned goroutine, and the **fork+exec is a re-exec of pmon itself** (`/proc/self/exe __ns_init__ CMD...`) via `os/exec`, whose fork always runs on the calling goroutine's *current* OS thread — the one we just unshared and pinned. The freshly exec'd pmon is born inside the new namespaces (PID 1 included), prints the proof lines, and `syscall.Exec`s into the real CMD. `LockOSThread` is load-bearing: without it, the goroutine that called `Unshare` could be rescheduled onto a different, un-unshared OS thread before the fork, silently discarding every namespace change. | + +All three: identical CLI, output lines, and exit codes, so one `verify.lua` +covers them. + +## Cgroup mechanics worth knowing + +- **The root cgroup is exempt from the "no internal process" rule.** A + non-root cgroup can't hold processes directly *and* delegate a controller + to children at the same time — but the root cgroup can, which is exactly + how `containerize` enables `+memory +cpu` in `/sys/fs/cgroup/cgroup.subtree_control` + without first relocating itself into a leaf. +- **Cgroup membership is inherited across fork, not reset by exec.** + `containerize` writes its *own* pid into the new cgroup before unsharing + or forking, so the pid-1 child — and CMD after it execs — are under the + limits from their very first instruction. +- **`memory.swap.max=0` makes the limit mean something.** Without it, a + cgroup that hits `memory.max` may just swap and slow down instead of + triggering the OOM killer — on a host with swap enabled, the negative + control would flake. `containerize` sets it (best-effort) alongside + `memory.max`. +- **PSI (`memory.pressure`) is a per-cgroup file**, not a single systemwide + number — reading `/memory.pressure` after the child exits shows the + stall this specific cgroup experienced, independent of pressure anywhere + else on the box. + +## Demo contract + +Each language directory has the standard `demo.sh`: + +- `./demo.sh build` — build only +- `./demo.sh run [args]` — run the built binary (`app`) +- With env `TARGET` set, `run` deploys to that lab VM via + `scripts/lab/deploy-to-vm.sh`; set `SUDO=1` — `containerize` needs root for + `unshare(2)` and cgroup writes. + +The top-level `./demo.sh [cpp|go|rust|all|build]` dispatches per language. + +## Try it (on the lab VM) + +```sh +export LIBVIRT_DEFAULT_URI=qemu:///system TARGET=systems-target +cd examples/32-namespaces-and-cgroups + +# pid + uts namespaces: a trivial CMD proves both +SUDO=1 ./demo.sh cpp run containerize --hostname pmon-box -- /usr/bin/echo hi +# pmon: child sees pid 1 +# pmon: hostname=pmon-box +# hi +# pmon: cgroup mem.pressure some=0.00 +# pmon: child exited status=0 + +# negative control: a real memory hog under a small memory.max is OOM-killed +IP=$(../../scripts/lab/vm-ip.sh systems-target) +ssh "fedora@$IP" 'cat > /tmp/hog.sh' <<'EOF' +#!/bin/sh +a=$(head -c 200000000 /dev/zero | tr '\0' x) +sleep 2 +echo should-not-print +EOF +ssh "fedora@$IP" 'chmod +x /tmp/hog.sh' +SUDO=1 ./demo.sh cpp run containerize --mem-max 67108864 -- sh /tmp/hog.sh +# pmon: child sees pid 1 +# pmon: hostname=pmon-containerized +# pmon: cgroup mem.pressure some= +# pmon: child killed signal=9 (KILL) <- never reaches "should-not-print" +``` + +Cross-check from a second guest shell while a `containerize` run is live: +`nsenter -t -p -m -u -n ps -ef` shows the contained process tree with +its own pid 1 and hostname, from the host's side of the fence. + +## A deploy-to-vm.sh gotcha this example works around + +`deploy-to-vm.sh` rebuilds the remote command line with a bare `$*`, which +flattens argv and loses any quoting a single argument relied on. A `CMD` like +`bash -c 'a=$(head -c N /dev/zero | tr ...); sleep 2'` — one argument full of +spaces and shell metacharacters — arrives on the guest corrupted. Both +`verify.lua` and the walkthrough above route around it the same way: keep +every `CMD` token free of embedded spaces (`/usr/bin/echo hi`, `sh +/tmp/hog.sh`), staging anything that needs real shell logic as a file on the +guest first. + +## Verification + +`verify.lua` (run per language under the runner's **vm mode**, +`TARGET=systems-target` with `SUDO=1`) asserts: + +1. **`containerize -- /usr/bin/echo cmd-ran`** prints `pmon: child sees pid 1` + and `pmon: hostname=pmon-vm-check`, the echo's own output appears, a + `pmon: cgroup mem.pressure some=` line is present, and the run exits 0 + with `pmon: child exited status=0`. +2. The guest's **real** hostname is fetched independently over `ssh` and + asserted **different** from `pmon-vm-check` — the fact pmon is even + allowed to compare the two is only interesting because they're both real, + independently-obtained values. +3. **Negative control** — a real memory hog (`head -c 200000000 /dev/zero | + tr '\0' x` captured into a shell variable, staged as a guest script) run + under `containerize --mem-max 67108864` (64 MiB, far below the ~190 MiB it + tries to hold) exits **137**, prints `pmon: child killed signal=9 (KILL)`, + and never reaches its own trailing `echo should-not-print` — proof the + cgroup limit terminated it mid-allocation rather than merely slowing it + down. + +Run it: + +```sh +LIBVIRT_DEFAULT_URI=qemu:///system \ + python3 scripts/test-all-examples.py --only 32-namespaces-and-cgroups --mode vm +``` + +**Why observe the effect, not a trace:** there is no syscall you can +`strace` from *outside* a process to confirm it believes it's pid 1 — that +belief is only visible in what the process itself reports, or in what +happens to it. Pairing "reports pid 1" with an independently-fetched host +hostname, and pairing "cgroup configured" with "hog actually dies", is what +turns three plausible-looking print statements into a demonstration that the +namespaces and the cgroup limit are real. diff --git a/examples/32-namespaces-and-cgroups/cpp/.clang-format b/examples/32-namespaces-and-cgroups/cpp/.clang-format new file mode 100644 index 0000000..2c8640f --- /dev/null +++ b/examples/32-namespaces-and-cgroups/cpp/.clang-format @@ -0,0 +1,8 @@ +--- +BasedOnStyle: LLVM +Language: Cpp +Standard: c++20 +IndentWidth: 4 +ColumnLimit: 100 +PointerAlignment: Left +AllowShortFunctionsOnASingleLine: Inline diff --git a/examples/32-namespaces-and-cgroups/cpp/.clang-tidy b/examples/32-namespaces-and-cgroups/cpp/.clang-tidy new file mode 100644 index 0000000..2f275ba --- /dev/null +++ b/examples/32-namespaces-and-cgroups/cpp/.clang-tidy @@ -0,0 +1,11 @@ +--- +Checks: > + bugprone-*, + modernize-*, + performance-*, + readability-*, + -modernize-use-trailing-return-type, + -readability-identifier-length +WarningsAsErrors: '' +HeaderFilterRegex: '.*' +FormatStyle: file diff --git a/examples/32-namespaces-and-cgroups/cpp/CMakeLists.txt b/examples/32-namespaces-and-cgroups/cpp/CMakeLists.txt new file mode 100644 index 0000000..c945061 --- /dev/null +++ b/examples/32-namespaces-and-cgroups/cpp/CMakeLists.txt @@ -0,0 +1,8 @@ +cmake_minimum_required(VERSION 3.25) +project(pmon CXX) + +set(CMAKE_CXX_STANDARD 23) +set(CMAKE_CXX_STANDARD_REQUIRED ON) +set(CMAKE_CXX_EXTENSIONS OFF) + +add_executable(app src/main.cpp) diff --git a/examples/32-namespaces-and-cgroups/cpp/CMakePresets.json b/examples/32-namespaces-and-cgroups/cpp/CMakePresets.json new file mode 100644 index 0000000..e31bd2f --- /dev/null +++ b/examples/32-namespaces-and-cgroups/cpp/CMakePresets.json @@ -0,0 +1,62 @@ +{ + "version": 6, + "cmakeMinimumRequired": { "major": 3, "minor": 25, "patch": 0 }, + "configurePresets": [ + { + "name": "_base", + "hidden": true, + "generator": "Ninja", + "binaryDir": "${sourceDir}/build/${presetName}", + "cacheVariables": { + "CMAKE_EXPORT_COMPILE_COMMANDS": "ON" + } + }, + { + "name": "release", + "inherits": "_base", + "displayName": "Release (-O2 with debug info)", + "cacheVariables": { "CMAKE_BUILD_TYPE": "RelWithDebInfo" } + }, + { + "name": "debug", + "inherits": "_base", + "displayName": "Debug", + "cacheVariables": { "CMAKE_BUILD_TYPE": "Debug" } + }, + { + "name": "asan", + "inherits": "_base", + "displayName": "ASan + UBSan instrumented build", + "cacheVariables": { + "CMAKE_BUILD_TYPE": "Debug", + "CMAKE_CXX_FLAGS": "-O1 -g -fno-omit-frame-pointer -fsanitize=address,undefined", + "CMAKE_EXE_LINKER_FLAGS": "-fsanitize=address,undefined" + } + }, + { + "name": "release-clang", + "inherits": "release", + "displayName": "Release (clang)", + "cacheVariables": { + "CMAKE_C_COMPILER": "clang", + "CMAKE_CXX_COMPILER": "clang++" + } + }, + { + "name": "debug-clang", + "inherits": "debug", + "displayName": "Debug (clang)", + "cacheVariables": { + "CMAKE_C_COMPILER": "clang", + "CMAKE_CXX_COMPILER": "clang++" + } + } + ], + "buildPresets": [ + { "name": "release", "configurePreset": "release" }, + { "name": "debug", "configurePreset": "debug" }, + { "name": "asan", "configurePreset": "asan" }, + { "name": "release-clang", "configurePreset": "release-clang" }, + { "name": "debug-clang", "configurePreset": "debug-clang" } + ] +} diff --git a/examples/32-namespaces-and-cgroups/cpp/demo.sh b/examples/32-namespaces-and-cgroups/cpp/demo.sh new file mode 100755 index 0000000..6af6509 --- /dev/null +++ b/examples/32-namespaces-and-cgroups/cpp/demo.sh @@ -0,0 +1,40 @@ +#!/usr/bin/env bash +set -euo pipefail + +cd "$(dirname "${BASH_SOURCE[0]}")" +REPO_ROOT="$(cd ../../.. && pwd)" +BIN=build/release/app + +build() { + cmake --preset release + cmake --build --preset release +} + +run() { + if [[ ! -x "$BIN" ]]; then + build + fi + if [[ -n "${TARGET:-}" ]]; then + "$REPO_ROOT/scripts/lab/deploy-to-vm.sh" "$TARGET" "$BIN" -- "$@" + else + "./$BIN" "$@" + fi +} + +case "${1:-}" in + build) + build + ;; + run) + shift + run "$@" + ;; + "") + build + run + ;; + *) + echo "usage: $0 [build|run [args]]" >&2 + exit 2 + ;; +esac diff --git a/examples/32-namespaces-and-cgroups/cpp/src/main.cpp b/examples/32-namespaces-and-cgroups/cpp/src/main.cpp new file mode 100644 index 0000000..7b46ca6 --- /dev/null +++ b/examples/32-namespaces-and-cgroups/cpp/src/main.cpp @@ -0,0 +1,356 @@ +// pmon v6 — namespaces & cgroups (chapter 32). +// +// Subcommand: +// pmon containerize [--hostname NAME] [--mem-max BYTES|max] +// [--cpu-max "QUOTA PERIOD"|max] [--cgroup NAME] +// -- CMD [ARGS...] +// +// `containerize` builds the two primitives every real container runtime is +// made of, by hand: +// +// 1. NAMESPACES — unshare(2) with CLONE_NEWNS|CLONE_NEWUTS|CLONE_NEWNET| +// CLONE_NEWPID puts the calling process into a fresh mount, UTS and +// network namespace immediately, and arranges for the NEXT clone3(2) +// child to become PID 1 of a fresh PID namespace (unshare(2) never +// moves the caller itself into a new PID namespace — only its future +// children). +// 2. CGROUP v2 LIMITS — a dedicated cgroup gets memory.max and cpu.max +// written before the child is ever created, so the limits are already +// live the instant CMD execs. +// +// Raw clone3(2) via on purpose (no libc clone() wrapper, +// no fork()/vfork() distinction to reason about, and it is the syscall the +// chapter is actually teaching). RAII closes every fd; std::expected carries +// every syscall failure; single-threaded throughout (mixing threads with +// unshare()/clone3() is its own footgun — see the Go build's comment for +// what goes wrong when a runtime does that under you). + +#include +#include +#include +#include +#include +#include +#include +#include +#include +#include +#include + +#include +#include +#include +#include +#include +#include +#include +#include +#include + +namespace { + +// A syscall failure: what we were attempting, plus the captured errno. +struct SysErr { + std::string what; + int err; +}; + +// Wrap a raw syscall return: <0 becomes an error carrying errno. +std::expected checked(long rc, std::string_view what) { + if (rc < 0) { + return std::unexpected(SysErr{std::string(what), errno}); + } + return {}; +} + +// RAII owner for a file descriptor — closed exactly once on scope exit. +class Fd { + public: + explicit Fd(int fd) noexcept : fd_(fd) {} + ~Fd() { + if (fd_ >= 0) ::close(fd_); + } + Fd(const Fd&) = delete; + Fd& operator=(const Fd&) = delete; + Fd(Fd&& other) noexcept : fd_(other.fd_) { other.fd_ = -1; } + Fd& operator=(Fd&&) = delete; + + [[nodiscard]] int get() const noexcept { return fd_; } + [[nodiscard]] bool valid() const noexcept { return fd_ >= 0; } + + private: + int fd_ = -1; +}; + +// clone3(2)'s argument struct. We deliberately declare only the +// CLONE_ARGS_SIZE_VER0 prefix (through `tls`) — redefines +// the CLONE_NEW* macros already pulled in by , so we avoid including +// it and hand-roll the layout the syscall actually reads. +struct clone_args_v0 { + std::uint64_t flags; + std::uint64_t pidfd; + std::uint64_t child_tid; + std::uint64_t parent_tid; + std::uint64_t exit_signal; + std::uint64_t stack; + std::uint64_t stack_size; + std::uint64_t tls; +}; + +// ---- tiny sysfs/cgroupfs helpers ----------------------------------------- + +std::expected write_file(const std::string& path, std::string_view data) { + Fd fd(::open(path.c_str(), O_WRONLY)); + if (!fd.valid()) return std::unexpected(SysErr{"open " + path, errno}); + if (auto r = checked(::write(fd.get(), data.data(), data.size()), "write " + path); !r) { + return r; + } + return {}; +} + +std::expected read_file(const std::string& path) { + Fd fd(::open(path.c_str(), O_RDONLY)); + if (!fd.valid()) return std::unexpected(SysErr{"open " + path, errno}); + std::array buf{}; + ssize_t n = ::read(fd.get(), buf.data(), buf.size()); + if (n < 0) return std::unexpected(SysErr{"read " + path, errno}); + return std::string(buf.data(), static_cast(n)); +} + +// Whether `tokens` (a whitespace-separated list, as cgroupfs writes it) +// contains `word` as a whole token — "cpu" must not match inside "cpuset". +bool has_token(std::string_view tokens, std::string_view word) { + std::size_t pos = 0; + while (pos < tokens.size()) { + std::size_t end = tokens.find_first_of(" \n\t", pos); + if (end == std::string_view::npos) end = tokens.size(); + if (tokens.substr(pos, end - pos) == word) return true; + pos = end + 1; + } + return false; +} + +// The root cgroup is exempt from the "no internal process" constraint, so we +// can enable controllers there even though it holds every process on the +// box — this is exactly how a fresh sibling cgroup gets memory/cpu control +// without first relocating ourselves into a leaf. +std::expected ensure_root_controllers() { + auto have = read_file("/sys/fs/cgroup/cgroup.subtree_control"); + if (!have) return std::unexpected(have.error()); + bool needs_mem = !has_token(*have, "memory"); + bool needs_cpu = !has_token(*have, "cpu"); + if (!needs_mem && !needs_cpu) return {}; + std::string add; + if (needs_mem) add += "+memory "; + if (needs_cpu) add += "+cpu "; + return write_file("/sys/fs/cgroup/cgroup.subtree_control", add); +} + +// Create (or reuse) /sys/fs/cgroup/, apply the limits, and move the +// calling process into it. Every subsequent clone3(2) child inherits cgroup +// membership automatically, so CMD is under the limits from its first +// instruction. +std::expected setup_cgroup(const std::string& name, + const std::string& mem_max, + const std::string& cpu_max) { + if (auto r = ensure_root_controllers(); !r) return std::unexpected(r.error()); + + std::string path = "/sys/fs/cgroup/" + name; + if (::mkdir(path.c_str(), 0755) != 0 && errno != EEXIST) { + return std::unexpected(SysErr{"mkdir " + path, errno}); + } + if (auto r = write_file(path + "/memory.max", mem_max); !r) return std::unexpected(r.error()); + if (auto r = write_file(path + "/cpu.max", cpu_max); !r) return std::unexpected(r.error()); + // Best-effort: no swap headroom, so a breach of memory.max is a real OOM + // kill instead of a silent slowdown via swap. Not every kernel exposes + // swap accounting the same way, so a failure here isn't fatal. + (void)write_file(path + "/memory.swap.max", "0"); + + std::string pid_line = std::to_string(::getpid()); + if (auto r = write_file(path + "/cgroup.procs", pid_line); !r) return std::unexpected(r.error()); + + return path; +} + +// Names as printed in "killed signal= ()". A hand-rolled table +// (identical across all three implementations and to pmon's earlier +// chapters) rather than strsignal(3), whose text is locale-dependent prose. +[[nodiscard]] std::string signal_name(int sig) { + static constexpr std::array names{ + "", "HUP", "INT", "QUIT", "ILL", "TRAP", "ABRT", "BUS", + "FPE", "KILL", "USR1", "SEGV", "USR2", "PIPE", "ALRM", "TERM", + "STKFLT", "CHLD", "CONT", "STOP", "TSTP", "TTIN", "TTOU", "URG", + "XCPU", "XFSZ", "VTALRM", "PROF", "WINCH", "IO", "PWR", "SYS"}; + if (sig >= 1 && sig < static_cast(names.size())) { + return std::string{names[sig]}; + } + return "SIG" + std::to_string(sig); +} + +// Parse "some avg10=X avg60=Y avg300=Z total=T\n..." and return X. +std::string parse_psi_some_avg10(const std::string& psi) { + auto some_pos = psi.find("some "); + if (some_pos == std::string::npos) return "?"; + auto avg_pos = psi.find("avg10=", some_pos); + if (avg_pos == std::string::npos) return "?"; + avg_pos += std::strlen("avg10="); + auto end = psi.find(' ', avg_pos); + return psi.substr(avg_pos, end == std::string::npos ? std::string::npos : end - avg_pos); +} + +// ---- containerize ---------------------------------------------------------- + +int cmd_containerize(std::span args) { + std::string hostname = "pmon-containerized"; + std::string mem_max = "max"; + std::string cpu_max = "max 100000"; + std::string cgroup_name; + std::span cmd; + + for (std::size_t i = 0; i < args.size(); ++i) { + std::string_view a = args[i]; + if (a == "--hostname" && i + 1 < args.size()) { + hostname = args[++i]; + } else if (a == "--mem-max" && i + 1 < args.size()) { + mem_max = args[++i]; + } else if (a == "--cpu-max" && i + 1 < args.size()) { + cpu_max = args[++i]; + } else if (a == "--cgroup" && i + 1 < args.size()) { + cgroup_name = args[++i]; + } else if (a == "--") { + cmd = args.subspan(i + 1); + break; + } else { + std::println(stderr, "containerize: unexpected argument: {}", a); + return 2; + } + } + if (cmd.empty()) { + std::println(stderr, "containerize: missing -- CMD"); + return 2; + } + if (::getuid() != 0) { + std::println(stderr, "containerize: must run as root"); + return 1; + } + if (cgroup_name.empty()) { + cgroup_name = "pmon-" + std::to_string(::getpid()); + } + + auto cg = setup_cgroup(cgroup_name, mem_max, cpu_max); + if (!cg) { + std::println(stderr, "containerize: {}: {}", cg.error().what, std::strerror(cg.error().err)); + return 1; + } + const std::string cgroup_path = *cg; + + // Mount, UTS and network namespaces take effect on THIS process right now. + // The PID namespace is deferred: the next clone3(2) child becomes PID 1 of + // a fresh one; we (the caller) stay in our original PID namespace so we + // can waitpid(2) the child the ordinary way. + if (auto r = checked(::unshare(CLONE_NEWNS | CLONE_NEWUTS | CLONE_NEWNET | CLONE_NEWPID), + "unshare"); + !r) { + std::println(stderr, "containerize: {}: {}", r.error().what, std::strerror(r.error().err)); + return 1; + } + + // Detach mount propagation recursively so nothing we do in this namespace + // (or CMD does) leaks a mount event back to the host's mount table. + if (auto r = checked(::mount("none", "/", nullptr, MS_REC | MS_PRIVATE, nullptr), + "mount MS_PRIVATE"); + !r) { + std::println(stderr, "containerize: {}: {}", r.error().what, std::strerror(r.error().err)); + return 1; + } + + // sethostname(2) now lands in the fresh UTS namespace, not the host's. + if (auto r = checked(::sethostname(hostname.c_str(), hostname.size()), "sethostname"); !r) { + std::println(stderr, "containerize: {}: {}", r.error().what, std::strerror(r.error().err)); + return 1; + } + + // clone3(2): flags=0 (the namespace work is already done above), stack=0 + // (kernel copy-on-write duplicates our stack exactly like fork(2)), + // exit_signal=SIGCHLD (so waitpid(2) below works the ordinary way). + clone_args_v0 cl{}; + cl.exit_signal = SIGCHLD; + long rc = ::syscall(SYS_clone3, &cl, sizeof(cl)); + if (rc < 0) { + std::println(stderr, "containerize: clone3: {}", std::strerror(errno)); + return 1; + } + + if (rc == 0) { + // --- child: PID 1 of the new PID namespace --- + if (::getpid() == 1) { + std::println("pmon: child sees pid 1"); + } + std::fflush(stdout); + + char buf[256]{}; + if (::gethostname(buf, sizeof(buf) - 1) == 0) { + std::println("pmon: hostname={}", buf); + } + std::fflush(stdout); + + std::vector argv(cmd.begin(), cmd.end()); + argv.push_back(nullptr); + ::execvp(argv[0], argv.data()); + std::println(stderr, "pmon: exec {}: {}", argv[0], std::strerror(errno)); + std::fflush(stderr); + _exit(127); + } + + // --- parent: waits, then reports the cgroup-side evidence --- + int status = 0; + if (::waitpid(static_cast(rc), &status, 0) < 0) { + std::println(stderr, "containerize: waitpid: {}", std::strerror(errno)); + return 1; + } + + auto psi = read_file(cgroup_path + "/memory.pressure"); + if (psi) { + std::println("pmon: cgroup mem.pressure some={}", parse_psi_some_avg10(*psi)); + } + + int exit_code = 0; + if (WIFEXITED(status)) { + int code = WEXITSTATUS(status); + std::println("pmon: child exited status={}", code); + exit_code = code; + } else if (WIFSIGNALED(status)) { + int sig = WTERMSIG(status); + std::println("pmon: child killed signal={} ({})", sig, signal_name(sig)); + exit_code = 128 + sig; + } + + // Best-effort cleanup; a live cgroup left behind is harmless (reused by + // its own name next run) and not worth failing the exit code over. + ::rmdir(cgroup_path.c_str()); + + return exit_code; +} + +void usage() { + std::println(stderr, + "usage:\n" + " pmon containerize [--hostname NAME] [--mem-max BYTES|max]\n" + " [--cpu-max \"QUOTA PERIOD\"|max] [--cgroup NAME]\n" + " -- CMD [ARGS...]"); +} + +} // namespace + +int main(int argc, char** argv) { + std::span all(argv, static_cast(argc)); + if (all.size() < 2) { + usage(); + return 2; + } + std::string_view sub = all[1]; + std::span rest = all.subspan(2); + if (sub == "containerize") return cmd_containerize(rest); + usage(); + return 2; +} diff --git a/examples/32-namespaces-and-cgroups/demo.sh b/examples/32-namespaces-and-cgroups/demo.sh new file mode 100755 index 0000000..afbbe4c --- /dev/null +++ b/examples/32-namespaces-and-cgroups/demo.sh @@ -0,0 +1,23 @@ +#!/usr/bin/env bash +set -euo pipefail +cd "$(dirname "$0")" + +langs=(cpp go rust) + +case "${1:-all}" in + cpp|go|rust) + lang="$1" + shift + exec "./$lang/demo.sh" "$@" + ;; + all) + for lang in "${langs[@]}"; do "./$lang/demo.sh"; done + ;; + build) + for lang in "${langs[@]}"; do "./$lang/demo.sh" build; done + ;; + *) + echo "usage: $0 [cpp|go|rust|all|build] [args...]" >&2 + exit 2 + ;; +esac diff --git a/examples/32-namespaces-and-cgroups/go/demo.sh b/examples/32-namespaces-and-cgroups/go/demo.sh new file mode 100755 index 0000000..28606df --- /dev/null +++ b/examples/32-namespaces-and-cgroups/go/demo.sh @@ -0,0 +1,23 @@ +#!/usr/bin/env bash +set -euo pipefail +cd "$(dirname "$0")" +REPO_ROOT="$(cd ../../.. && pwd)" +BIN=bin/app + +build() { + go build -o "$BIN" . +} + +run() { + if [[ -n "${TARGET:-}" ]]; then + exec "$REPO_ROOT/scripts/lab/deploy-to-vm.sh" "$TARGET" "$BIN" -- "$@" + fi + exec "./$BIN" "$@" +} + +case "${1:-}" in + build) build ;; + run) shift; run "$@" ;; + "") build; run ;; + *) echo "usage: $0 [build|run [args...]]" >&2; exit 2 ;; +esac diff --git a/examples/32-namespaces-and-cgroups/go/go.mod b/examples/32-namespaces-and-cgroups/go/go.mod new file mode 100644 index 0000000..29ceeba --- /dev/null +++ b/examples/32-namespaces-and-cgroups/go/go.mod @@ -0,0 +1,7 @@ +module github.com/patterncatalyst/linux-systems-programming/examples/32-namespaces-and-cgroups/go + +go 1.26 + +toolchain go1.26.5 + +require golang.org/x/sys v0.47.0 diff --git a/examples/32-namespaces-and-cgroups/go/go.sum b/examples/32-namespaces-and-cgroups/go/go.sum new file mode 100644 index 0000000..37ee2d4 --- /dev/null +++ b/examples/32-namespaces-and-cgroups/go/go.sum @@ -0,0 +1,2 @@ +golang.org/x/sys v0.47.0 h1:o7XGOvZQCADBQQ4Y7VNq2dRWQR7JmOUW8Kxx4ZsNgWs= +golang.org/x/sys v0.47.0/go.mod h1:4GL1E5IUh+htKOUEOaiffhrAeqysfVGipDYzABqnCmw= diff --git a/examples/32-namespaces-and-cgroups/go/main.go b/examples/32-namespaces-and-cgroups/go/main.go new file mode 100644 index 0000000..cd81b7b --- /dev/null +++ b/examples/32-namespaces-and-cgroups/go/main.go @@ -0,0 +1,355 @@ +// pmon v6 — namespaces & cgroups (chapter 32). +// +// Subcommand: +// +// pmon containerize [--hostname NAME] [--mem-max BYTES|max] +// [--cpu-max "QUOTA PERIOD"|max] [--cgroup NAME] +// -- CMD [ARGS...] +// +// Go note — why this can't be the straight-line fork(2) the C++/Rust builds +// use: +// +// A Go program is multithreaded before main() runs, and unshare(2) changes +// only the CALLING THREAD's namespaces — the first attempt below called +// unix.Unshare directly on a runtime.LockOSThread()-pinned goroutine, then +// re-exec'd the binary through os/exec so the fork would land on that same, +// already-unshared thread. It built, and the pid/hostname proof lines were +// real, but the memory-hog negative control failed on real hardware with +// "fork/exec: cannot allocate memory" (see the chapter's Errors section) — +// a genuine kernel restriction: unshare(CLONE_NEWPID) lets the caller's +// process create exactly ONE child in the fresh PID namespace before further +// forks there return ENOMEM, and os/exec's own vfork/pidfd fast path +// performs an internal probe fork on first use that silently consumes that +// one shot. +// +// The fix — and the one actually shipped here — is the same move Chapter +// 14 made for capabilities: stop calling the namespace syscall directly in +// live Go code, and instead describe it declaratively on SysProcAttr, so the +// runtime's own fork/exec trampoline (which never has this problem — it +// creates the namespaces AND the real child in the same clone(2)) applies +// it. `Cloneflags` on the re-exec's SysProcAttr creates the child already +// inside fresh mount/uts/net/pid namespaces, in one step, with no prior +// unshare() and no LockOSThread needed at all. The re-exec target — pmon +// itself, invoked with the hidden "__ns_init__" marker — is therefore where +// the mount-private remount and sethostname now run: it is already living +// inside the new namespaces the instant its code starts, since they were +// established at clone(2) time and carried across its own execve. +package main + +import ( + "errors" + "fmt" + "os" + "os/exec" + "strconv" + "strings" + "syscall" + + "golang.org/x/sys/unix" +) + +// Hidden re-exec marker — not a public subcommand, so it stays out of usage(). +const reexecMarker = "__ns_init__" + +// Names as printed in "killed signal= ()". A hand-rolled table +// (identical across all three implementations and to pmon's earlier +// chapters) rather than the runtime's Signal.String(), whose text +// ("killed", "terminated") is prose, not the C convention. +var signalNames = [32]string{ + "", "HUP", "INT", "QUIT", "ILL", "TRAP", "ABRT", "BUS", + "FPE", "KILL", "USR1", "SEGV", "USR2", "PIPE", "ALRM", "TERM", + "STKFLT", "CHLD", "CONT", "STOP", "TSTP", "TTIN", "TTOU", "URG", + "XCPU", "XFSZ", "VTALRM", "PROF", "WINCH", "IO", "PWR", "SYS", +} + +func signalName(sig int) string { + if sig >= 1 && sig < len(signalNames) { + return signalNames[sig] + } + return fmt.Sprintf("SIG%d", sig) +} + +// ---- cgroup helpers -------------------------------------------------- + +// hasToken reports whether tokens (a whitespace-separated list, as +// cgroupfs writes it) contains word as a whole token — "cpu" must not +// match inside "cpuset". +func hasToken(tokens, word string) bool { + for _, t := range strings.Fields(tokens) { + if t == word { + return true + } + } + return false +} + +// The root cgroup is exempt from the "no internal process" constraint, so +// controllers can be enabled there even though it holds every process on +// the box — this is how a fresh sibling cgroup gets memory/cpu control +// without first relocating ourselves into a leaf. +func ensureRootControllers() error { + data, err := os.ReadFile("/sys/fs/cgroup/cgroup.subtree_control") + if err != nil { + return fmt.Errorf("read cgroup.subtree_control: %w", err) + } + have := string(data) + var add string + if !hasToken(have, "memory") { + add += "+memory " + } + if !hasToken(have, "cpu") { + add += "+cpu " + } + if add == "" { + return nil + } + if err := os.WriteFile("/sys/fs/cgroup/cgroup.subtree_control", []byte(add), 0o644); err != nil { + return fmt.Errorf("enable controllers: %w", err) + } + return nil +} + +// setupCgroup creates (or reuses) /sys/fs/cgroup/, applies the +// limits, and moves the calling process into it. Every subsequent +// fork/exec inherits cgroup membership automatically, so CMD is under the +// limits from its first instruction. +func setupCgroup(name, memMax, cpuMax string) (string, error) { + if err := ensureRootControllers(); err != nil { + return "", err + } + path := "/sys/fs/cgroup/" + name + if err := os.Mkdir(path, 0o755); err != nil && !os.IsExist(err) { + return "", fmt.Errorf("mkdir %s: %w", path, err) + } + if err := os.WriteFile(path+"/memory.max", []byte(memMax), 0o644); err != nil { + return "", fmt.Errorf("write memory.max: %w", err) + } + if err := os.WriteFile(path+"/cpu.max", []byte(cpuMax), 0o644); err != nil { + return "", fmt.Errorf("write cpu.max: %w", err) + } + // Best-effort: no swap headroom, so a breach of memory.max is a real + // OOM kill instead of a silent slowdown via swap. Not every kernel + // exposes swap accounting the same way, so a failure here isn't fatal. + _ = os.WriteFile(path+"/memory.swap.max", []byte("0"), 0o644) + pid := strconv.Itoa(os.Getpid()) + if err := os.WriteFile(path+"/cgroup.procs", []byte(pid), 0o644); err != nil { + return "", fmt.Errorf("write cgroup.procs: %w", err) + } + return path, nil +} + +// parsePSISomeAvg10 extracts X from "some avg10=X avg60=Y avg300=Z total=T". +func parsePSISomeAvg10(psi string) string { + idx := strings.Index(psi, "some ") + if idx < 0 { + return "?" + } + rest := psi[idx:] + a := strings.Index(rest, "avg10=") + if a < 0 { + return "?" + } + rest = rest[a+len("avg10="):] + if sp := strings.IndexByte(rest, ' '); sp >= 0 { + rest = rest[:sp] + } + return rest +} + +// ---- containerize (outer launcher) ------------------------------------ + +func cmdContainerize(args []string) int { + hostname := "pmon-containerized" + memMax := "max" + cpuMax := "max 100000" + cgroupName := "" + var cmd []string + + for i := 0; i < len(args); i++ { + switch args[i] { + case "--hostname": + if i+1 >= len(args) { + fmt.Fprintln(os.Stderr, "containerize: --hostname needs a value") + return 2 + } + hostname = args[i+1] + i++ + case "--mem-max": + if i+1 >= len(args) { + fmt.Fprintln(os.Stderr, "containerize: --mem-max needs a value") + return 2 + } + memMax = args[i+1] + i++ + case "--cpu-max": + if i+1 >= len(args) { + fmt.Fprintln(os.Stderr, "containerize: --cpu-max needs a value") + return 2 + } + cpuMax = args[i+1] + i++ + case "--cgroup": + if i+1 >= len(args) { + fmt.Fprintln(os.Stderr, "containerize: --cgroup needs a value") + return 2 + } + cgroupName = args[i+1] + i++ + case "--": + cmd = args[i+1:] + i = len(args) + default: + fmt.Fprintf(os.Stderr, "containerize: unexpected argument: %s\n", args[i]) + return 2 + } + } + if len(cmd) == 0 { + fmt.Fprintln(os.Stderr, "containerize: missing -- CMD") + return 2 + } + if os.Getuid() != 0 { + fmt.Fprintln(os.Stderr, "containerize: must run as root") + return 1 + } + if cgroupName == "" { + cgroupName = fmt.Sprintf("pmon-%d", os.Getpid()) + } + + cgroupPath, err := setupCgroup(cgroupName, memMax, cpuMax) + if err != nil { + fmt.Fprintf(os.Stderr, "containerize: %v\n", err) + return 1 + } + + self, err := os.Executable() + if err != nil { + fmt.Fprintf(os.Stderr, "containerize: os.Executable: %v\n", err) + return 1 + } + // The hostname has to travel to the re-exec'd child: it, not this + // process, is the one that ends up inside the fresh UTS namespace (see + // the package comment). + initArgs := append([]string{reexecMarker, hostname, "--"}, cmd...) + c := exec.Command(self, initArgs...) + c.Stdin, c.Stdout, c.Stderr = os.Stdin, os.Stdout, os.Stderr + // Cloneflags asks the runtime's own fork/exec trampoline to create the + // child already inside fresh mount/uts/net/pid namespaces, atomically, + // in the SAME clone(2) that creates the process — no separate unshare(2) + // on our side, and so none of the "one child per fresh PID namespace" + // restriction that broke the direct-unshare attempt. + c.SysProcAttr = &syscall.SysProcAttr{ + Cloneflags: syscall.CLONE_NEWNS | syscall.CLONE_NEWUTS | syscall.CLONE_NEWNET | syscall.CLONE_NEWPID, + } + if err := c.Start(); err != nil { + fmt.Fprintf(os.Stderr, "containerize: exec %s: %v\n", cmd[0], err) + return 1 + } + + waitErr := c.Wait() + + if psi, err := os.ReadFile(cgroupPath + "/memory.pressure"); err == nil { + fmt.Printf("pmon: cgroup mem.pressure some=%s\n", parsePSISomeAvg10(string(psi))) + } + + exitCode := 0 + switch { + case waitErr == nil: + fmt.Println("pmon: child exited status=0") + default: + var ee *exec.ExitError + if errors.As(waitErr, &ee) { + ws := ee.Sys().(syscall.WaitStatus) + if ws.Signaled() { + sig := int(ws.Signal()) + fmt.Printf("pmon: child killed signal=%d (%s)\n", sig, signalName(sig)) + exitCode = 128 + sig + } else { + exitCode = ws.ExitStatus() + fmt.Printf("pmon: child exited status=%d\n", exitCode) + } + } else { + fmt.Fprintf(os.Stderr, "containerize: wait: %v\n", waitErr) + exitCode = 1 + } + } + + _ = os.Remove(cgroupPath) // best-effort; a leftover cgroup is harmless + + return exitCode +} + +// ---- __ns_init__ (the re-exec'd pid-1 stub) --------------------------- +// +// args: -- CMD [ARGS...]. Already running inside the fresh +// mount/uts/net/pid namespaces (Cloneflags put THIS process there at +// clone(2) time, before this code's own execve even happened) — but the +// mount-private remount and sethostname are still ours to do, since the +// parent process never entered these namespaces itself. Then prints the +// proof lines and execs into the real CMD, replacing itself — CMD keeps +// PID 1. +func cmdNsInit(args []string) int { + if len(args) < 2 || args[1] != "--" { + fmt.Fprintln(os.Stderr, "__ns_init__: expected -- CMD") + return 2 + } + hostname := args[0] + cmd := args[2:] + if len(cmd) == 0 { + fmt.Fprintln(os.Stderr, "__ns_init__: missing CMD") + return 2 + } + + // Detach mount propagation recursively so nothing here (or CMD) leaks a + // mount event back to the host's mount table. + if err := unix.Mount("none", "/", "", unix.MS_REC|unix.MS_PRIVATE, ""); err != nil { + fmt.Fprintf(os.Stderr, "__ns_init__: mount MS_PRIVATE: %v\n", err) + return 1 + } + // sethostname(2) lands in the fresh UTS namespace, not the host's. + if err := unix.Sethostname([]byte(hostname)); err != nil { + fmt.Fprintf(os.Stderr, "__ns_init__: sethostname: %v\n", err) + return 1 + } + + if os.Getpid() == 1 { + fmt.Println("pmon: child sees pid 1") + } + if hn, err := os.Hostname(); err == nil { + fmt.Printf("pmon: hostname=%s\n", hn) + } + path, err := exec.LookPath(cmd[0]) + if err != nil { + fmt.Fprintf(os.Stderr, "pmon: exec %s: %v\n", cmd[0], err) + return 127 + } + if err := syscall.Exec(path, cmd, os.Environ()); err != nil { + fmt.Fprintf(os.Stderr, "pmon: exec %s: %v\n", cmd[0], err) + return 127 + } + return 0 // unreachable: exec never returns on success +} + +func usage() { + fmt.Fprintln(os.Stderr, "usage:") + fmt.Fprintln(os.Stderr, " pmon containerize [--hostname NAME] [--mem-max BYTES|max]") + fmt.Fprintln(os.Stderr, " [--cpu-max \"QUOTA PERIOD\"|max] [--cgroup NAME]") + fmt.Fprintln(os.Stderr, " -- CMD [ARGS...]") +} + +func main() { + if len(os.Args) < 2 { + usage() + os.Exit(2) + } + var code int + switch os.Args[1] { + case "containerize": + code = cmdContainerize(os.Args[2:]) + case reexecMarker: + code = cmdNsInit(os.Args[2:]) + default: + usage() + code = 2 + } + os.Exit(code) +} diff --git a/examples/32-namespaces-and-cgroups/rust/Cargo.lock b/examples/32-namespaces-and-cgroups/rust/Cargo.lock new file mode 100644 index 0000000..aa8bf3f --- /dev/null +++ b/examples/32-namespaces-and-cgroups/rust/Cargo.lock @@ -0,0 +1,46 @@ +# This file is automatically @generated by Cargo. +# It is not intended for manual editing. +version = 4 + +[[package]] +name = "bitflags" +version = "2.13.1" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "b588b76d00fde79687d7646a9b5bdf3cc0f655e0bbd080335a95d7e96f3587da" + +[[package]] +name = "cfg-if" +version = "1.0.4" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "9330f8b2ff13f34540b44e946ef35111825727b38d33286ef986142615121801" + +[[package]] +name = "cfg_aliases" +version = "0.2.2" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "f079e83a288787bcd14a6aea84cee5c87a67c5a3e660c30f557a3d24761b3527" + +[[package]] +name = "libc" +version = "0.2.186" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "68ab91017fe16c622486840e4c83c9a37afeff978bd239b5293d61ece587de66" + +[[package]] +name = "nix" +version = "0.31.3" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "cf20d2fde8ff38632c426f1165ed7436270b44f199fc55284c38276f9db47c3d" +dependencies = [ + "bitflags", + "cfg-if", + "cfg_aliases", + "libc", +] + +[[package]] +name = "pmon" +version = "6.0.0" +dependencies = [ + "nix", +] diff --git a/examples/32-namespaces-and-cgroups/rust/Cargo.toml b/examples/32-namespaces-and-cgroups/rust/Cargo.toml new file mode 100644 index 0000000..e433af9 --- /dev/null +++ b/examples/32-namespaces-and-cgroups/rust/Cargo.toml @@ -0,0 +1,11 @@ +[package] +name = "pmon" +version = "6.0.0" +edition = "2024" + +[[bin]] +name = "app" +path = "src/main.rs" + +[dependencies] +nix = { version = "0.31.3", features = ["sched", "mount", "hostname", "process", "fs", "signal", "user"] } diff --git a/examples/32-namespaces-and-cgroups/rust/demo.sh b/examples/32-namespaces-and-cgroups/rust/demo.sh new file mode 100755 index 0000000..220feea --- /dev/null +++ b/examples/32-namespaces-and-cgroups/rust/demo.sh @@ -0,0 +1,26 @@ +#!/usr/bin/env bash +set -euo pipefail +cd "$(dirname "${BASH_SOURCE[0]}")" + +BIN=target/release/app + +build() { + cargo build --release +} + +run() { + if [[ -n "${TARGET:-}" ]]; then + local repo_root + repo_root=$(cd ../../.. && pwd) + "$repo_root/scripts/lab/deploy-to-vm.sh" "$TARGET" "$BIN" -- "$@" + else + "./$BIN" "$@" + fi +} + +case "${1:-}" in + build) build ;; + run) shift; run "$@" ;; + "") build; run ;; + *) echo "usage: $0 [build|run [args...]]" >&2; exit 2 ;; +esac diff --git a/examples/32-namespaces-and-cgroups/rust/rust-toolchain.toml b/examples/32-namespaces-and-cgroups/rust/rust-toolchain.toml new file mode 100644 index 0000000..4099f1a --- /dev/null +++ b/examples/32-namespaces-and-cgroups/rust/rust-toolchain.toml @@ -0,0 +1,3 @@ +[toolchain] +channel = "1.97.1" +components = ["rustfmt", "clippy", "rust-src"] diff --git a/examples/32-namespaces-and-cgroups/rust/src/main.rs b/examples/32-namespaces-and-cgroups/rust/src/main.rs new file mode 100644 index 0000000..67f1e82 --- /dev/null +++ b/examples/32-namespaces-and-cgroups/rust/src/main.rs @@ -0,0 +1,314 @@ +//! pmon v6 — namespaces & cgroups (chapter 32). +//! +//! Subcommand: +//! pmon containerize [--hostname NAME] [--mem-max BYTES|max] +//! [--cpu-max "QUOTA PERIOD"|max] [--cgroup NAME] +//! -- CMD [ARGS...] +//! +//! Same two primitives as the C++ build, through `nix` instead of raw libc +//! calls: +//! +//! 1. NAMESPACES — unshare(2) with CLONE_NEWNS|CLONE_NEWUTS|CLONE_NEWNET| +//! CLONE_NEWPID puts the caller into a fresh mount, UTS and network +//! namespace immediately, and arranges for the next fork(2) child to +//! become PID 1 of a fresh PID namespace (unshare(2) never moves the +//! caller itself into a new PID namespace — only its future children). +//! 2. CGROUP v2 LIMITS — a dedicated cgroup gets memory.max and cpu.max +//! written before the child is ever created, so the limits are already +//! live the instant CMD execs. +//! +//! `fork(2)` (not clone3) here: nix's `fork` gives the same "no code runs +//! between clone and exec except what we ourselves write" shape as the +//! C++ build's `clone3(flags=0)`. Its safety contract is the same as libc +//! fork's — the child may only call async-signal-safe operations until it +//! execs — which is exactly what the child below does: read its own pid, +//! read the hostname, print, exec. + +use std::ffi::CString; +use std::fs; +use std::io::Write; + +use nix::mount::{MsFlags, mount}; +use nix::sched::{CloneFlags, unshare}; +use nix::sys::wait::{WaitStatus, waitpid}; +use nix::unistd::{ForkResult, Uid, execvp, fork, gethostname, getpid, sethostname}; + +/// Names as printed in "killed signal= ()". A hand-rolled table +/// (identical across all three implementations and to pmon's earlier +/// chapters) rather than strsignal(3) or `Signal`'s own text, both of which +/// are library/locale-dependent prose. +const SIGNAL_NAMES: [&str; 32] = [ + "", "HUP", "INT", "QUIT", "ILL", "TRAP", "ABRT", "BUS", "FPE", "KILL", "USR1", "SEGV", "USR2", + "PIPE", "ALRM", "TERM", "STKFLT", "CHLD", "CONT", "STOP", "TSTP", "TTIN", "TTOU", "URG", + "XCPU", "XFSZ", "VTALRM", "PROF", "WINCH", "IO", "PWR", "SYS", +]; + +fn signal_name(sig: i32) -> String { + if sig >= 1 && (sig as usize) < SIGNAL_NAMES.len() { + SIGNAL_NAMES[sig as usize].to_string() + } else { + format!("SIG{sig}") + } +} + +// ---- cgroup helpers -------------------------------------------------- + +/// Whether `tokens` (a whitespace-separated list, as cgroupfs writes it) +/// contains `word` as a whole token — "cpu" must not match inside "cpuset". +fn has_token(tokens: &str, word: &str) -> bool { + tokens.split_whitespace().any(|t| t == word) +} + +/// The root cgroup is exempt from the "no internal process" constraint, so +/// controllers can be enabled there even though it holds every process on +/// the box — this is how a fresh sibling cgroup gets memory/cpu control +/// without first relocating ourselves into a leaf. +fn ensure_root_controllers() -> Result<(), String> { + let have = fs::read_to_string("/sys/fs/cgroup/cgroup.subtree_control") + .map_err(|e| format!("read cgroup.subtree_control: {e}"))?; + let mut add = String::new(); + if !has_token(&have, "memory") { + add.push_str("+memory "); + } + if !has_token(&have, "cpu") { + add.push_str("+cpu "); + } + if add.is_empty() { + return Ok(()); + } + fs::write("/sys/fs/cgroup/cgroup.subtree_control", add) + .map_err(|e| format!("enable controllers: {e}")) +} + +/// Creates (or reuses) `/sys/fs/cgroup/`, applies the limits, and +/// moves the calling process into it. Every subsequent fork inherits +/// cgroup membership automatically, so CMD is under the limits from its +/// first instruction. +fn setup_cgroup(name: &str, mem_max: &str, cpu_max: &str) -> Result { + ensure_root_controllers()?; + let path = format!("/sys/fs/cgroup/{name}"); + if let Err(e) = fs::create_dir(&path) + && e.kind() != std::io::ErrorKind::AlreadyExists + { + return Err(format!("mkdir {path}: {e}")); + } + fs::write(format!("{path}/memory.max"), mem_max) + .map_err(|e| format!("write memory.max: {e}"))?; + fs::write(format!("{path}/cpu.max"), cpu_max).map_err(|e| format!("write cpu.max: {e}"))?; + // Best-effort: no swap headroom, so a breach of memory.max is a real + // OOM kill instead of a silent slowdown via swap. Not every kernel + // exposes swap accounting the same way, so a failure here isn't fatal. + let _ = fs::write(format!("{path}/memory.swap.max"), "0"); + fs::write(format!("{path}/cgroup.procs"), getpid().to_string()) + .map_err(|e| format!("write cgroup.procs: {e}"))?; + Ok(path) +} + +/// Extracts X from "some avg10=X avg60=Y avg300=Z total=T". +fn parse_psi_some_avg10(psi: &str) -> String { + let Some(some_pos) = psi.find("some ") else { + return "?".into(); + }; + let rest = &psi[some_pos..]; + let Some(a) = rest.find("avg10=") else { + return "?".into(); + }; + let rest = &rest[a + "avg10=".len()..]; + match rest.find(' ') { + Some(sp) => rest[..sp].to_string(), + None => rest.to_string(), + } +} + +// ---- containerize ------------------------------------------------------ + +fn cmd_containerize(args: &[String]) -> i32 { + let mut hostname = "pmon-containerized".to_string(); + let mut mem_max = "max".to_string(); + let mut cpu_max = "max 100000".to_string(); + let mut cgroup_name = String::new(); + let mut cmd: Vec = Vec::new(); + + let mut i = 0; + while i < args.len() { + match args[i].as_str() { + "--hostname" => { + let Some(v) = args.get(i + 1) else { + eprintln!("containerize: --hostname needs a value"); + return 2; + }; + hostname = v.clone(); + i += 1; + } + "--mem-max" => { + let Some(v) = args.get(i + 1) else { + eprintln!("containerize: --mem-max needs a value"); + return 2; + }; + mem_max = v.clone(); + i += 1; + } + "--cpu-max" => { + let Some(v) = args.get(i + 1) else { + eprintln!("containerize: --cpu-max needs a value"); + return 2; + }; + cpu_max = v.clone(); + i += 1; + } + "--cgroup" => { + let Some(v) = args.get(i + 1) else { + eprintln!("containerize: --cgroup needs a value"); + return 2; + }; + cgroup_name = v.clone(); + i += 1; + } + "--" => { + cmd = args[i + 1..].to_vec(); + break; + } + other => { + eprintln!("containerize: unexpected argument: {other}"); + return 2; + } + } + i += 1; + } + if cmd.is_empty() { + eprintln!("containerize: missing -- CMD"); + return 2; + } + if !Uid::effective().is_root() { + eprintln!("containerize: must run as root"); + return 1; + } + if cgroup_name.is_empty() { + cgroup_name = format!("pmon-{}", getpid()); + } + + let cgroup_path = match setup_cgroup(&cgroup_name, &mem_max, &cpu_max) { + Ok(p) => p, + Err(e) => { + eprintln!("containerize: {e}"); + return 1; + } + }; + + // Mount, UTS and network namespaces take effect on THIS process right + // now. The PID namespace is deferred: the next fork(2) child becomes + // PID 1 of a fresh one; we (the caller) stay in our original PID + // namespace so we can waitpid(2) the child the ordinary way. + if let Err(e) = unshare( + CloneFlags::CLONE_NEWNS + | CloneFlags::CLONE_NEWUTS + | CloneFlags::CLONE_NEWNET + | CloneFlags::CLONE_NEWPID, + ) { + eprintln!("containerize: unshare: {e}"); + return 1; + } + + // Detach mount propagation recursively so nothing we (or CMD) do in + // this namespace leaks a mount event back to the host's mount table. + if let Err(e) = mount( + Some("none"), + "/", + None::<&str>, + MsFlags::MS_REC | MsFlags::MS_PRIVATE, + None::<&str>, + ) { + eprintln!("containerize: mount MS_PRIVATE: {e}"); + return 1; + } + + // sethostname(2) now lands in the fresh UTS namespace, not the host's. + if let Err(e) = sethostname(&hostname) { + eprintln!("containerize: sethostname: {e}"); + return 1; + } + + // SAFETY: the child performs only async-signal-safe operations + // (getpid, gethostname, writes to already-open stdio fds, execvp) + // before it either execs or _exits — it never returns up through this + // stack frame into arbitrary Rust code, satisfying fork(2)'s contract. + match unsafe { fork() } { + Ok(ForkResult::Child) => { + if getpid().as_raw() == 1 { + println!("pmon: child sees pid 1"); + } + if let Ok(hn) = gethostname() { + println!("pmon: hostname={}", hn.to_string_lossy()); + } + let _ = std::io::stdout().flush(); + + let prog = CString::new(cmd[0].as_str()).unwrap_or_default(); + let cargs: Vec = cmd + .iter() + .map(|s| CString::new(s.as_str()).unwrap_or_default()) + .collect(); + // execvp never returns on success. + let err = execvp(&prog, &cargs).unwrap_err(); + eprintln!("pmon: exec {}: {err}", cmd[0]); + std::process::exit(127); + } + Ok(ForkResult::Parent { child }) => { + let status = waitpid(child, None); + + if let Ok(psi) = fs::read_to_string(format!("{cgroup_path}/memory.pressure")) { + println!( + "pmon: cgroup mem.pressure some={}", + parse_psi_some_avg10(&psi) + ); + } + + let exit_code = match status { + Ok(WaitStatus::Exited(_, code)) => { + println!("pmon: child exited status={code}"); + code + } + Ok(WaitStatus::Signaled(_, sig, _)) => { + let n = sig as i32; + println!("pmon: child killed signal={n} ({})", signal_name(n)); + 128 + n + } + Ok(_) => 0, + Err(e) => { + eprintln!("containerize: waitpid: {e}"); + 1 + } + }; + + let _ = fs::remove_dir(&cgroup_path); // best-effort cleanup + + exit_code + } + Err(e) => { + eprintln!("containerize: fork: {e}"); + 1 + } + } +} + +fn usage() { + eprintln!("usage:"); + eprintln!(" pmon containerize [--hostname NAME] [--mem-max BYTES|max]"); + eprintln!(" [--cpu-max \"QUOTA PERIOD\"|max] [--cgroup NAME]"); + eprintln!(" -- CMD [ARGS...]"); +} + +fn main() { + let args: Vec = std::env::args().collect(); + if args.len() < 2 { + usage(); + std::process::exit(2); + } + let code = match args[1].as_str() { + "containerize" => cmd_containerize(&args[2..]), + _ => { + usage(); + 2 + } + }; + std::process::exit(code); +} diff --git a/examples/32-namespaces-and-cgroups/verify.lua b/examples/32-namespaces-and-cgroups/verify.lua new file mode 100644 index 0000000..ce5aaba --- /dev/null +++ b/examples/32-namespaces-and-cgroups/verify.lua @@ -0,0 +1,125 @@ +-- Verify pmon v6 (namespaces & cgroups) — a VM example. Behavioral assertions: +-- +-- 1. PID NAMESPACE — `containerize -- /usr/bin/echo cmd-ran` must print +-- "pmon: child sees pid 1": the child only sees itself as pid 1 because +-- it is the first (and only) task in a fresh PID namespace. +-- 2. UTS NAMESPACE — the same run must print "pmon: hostname=" with +-- the hostname we asked for, and that name must be DIFFERENT from the +-- guest's real hostname (fetched independently over ssh) — proof the +-- change landed in a private UTS namespace, not the host's. +-- 3. CGROUP MEMORY LIMIT (negative control) — a real memory hog started +-- under `--mem-max` far below what it tries to allocate must be +-- OOM-killed (SIGKILL, exit 137), not merely slowed down. Without the +-- cgroup limit doing real work, the same hog would run to completion; +-- the kill is the proof memory.max is live, not decorative. +-- +-- Runs under the runner's vm mode (TARGET=systems-target, deploy via +-- deploy-to-vm.sh with SUDO=1). deploy-to-vm.sh flattens CMD's argv with a +-- bare `$*` on the remote side, so any CMD token containing spaces/shell +-- metacharacters (a `bash -c '...'` one-liner, say) gets corrupted in +-- transit. We route around that: the pid/hostname check uses a CMD of +-- plain, space-free tokens (`/usr/bin/echo cmd-ran`), and the memory hog is +-- staged as a real script file on the guest first, then invoked as +-- `sh /tmp/pmon-hog.sh` — two more plain tokens. + +local script_dir = arg[0]:match("(.*/)") or "./" +local repo_root = os.getenv("REPO_ROOT") or (script_dir .. "../..") +package.path = package.path .. ";" .. repo_root .. "/scripts/lib/?.lua" +local checks = require("checks") + +local lang = os.getenv("LSP_LANG") +if lang ~= "cpp" and lang ~= "go" and lang ~= "rust" then + checks.skip("LSP_LANG must be cpp|go|rust (got " .. tostring(lang) .. ")") +end + +local target = os.getenv("TARGET") +if not target or target == "" then + checks.skip("no TARGET set — this is a vm example (run with --mode vm)") +end + +local home = os.getenv("HOME") or "/root" +local ssh_opts = + "-o StrictHostKeyChecking=accept-new -o UserKnownHostsFile=" .. home .. "/.ssh/known_hosts" + +-- Resolve the guest IP (same helper deploy-to-vm.sh uses). +local ipf = io.popen(repo_root .. "/scripts/lab/vm-ip.sh '" .. target .. "' 2>/dev/null") +local ip = ipf and ipf:read("*l") or nil +if ipf then ipf:close() end +if not ip or ip == "" then + checks.skip("cannot resolve IP for VM '" .. target .. "' (lab down?)") +end + +-- --------------------------------------------------------------------------- +-- 1 & 2: pid namespace + uts namespace, via a trivial CMD. +-- --------------------------------------------------------------------------- + +local ns_hostname = "pmon-vm-check" +local pos = checks.run(string.format( + "SUDO=1 ./demo.sh %s run containerize --hostname %s -- /usr/bin/echo cmd-ran", + lang, ns_hostname)) +checks.expect_exit(pos, 0, lang .. ": containerize (echo) exits 0") +checks.expect_match(pos.out, "pmon: child sees pid 1", + lang .. ": child observes itself as pid 1 (pid namespace)") +-- ns_hostname contains literal '-', a Lua pattern quantifier magic char, so +-- it must be escaped (%-) before use as a pattern, not just concatenated. +checks.expect_match(pos.out, "pmon: hostname=" .. ns_hostname:gsub("%-", "%%-"), + lang .. ": child observes the requested hostname (uts namespace)") +checks.expect_match(pos.out, "cmd%-ran", + lang .. ": CMD itself ran (echo output present)") +checks.expect_match(pos.out, "pmon: cgroup mem%.pressure some=%d+%.%d+", + lang .. ": cgroup PSI memory.pressure was read") +checks.expect_match(pos.out, "pmon: child exited status=0", + lang .. ": clean exit mirrored") + +-- The uts namespace only proves anything if the name we set is not the +-- guest's own hostname to begin with (fetched independently, not from pmon). +local hn = checks.run(string.format("ssh %s 'fedora@%s' hostname", ssh_opts, ip)) +checks.expect_exit(hn, 0, "fetch guest's real hostname over ssh") +local host_hostname = hn.out:match("^%s*(.-)%s*$") +if host_hostname == ns_hostname then + checks.expect_match("mismatch-forced", "never-matches", + lang .. ": test hostname '" .. ns_hostname .. "' collided with the real guest hostname") +else + checks.expect_match("uts namespace isolated the hostname change", "isolated", + lang .. ": containerized hostname (" .. ns_hostname .. ") differs from guest's real hostname (" .. + host_hostname .. ")") +end + +-- --------------------------------------------------------------------------- +-- 3: cgroup memory.max negative control — a real hog gets OOM-killed. +-- --------------------------------------------------------------------------- + +-- Stage the hog as a file (not a `bash -c '...'` argument — see file header) +-- so deploy-to-vm.sh's flattening never sees its internal spaces/pipe/`;`. +local hog_path = "/tmp/pmon-hog.sh" +local stage = checks.run(string.format( + [[ssh %s 'fedora@%s' "cat > %s && chmod +x %s" <<'HOGEOF' +#!/bin/sh +a=$(head -c 200000000 /dev/zero | tr '\0' x) +sleep 2 +echo should-not-print +HOGEOF +]], ssh_opts, ip, hog_path, hog_path)) +checks.expect_exit(stage, 0, lang .. ": stage memory-hog script on guest") + +-- 64 MiB ceiling, no swap (containerize disables memory.swap.max itself): +-- comfortably above any of the three runtimes' own baseline RSS, but far +-- below the ~190 MiB the hog tries to hold, so the hog — not pmon itself — +-- is what the cgroup OOM killer picks. +local neg = checks.run(string.format( + "SUDO=1 ./demo.sh %s run containerize --mem-max 67108864 --cgroup pmon-hog-%s -- sh %s", + lang, lang, hog_path)) +checks.expect_exit(neg, 137, lang .. ": memory hog under memory.max is OOM-killed (exit 137)") +checks.expect_match(neg.out, "pmon: child sees pid 1", + lang .. ": hog run still entered the pid namespace before being killed") +checks.expect_match(neg.out, "pmon: child killed signal=9 %(KILL%)", + lang .. ": kill is reported as SIGKILL, not a graceful exit") +if neg.out:find("should%-not%-print") then + checks.expect_match("hog completed", "never-matches", + lang .. ": hog ran to completion — memory.max did not constrain it") +else + checks.expect_match("hog did not complete", "did not complete", + lang .. ": hog never reached its post-allocation echo — killed mid-allocation") +end + +checks.finish() diff --git a/examples/33-seccomp-and-landlock/README.md b/examples/33-seccomp-and-landlock/README.md new file mode 100644 index 0000000..545e187 --- /dev/null +++ b/examples/33-seccomp-and-landlock/README.md @@ -0,0 +1,122 @@ +# 33 — seccomp + Landlock (`fwatch` v3) + +`fwatch` gets sandboxed. `watch --sandbox DIR` applies two independent +kernel access-control layers before the epoll/inotify watch loop from +[chapter 09](../09-event-loops-epoll-inotify) ever runs: + +| Layer | What it restricts | Mechanism | +|---|---|---| +| **Landlock** | filesystem **reads**, to `DIR` only | an unprivileged, self-imposed ruleset the kernel enforces against this process and every descendant — it cannot be lifted short of exec-ing something that never asked for it | +| **seccomp** | which **syscalls** may run at all | a syscall allowlist covering only what the watch loop needs; anything else returns `EPERM` | + +Two `probe` subcommands are negative controls that apply ONE layer in +isolation and prove it denies exactly what it claims to: + +``` +fwatch: landlock ABI= enforced +fwatch: seccomp filter installed ( syscalls allowed) +``` + +| Language | Landlock | seccomp | +|---|---|---| +| **C++23** | raw `syscall(2)` (``, no libc wrapper exists) | `libseccomp` | +| **Go 1.26** | raw `syscall(2)` via `golang.org/x/sys/unix`'s `SYS_LANDLOCK_*` numbers | a hand-assembled classic-BPF program installed via the raw `seccomp(2)` syscall | +| **Rust (edition 2024)** | the `landlock` crate | the `seccompiler` crate (Firecracker's BPF compiler) | + +**Go runtime caveat.** Both mechanisms are per-*thread* kernel state by +default. The Go runtime schedules goroutines across OS threads it creates and +destroys on its own schedule (GC workers, sysmon, threads parked in blocking +syscalls); restricting only the calling thread would leave the others free, +and a goroutine later migrated onto one of them would silently run outside +the sandbox. The Go implementation avoids this with `runtime.LockOSThread()` +(pins the sandbox-installing goroutine to a stable OS thread) plus the +`LANDLOCK_RESTRICT_SELF_TSYNC` and `SECCOMP_FILTER_FLAG_TSYNC` flags, which +apply the new state to **every** thread already running in the process, not +just the caller's — confirmed locally: a goroutine deliberately scheduled +after the sandbox is installed still gets EACCES/EPERM. + +The Landlock ABI has no libc/x/sys wrapper in any of the three languages; +every version probes it at runtime with +`landlock_create_ruleset(NULL, 0, LANDLOCK_CREATE_RULESET_VERSION)` rather +than assuming a version — the printed `ABI=` is a live kernel query, not a +constant. + +## CLI + +``` +usage: fwatch + snapshot DIR one line per regular file + diff OLD NEW compare two snapshots + watch DIR [--timeout-ms T] unsandboxed watch (chapter 09 behavior, unchanged) + watch --sandbox DIR [--timeout-ms T] Landlock+seccomp sandboxed watch + probe --sandbox DIR --outside PATH negative control: open PATH (outside DIR) under Landlock + probe --forbidden-syscall negative control: socket(2) under a seccomp allowlist that omits it +``` + +- `snapshot` / `diff` / unsandboxed `watch` are unchanged from chapter 09. +- `watch --sandbox DIR` — applies Landlock (read+watch access restricted to + `DIR`) then the seccomp allowlist, prints both status lines to stderr, then + runs the identical watch loop. Events, debounce, and the + `(timeout)`/`(signal)` exit lines are exactly as in chapter 09 — the + sandbox is invisible to the loop's own logic. +- `probe --sandbox DIR --outside PATH` — restricts Landlock to `DIR`, then + attempts to open `PATH` (expected to be outside `DIR`) for reading. +- `probe --forbidden-syscall` — installs the same seccomp allowlist (which + never includes `socket(2)`), then calls `socket(AF_INET, SOCK_STREAM, 0)`. + +### Exit codes + +| Code | Meaning | +|---|---| +| `0` | success (snapshot/diff/watch, including a sandboxed watch's clean timeout/signal exit) | +| `1` | runtime error, or a probe hit something other than the expected denial | +| `2` | usage error | +| `20` | **probe confirmed denial** — the sandbox blocked what it should (the passing case for a negative control) | +| `21` | **probe was NOT denied** — the sandbox failed to block what it should (a real sandbox bug) | + +## Try it + +```sh +./demo.sh cpp build +mkdir -p /tmp/fw-sandboxed +./cpp/build/release/app watch --sandbox /tmp/fw-sandboxed --timeout-ms 5000 & +sleep 0.3; echo hi > /tmp/fw-sandboxed/x.txt; rm /tmp/fw-sandboxed/x.txt +wait # landlock/seccomp status lines, then the same event stream as ch09 + +# negative controls +./cpp/build/release/app probe --forbidden-syscall; echo "exit=$?" # EPERM, exit 20 +./cpp/build/release/app probe --sandbox /tmp/fw-sandboxed --outside /etc/hostname; echo "exit=$?" # EACCES, exit 20 +``` + +Landlock and seccomp are both **unprivileged** self-sandboxing mechanisms — +no root is needed to run any of the above on a Linux 5.13+ kernel with +Landlock enabled. + +## Verification + +This is a **vm**-mode example (`examples/manifest.yaml`): `verify.lua` +deploys the built binary and `run-sandbox-checks.sh` to `systems-target` and +runs the driver as root (the book's convention for every +namespace/cgroup/seccomp/Landlock demo, even though these two mechanisms +don't themselves require it). The driver exercises, in one ssh round trip: + +1. `probe --forbidden-syscall` — asserts `EPERM` and exit `20`. +2. `probe --sandbox ... --outside ...` — asserts `EACCES` and exit `20`. +3. `watch --sandbox ...` with a file created/modified/deleted inside the + tree while it runs — asserts an `event: ... inside.txt` line, the + `(timeout)` exit line, and exit `0` (the positive control: sandboxing + doesn't break the watcher). + +`SKIP`s cleanly (exit 77) if `TARGET` isn't set or the lab VM is unreachable. + +## Layout + +``` +33-seccomp-and-landlock/ +├── demo.sh # dispatcher: ./demo.sh [cpp|go|rust|all|build] [args...] +├── run-sandbox-checks.sh # staged + run as root by verify.lua on the guest +├── verify.lua # automated check driven by the runner +├── cpp/ # CMake preset build (links libseccomp), demo.sh +├── go/ # go build, demo.sh +└── rust/ # cargo build (landlock + seccompiler crates), demo.sh +``` diff --git a/examples/33-seccomp-and-landlock/cpp/.clang-format b/examples/33-seccomp-and-landlock/cpp/.clang-format new file mode 100644 index 0000000..2c8640f --- /dev/null +++ b/examples/33-seccomp-and-landlock/cpp/.clang-format @@ -0,0 +1,8 @@ +--- +BasedOnStyle: LLVM +Language: Cpp +Standard: c++20 +IndentWidth: 4 +ColumnLimit: 100 +PointerAlignment: Left +AllowShortFunctionsOnASingleLine: Inline diff --git a/examples/33-seccomp-and-landlock/cpp/.clang-tidy b/examples/33-seccomp-and-landlock/cpp/.clang-tidy new file mode 100644 index 0000000..2f275ba --- /dev/null +++ b/examples/33-seccomp-and-landlock/cpp/.clang-tidy @@ -0,0 +1,11 @@ +--- +Checks: > + bugprone-*, + modernize-*, + performance-*, + readability-*, + -modernize-use-trailing-return-type, + -readability-identifier-length +WarningsAsErrors: '' +HeaderFilterRegex: '.*' +FormatStyle: file diff --git a/examples/33-seccomp-and-landlock/cpp/CMakeLists.txt b/examples/33-seccomp-and-landlock/cpp/CMakeLists.txt new file mode 100644 index 0000000..7960328 --- /dev/null +++ b/examples/33-seccomp-and-landlock/cpp/CMakeLists.txt @@ -0,0 +1,12 @@ +cmake_minimum_required(VERSION 3.25) +project(fwatch_seccomp_landlock CXX) + +set(CMAKE_CXX_STANDARD 23) +set(CMAKE_CXX_STANDARD_REQUIRED ON) +set(CMAKE_CXX_EXTENSIONS OFF) + +find_package(PkgConfig REQUIRED) +pkg_check_modules(SECCOMP REQUIRED IMPORTED_TARGET libseccomp) + +add_executable(app src/main.cpp) +target_link_libraries(app PRIVATE PkgConfig::SECCOMP) diff --git a/examples/33-seccomp-and-landlock/cpp/CMakePresets.json b/examples/33-seccomp-and-landlock/cpp/CMakePresets.json new file mode 100644 index 0000000..e31bd2f --- /dev/null +++ b/examples/33-seccomp-and-landlock/cpp/CMakePresets.json @@ -0,0 +1,62 @@ +{ + "version": 6, + "cmakeMinimumRequired": { "major": 3, "minor": 25, "patch": 0 }, + "configurePresets": [ + { + "name": "_base", + "hidden": true, + "generator": "Ninja", + "binaryDir": "${sourceDir}/build/${presetName}", + "cacheVariables": { + "CMAKE_EXPORT_COMPILE_COMMANDS": "ON" + } + }, + { + "name": "release", + "inherits": "_base", + "displayName": "Release (-O2 with debug info)", + "cacheVariables": { "CMAKE_BUILD_TYPE": "RelWithDebInfo" } + }, + { + "name": "debug", + "inherits": "_base", + "displayName": "Debug", + "cacheVariables": { "CMAKE_BUILD_TYPE": "Debug" } + }, + { + "name": "asan", + "inherits": "_base", + "displayName": "ASan + UBSan instrumented build", + "cacheVariables": { + "CMAKE_BUILD_TYPE": "Debug", + "CMAKE_CXX_FLAGS": "-O1 -g -fno-omit-frame-pointer -fsanitize=address,undefined", + "CMAKE_EXE_LINKER_FLAGS": "-fsanitize=address,undefined" + } + }, + { + "name": "release-clang", + "inherits": "release", + "displayName": "Release (clang)", + "cacheVariables": { + "CMAKE_C_COMPILER": "clang", + "CMAKE_CXX_COMPILER": "clang++" + } + }, + { + "name": "debug-clang", + "inherits": "debug", + "displayName": "Debug (clang)", + "cacheVariables": { + "CMAKE_C_COMPILER": "clang", + "CMAKE_CXX_COMPILER": "clang++" + } + } + ], + "buildPresets": [ + { "name": "release", "configurePreset": "release" }, + { "name": "debug", "configurePreset": "debug" }, + { "name": "asan", "configurePreset": "asan" }, + { "name": "release-clang", "configurePreset": "release-clang" }, + { "name": "debug-clang", "configurePreset": "debug-clang" } + ] +} diff --git a/examples/33-seccomp-and-landlock/cpp/demo.sh b/examples/33-seccomp-and-landlock/cpp/demo.sh new file mode 100755 index 0000000..6af6509 --- /dev/null +++ b/examples/33-seccomp-and-landlock/cpp/demo.sh @@ -0,0 +1,40 @@ +#!/usr/bin/env bash +set -euo pipefail + +cd "$(dirname "${BASH_SOURCE[0]}")" +REPO_ROOT="$(cd ../../.. && pwd)" +BIN=build/release/app + +build() { + cmake --preset release + cmake --build --preset release +} + +run() { + if [[ ! -x "$BIN" ]]; then + build + fi + if [[ -n "${TARGET:-}" ]]; then + "$REPO_ROOT/scripts/lab/deploy-to-vm.sh" "$TARGET" "$BIN" -- "$@" + else + "./$BIN" "$@" + fi +} + +case "${1:-}" in + build) + build + ;; + run) + shift + run "$@" + ;; + "") + build + run + ;; + *) + echo "usage: $0 [build|run [args]]" >&2 + exit 2 + ;; +esac diff --git a/examples/33-seccomp-and-landlock/cpp/src/main.cpp b/examples/33-seccomp-and-landlock/cpp/src/main.cpp new file mode 100644 index 0000000..9431a0f --- /dev/null +++ b/examples/33-seccomp-and-landlock/cpp/src/main.cpp @@ -0,0 +1,709 @@ +// fwatch v3 — the ch09 watcher, now sandboxed. +// +// `watch --sandbox DIR` applies two independent kernel access-control layers +// before the epoll loop ever runs: +// +// Landlock — a ruleset that restricts filesystem READS to DIR and nothing +// else. It is an unprivileged, self-imposed sandbox: the kernel +// enforces it against this process and every descendant, and it +// cannot be lifted short of exec-ing something that never asked +// for it in the first place. +// seccomp — a syscall allowlist (libseccomp) covering only the syscalls +// the watch loop actually issues (epoll/inotify/timerfd/ +// signalfd plumbing, read/write/close, the allocator). Anything +// else returns EPERM instead of running. +// +// `probe --sandbox DIR --outside PATH` and `probe --forbidden-syscall` are +// negative controls that apply ONE of the two layers in isolation and prove +// it actually denies what it claims to: a read outside the Landlock tree, and +// a syscall outside the seccomp allowlist. +// +// Landlock ABI has no libc wrapper; every call here goes through the raw +// syscall(2) numbers in / . The ABI is +// probed at runtime (LANDLOCK_CREATE_RULESET_VERSION) rather than assumed, in +// line with "don't hardcode the ABI" — a program built against a newer +// kernel's headers must still degrade sanely on an older one. + +#include +#include +#include +#include +#include +#include +#include +#include +#include +#include +#include +#include +#include +#include +#include +#include +#include +#include +#include + +#include +#include +#include +#include +#include +#include +#include +#include +#include +#include +#include +#include +#include + +namespace { + +namespace fs = std::filesystem; +using std::chrono::milliseconds; +using std::chrono::nanoseconds; +using std::chrono::steady_clock; + +constexpr auto kDebounce = milliseconds{100}; + +[[nodiscard]] std::error_code last_error() { + return {errno, std::system_category()}; +} + +// --------------------------------------------------------------------------- +// RAII file descriptor — the only owner of every fd in this program. +// --------------------------------------------------------------------------- + +class Fd { +public: + Fd() = default; + explicit Fd(int fd) : fd_{fd} {} + Fd(const Fd&) = delete; + Fd& operator=(const Fd&) = delete; + Fd(Fd&& other) noexcept : fd_{std::exchange(other.fd_, -1)} {} + Fd& operator=(Fd&& other) noexcept { + if (this != &other) { + reset(); + fd_ = std::exchange(other.fd_, -1); + } + return *this; + } + ~Fd() { reset(); } + + void reset() noexcept { + if (fd_ >= 0) { + ::close(fd_); + } + fd_ = -1; + } + [[nodiscard]] int get() const noexcept { return fd_; } + +private: + int fd_ = -1; +}; + +[[nodiscard]] std::expected checked(int fd) { + if (fd < 0) { + return std::unexpected(last_error()); + } + return Fd{fd}; +} + +// --------------------------------------------------------------------------- +// Landlock — raw syscalls, no libc wrapper exists. +// --------------------------------------------------------------------------- + +[[nodiscard]] long sys_landlock_create_ruleset(const landlock_ruleset_attr* attr, std::size_t size, + std::uint32_t flags) { + return ::syscall(SYS_landlock_create_ruleset, attr, size, flags); +} + +[[nodiscard]] long sys_landlock_add_rule(int ruleset_fd, landlock_rule_type type, + const void* attr, std::uint32_t flags) { + return ::syscall(SYS_landlock_add_rule, ruleset_fd, type, attr, flags); +} + +[[nodiscard]] long sys_landlock_restrict_self(int ruleset_fd, std::uint32_t flags) { + return ::syscall(SYS_landlock_restrict_self, ruleset_fd, flags); +} + +// Probe the running kernel's supported Landlock ABI. Returns 0 if Landlock +// is not built in / not enabled at boot (never hardcoded). +[[nodiscard]] int landlock_abi() { + const long v = + sys_landlock_create_ruleset(nullptr, 0, LANDLOCK_CREATE_RULESET_VERSION); + return v < 0 ? 0 : static_cast(v); +} + +// Create a ruleset that handles READ_FILE + READ_DIR (available since ABI +// v1), add one path-beneath rule granting exactly those rights under `dir`, +// then restrict_self. On success the calling process — and everything it +// execs from here on — can only read inside `dir`. +[[nodiscard]] std::expected apply_landlock(const std::string& dir) { + constexpr std::uint64_t access = LANDLOCK_ACCESS_FS_READ_FILE | LANDLOCK_ACCESS_FS_READ_DIR; + landlock_ruleset_attr attr{.handled_access_fs = access, .handled_access_net = 0, .scoped = 0}; + + const long rs = sys_landlock_create_ruleset(&attr, sizeof(attr), 0); + if (rs < 0) { + return std::unexpected(last_error()); + } + Fd ruleset{static_cast(rs)}; + + auto dirfd = checked(::open(dir.c_str(), O_PATH | O_DIRECTORY | O_CLOEXEC)); + if (!dirfd) { + return std::unexpected(dirfd.error()); + } + landlock_path_beneath_attr pb{.allowed_access = access, .parent_fd = dirfd->get()}; + if (sys_landlock_add_rule(ruleset.get(), LANDLOCK_RULE_PATH_BENEATH, &pb, 0) < 0) { + return std::unexpected(last_error()); + } + dirfd->reset(); + + // Landlock requires no_new_privs (or CAP_SYS_ADMIN) before restrict_self. + if (::prctl(PR_SET_NO_NEW_PRIVS, 1, 0, 0, 0) < 0) { + return std::unexpected(last_error()); + } + if (sys_landlock_restrict_self(ruleset.get(), 0) < 0) { + return std::unexpected(last_error()); + } + return {}; +} + +// --------------------------------------------------------------------------- +// seccomp — libseccomp allowlist covering exactly what the watch loop needs. +// --------------------------------------------------------------------------- + +// Syscalls the epoll/inotify/timerfd/signalfd watch loop issues, empirically +// confirmed with `strace -f` across a full watch session (create/modify/ +// delete + debounce flush + timeout exit). Anything not on this list is +// denied with EPERM once the filter loads. +constexpr std::array kWatchSyscalls{ + SCMP_SYS(read), SCMP_SYS(write), SCMP_SYS(close), + SCMP_SYS(inotify_init1), SCMP_SYS(inotify_add_watch), SCMP_SYS(timerfd_create), + SCMP_SYS(timerfd_settime), SCMP_SYS(signalfd4), SCMP_SYS(epoll_create1), + SCMP_SYS(epoll_ctl), SCMP_SYS(epoll_wait), SCMP_SYS(rt_sigprocmask), + SCMP_SYS(mmap), SCMP_SYS(munmap), SCMP_SYS(mremap), + SCMP_SYS(brk), SCMP_SYS(rt_sigreturn), SCMP_SYS(exit_group), + SCMP_SYS(exit), SCMP_SYS(fstat), SCMP_SYS(newfstatat), + SCMP_SYS(lseek), SCMP_SYS(clock_gettime), SCMP_SYS(clock_nanosleep), + SCMP_SYS(nanosleep), SCMP_SYS(getrandom), +}; + +// Installs an allow-listed set of syscalls (`allowed`); everything else +// returns errno `deny_errno`. Returns the number of syscalls admitted. +[[nodiscard]] std::expected +install_seccomp(const int* allowed, std::size_t count, int deny_errno) { + scmp_filter_ctx ctx = ::seccomp_init(SCMP_ACT_ERRNO(deny_errno)); + if (ctx == nullptr) { + return std::unexpected(std::make_error_code(std::errc::not_enough_memory)); + } + for (std::size_t i = 0; i < count; ++i) { + if (::seccomp_rule_add(ctx, SCMP_ACT_ALLOW, allowed[i], 0) < 0) { + ::seccomp_release(ctx); + return std::unexpected(last_error()); + } + } + const int loaded = ::seccomp_load(ctx); + ::seccomp_release(ctx); + if (loaded < 0) { + return std::unexpected(std::error_code{-loaded, std::system_category()}); + } + return count; +} + +// --------------------------------------------------------------------------- +// v0: snapshot + diff +// --------------------------------------------------------------------------- + +struct FileState { + std::string name; + std::uint64_t size; + std::int64_t mtime_ns; +}; + +[[nodiscard]] std::expected, std::error_code> +scan_dir(const std::string& dir) { + std::error_code ec; + std::vector out; + for (const auto& entry : fs::directory_iterator{dir, ec}) { + struct stat st{}; + if (::fstatat(AT_FDCWD, entry.path().c_str(), &st, AT_SYMLINK_NOFOLLOW) != 0) { + continue; // raced with a concurrent delete + } + if (!S_ISREG(st.st_mode)) { + continue; + } + out.push_back(FileState{ + .name = entry.path().filename().string(), + .size = static_cast(st.st_size), + .mtime_ns = static_cast(st.st_mtim.tv_sec) * 1'000'000'000 + + st.st_mtim.tv_nsec, + }); + } + if (ec) { + return std::unexpected(ec); + } + std::ranges::sort(out, {}, &FileState::name); + return out; +} + +int cmd_snapshot(const std::string& dir) { + const auto files = scan_dir(dir); + if (!files) { + std::println(stderr, "fwatch: snapshot: {}: {}", dir, files.error().message()); + return 1; + } + for (const auto& f : *files) { + std::println("{}\t{}\t{}", f.name, f.size, f.mtime_ns); + } + return 0; +} + +[[nodiscard]] std::expected, std::error_code> +load_snapshot(const std::string& path) { + std::ifstream in{path}; + if (!in) { + return std::unexpected(std::make_error_code(std::errc::no_such_file_or_directory)); + } + std::map out; + std::string line; + while (std::getline(in, line)) { + const auto tab2 = line.rfind('\t'); + if (tab2 == std::string::npos || tab2 == 0) { + continue; + } + const auto tab1 = line.rfind('\t', tab2 - 1); + if (tab1 == std::string::npos) { + continue; + } + out.insert_or_assign(line.substr(0, tab1), line.substr(tab1 + 1)); + } + return out; +} + +int cmd_diff(const std::string& old_path, const std::string& new_path) { + const auto old_snap = load_snapshot(old_path); + if (!old_snap) { + std::println(stderr, "fwatch: diff: {}: {}", old_path, old_snap.error().message()); + return 1; + } + const auto new_snap = load_snapshot(new_path); + if (!new_snap) { + std::println(stderr, "fwatch: diff: {}: {}", new_path, new_snap.error().message()); + return 1; + } + std::set names; + for (const auto& [name, _] : *old_snap) { + names.insert(name); + } + for (const auto& [name, _] : *new_snap) { + names.insert(name); + } + for (const auto& name : names) { + const auto o = old_snap->find(name); + const auto n = new_snap->find(name); + if (o == old_snap->end()) { + std::println("created {}", name); + } else if (n == new_snap->end()) { + std::println("deleted {}", name); + } else if (o->second != n->second) { + std::println("modified {}", name); + } + } + return 0; +} + +// --------------------------------------------------------------------------- +// v1: watch — one epoll loop, three fds, zero threads beyond main. +// --------------------------------------------------------------------------- + +enum class Kind : std::uint8_t { created, modified, deleted }; + +[[nodiscard]] std::string_view kind_name(Kind k) { + switch (k) { + case Kind::created: return "created"; + case Kind::modified: return "modified"; + case Kind::deleted: return "deleted"; + } + return "?"; +} + +[[nodiscard]] std::optional classify(std::uint32_t mask) { + if ((mask & (IN_CREATE | IN_MOVED_TO)) != 0) { + return Kind::created; + } + if ((mask & (IN_DELETE | IN_MOVED_FROM)) != 0) { + return Kind::deleted; + } + if ((mask & (IN_MODIFY | IN_ATTRIB)) != 0) { + return Kind::modified; + } + return std::nullopt; +} + +[[nodiscard]] Kind merge(std::optional old_kind, Kind new_kind) { + if (!old_kind) { + return new_kind; + } + if (new_kind == Kind::deleted) { + return Kind::deleted; + } + if (*old_kind == Kind::created) { + return Kind::created; + } + return Kind::modified; +} + +struct Pending { + Kind kind; + steady_clock::time_point due; +}; + +[[nodiscard]] std::expected make_signalfd() { + sigset_t mask; + sigemptyset(&mask); + sigaddset(&mask, SIGINT); + sigaddset(&mask, SIGTERM); + if (::sigprocmask(SIG_BLOCK, &mask, nullptr) != 0) { + return std::unexpected(last_error()); + } + return checked(::signalfd(-1, &mask, SFD_NONBLOCK | SFD_CLOEXEC)); +} + +[[nodiscard]] std::expected arm_timer(const Fd& tfd, nanoseconds rel) { + if (rel <= nanoseconds::zero()) { + rel = nanoseconds{1}; + } + itimerspec spec{}; + spec.it_value.tv_sec = rel.count() / 1'000'000'000; + spec.it_value.tv_nsec = rel.count() % 1'000'000'000; + if (::timerfd_settime(tfd.get(), 0, &spec, nullptr) != 0) { + return std::unexpected(last_error()); + } + return {}; +} + +void flush(std::map& pending, steady_clock::time_point now, bool all) { + for (auto it = pending.begin(); it != pending.end();) { + if (all || it->second.due <= now) { + std::println("event: {} {}", kind_name(it->second.kind), it->first); + it = pending.erase(it); + } else { + ++it; + } + } + std::fflush(stdout); +} + +// The watch loop itself. Everything it needs (Landlock + seccomp, if +// requested) has already been set up by the caller before this runs; the +// loop's own logic never needs to know whether it is sandboxed. +int run_watch_loop(const std::string& dir, int timeout_ms) { + auto inotify = checked(::inotify_init1(IN_NONBLOCK | IN_CLOEXEC)); + if (!inotify) { + std::println(stderr, "fwatch: watch: inotify_init1: {}", inotify.error().message()); + return 1; + } + constexpr std::uint32_t watch_mask = + IN_CREATE | IN_MODIFY | IN_ATTRIB | IN_DELETE | IN_MOVED_FROM | IN_MOVED_TO; + if (::inotify_add_watch(inotify->get(), dir.c_str(), watch_mask) < 0) { + std::println(stderr, "fwatch: watch: {}: {}", dir, last_error().message()); + return 1; + } + auto timer = checked(::timerfd_create(CLOCK_MONOTONIC, TFD_NONBLOCK | TFD_CLOEXEC)); + if (!timer) { + std::println(stderr, "fwatch: watch: timerfd_create: {}", timer.error().message()); + return 1; + } + auto sigfd = make_signalfd(); + if (!sigfd) { + std::println(stderr, "fwatch: watch: signalfd: {}", sigfd.error().message()); + return 1; + } + auto epoll = checked(::epoll_create1(EPOLL_CLOEXEC)); + if (!epoll) { + std::println(stderr, "fwatch: watch: epoll_create1: {}", epoll.error().message()); + return 1; + } + + enum : std::uint32_t { tok_inotify, tok_timer, tok_signal }; + const auto add = [&](const Fd& fd, + std::uint32_t token) -> std::expected { + epoll_event ev{.events = EPOLLIN, .data = {.u32 = token}}; + if (::epoll_ctl(epoll->get(), EPOLL_CTL_ADD, fd.get(), &ev) != 0) { + return std::unexpected(last_error()); + } + return {}; + }; + auto registered = add(*inotify, tok_inotify) + .and_then([&] { return add(*timer, tok_timer); }) + .and_then([&] { return add(*sigfd, tok_signal); }); + if (!registered) { + std::println(stderr, "fwatch: watch: epoll_ctl: {}", registered.error().message()); + return 1; + } + + std::println(stderr, "fwatch: watching {}", dir); + const auto deadline = steady_clock::now() + milliseconds{timeout_ms}; + std::map pending; + + for (;;) { + auto next = deadline; + for (const auto& [_, p] : pending) { + next = std::min(next, p.due); + } + if (auto armed = arm_timer(*timer, next - steady_clock::now()); !armed) { + std::println(stderr, "fwatch: watch: timerfd_settime: {}", armed.error().message()); + return 1; + } + + std::array events{}; + const int n = ::epoll_wait(epoll->get(), events.data(), events.size(), -1); + if (n < 0) { + if (errno == EINTR) { + continue; + } + std::println(stderr, "fwatch: watch: epoll_wait: {}", last_error().message()); + return 1; + } + + for (int i = 0; i < n; ++i) { + switch (events[i].data.u32) { + case tok_inotify: { + alignas(inotify_event) char buf[4096]; + for (;;) { + const ssize_t got = ::read(inotify->get(), buf, sizeof buf); + if (got <= 0) { + break; + } + const auto now = steady_clock::now(); + for (const char* p = buf; p < buf + got;) { + const auto* ev = reinterpret_cast(p); + p += sizeof(inotify_event) + ev->len; + if (ev->len == 0) { + continue; + } + const auto kind = classify(ev->mask); + if (!kind) { + continue; + } + const std::string name{ev->name}; + const auto it = pending.find(name); + const auto old_kind = it == pending.end() + ? std::nullopt + : std::optional{it->second.kind}; + pending.insert_or_assign( + name, Pending{merge(old_kind, *kind), now + kDebounce}); + } + } + break; + } + case tok_timer: { + std::uint64_t expirations = 0; + std::ignore = ::read(timer->get(), &expirations, sizeof expirations); + const auto now = steady_clock::now(); + if (now >= deadline) { + flush(pending, now, /*all=*/true); + std::println("fwatch: exiting (timeout)"); + return 0; + } + flush(pending, now, /*all=*/false); + break; + } + case tok_signal: { + signalfd_siginfo info{}; + std::ignore = ::read(sigfd->get(), &info, sizeof info); + flush(pending, steady_clock::now(), /*all=*/true); + std::println("fwatch: exiting (signal)"); + return 0; + } + default: + break; + } + } + } +} + +int cmd_watch(const std::string& dir, int timeout_ms, bool sandbox) { + if (sandbox) { + const int abi = landlock_abi(); + if (abi <= 0) { + std::println(stderr, "fwatch: watch: Landlock not supported by this kernel"); + return 1; + } + if (auto ll = apply_landlock(dir); !ll) { + std::println(stderr, "fwatch: watch: landlock: {}", ll.error().message()); + return 1; + } + std::println(stderr, "fwatch: landlock ABI={} enforced", abi); + + const auto installed = + install_seccomp(kWatchSyscalls.data(), kWatchSyscalls.size(), EPERM); + if (!installed) { + std::println(stderr, "fwatch: watch: seccomp: {}", installed.error().message()); + return 1; + } + std::println(stderr, "fwatch: seccomp filter installed ({} syscalls allowed)", + *installed); + } + return run_watch_loop(dir, timeout_ms); +} + +// --------------------------------------------------------------------------- +// probes — negative controls, one Landlock, one seccomp, exercised alone. +// --------------------------------------------------------------------------- + +// Exit codes for the probes are distinct from the ordinary 0/1/2 contract so +// a verifier can tell "confirmed denied" (the PASSING case for a negative +// control) apart from "ran into some other error" or "was NOT denied" (a +// sandbox bug). +constexpr int kProbeDenied = 20; // Landlock/seccomp did deny it, as expected +constexpr int kProbeNotDenied = 21; // sandbox failed to deny it — a real bug + +int cmd_probe_outside(const std::string& sandbox_dir, const std::string& outside_path) { + const int abi = landlock_abi(); + if (abi <= 0) { + std::println(stderr, "fwatch: probe: Landlock not supported by this kernel"); + return 1; + } + if (auto ll = apply_landlock(sandbox_dir); !ll) { + std::println(stderr, "fwatch: probe: landlock: {}", ll.error().message()); + return 1; + } + std::println(stderr, "fwatch: landlock ABI={} enforced", abi); + + errno = 0; + const int fd = ::open(outside_path.c_str(), O_RDONLY); + if (fd >= 0) { + ::close(fd); + std::println("fwatch: probe outside {}: opened (landlock did NOT block this)", + outside_path); + return kProbeNotDenied; + } + const auto ec = last_error(); + if (ec == std::errc::permission_denied) { + std::println("fwatch: probe outside {}: EACCES ({})", outside_path, ec.message()); + return kProbeDenied; + } + std::println("fwatch: probe outside {}: unexpected error: {}", outside_path, ec.message()); + return 1; +} + +int cmd_probe_forbidden_syscall() { + // Deliberately omit socket(2) from the allowlist. + const auto installed = + install_seccomp(kWatchSyscalls.data(), kWatchSyscalls.size(), EPERM); + if (!installed) { + std::println(stderr, "fwatch: probe: seccomp: {}", installed.error().message()); + return 1; + } + std::println(stderr, "fwatch: seccomp filter installed ({} syscalls allowed)", *installed); + + errno = 0; + const int s = ::socket(AF_INET, SOCK_STREAM, 0); + if (s >= 0) { + ::close(s); + std::println("fwatch: probe forbidden-syscall: socket() unexpectedly succeeded"); + return kProbeNotDenied; + } + const auto ec = last_error(); + if (ec == std::errc::operation_not_permitted) { + std::println("fwatch: probe forbidden-syscall: EPERM ({})", ec.message()); + return kProbeDenied; + } + std::println("fwatch: probe forbidden-syscall: unexpected error: {}", ec.message()); + return 1; +} + +// --------------------------------------------------------------------------- +// CLI +// --------------------------------------------------------------------------- + +int usage() { + std::println(stderr, "usage: fwatch "); + std::println(stderr, + " snapshot DIR one line per regular file"); + std::println(stderr, + " diff OLD NEW compare two snapshots"); + std::println(stderr, + " watch DIR [--timeout-ms T] unsandboxed watch"); + std::println(stderr, + " watch --sandbox DIR [--timeout-ms T] Landlock+seccomp sandboxed watch"); + std::println(stderr, + " probe --sandbox DIR --outside PATH negative control: open PATH " + "(outside DIR) under Landlock"); + std::println(stderr, + " probe --forbidden-syscall negative control: socket(2) " + "under a seccomp allowlist that omits it"); + return 2; +} + +} // namespace + +int main(int argc, char** argv) { + const std::vector args{argv + 1, argv + argc}; + if (args.empty()) { + return usage(); + } + const auto cmd = args[0]; + + if (cmd == "snapshot" && args.size() == 2) { + return cmd_snapshot(std::string{args[1]}); + } + if (cmd == "diff" && args.size() == 3) { + return cmd_diff(std::string{args[1]}, std::string{args[2]}); + } + if (cmd == "watch") { + bool sandbox = false; + std::optional dir; + int timeout_ms = 2000; + for (std::size_t i = 1; i < args.size(); ++i) { + if (args[i] == "--sandbox" && i + 1 < args.size()) { + sandbox = true; + dir = std::string{args[++i]}; + } else if (args[i] == "--timeout-ms" && i + 1 < args.size()) { + try { + timeout_ms = std::stoi(std::string{args[++i]}); + } catch (const std::exception&) { + return usage(); + } + if (timeout_ms <= 0) { + return usage(); + } + } else if (!dir) { + dir = std::string{args[i]}; + } else { + return usage(); + } + } + if (!dir) { + return usage(); + } + return cmd_watch(*dir, timeout_ms, sandbox); + } + if (cmd == "probe") { + std::optional sandbox_dir; + std::optional outside; + bool forbidden_syscall = false; + for (std::size_t i = 1; i < args.size(); ++i) { + if (args[i] == "--sandbox" && i + 1 < args.size()) { + sandbox_dir = std::string{args[++i]}; + } else if (args[i] == "--outside" && i + 1 < args.size()) { + outside = std::string{args[++i]}; + } else if (args[i] == "--forbidden-syscall") { + forbidden_syscall = true; + } else { + return usage(); + } + } + if (forbidden_syscall && !sandbox_dir && !outside) { + return cmd_probe_forbidden_syscall(); + } + if (sandbox_dir && outside && !forbidden_syscall) { + return cmd_probe_outside(*sandbox_dir, *outside); + } + return usage(); + } + return usage(); +} diff --git a/examples/33-seccomp-and-landlock/demo.sh b/examples/33-seccomp-and-landlock/demo.sh new file mode 100755 index 0000000..afbbe4c --- /dev/null +++ b/examples/33-seccomp-and-landlock/demo.sh @@ -0,0 +1,23 @@ +#!/usr/bin/env bash +set -euo pipefail +cd "$(dirname "$0")" + +langs=(cpp go rust) + +case "${1:-all}" in + cpp|go|rust) + lang="$1" + shift + exec "./$lang/demo.sh" "$@" + ;; + all) + for lang in "${langs[@]}"; do "./$lang/demo.sh"; done + ;; + build) + for lang in "${langs[@]}"; do "./$lang/demo.sh" build; done + ;; + *) + echo "usage: $0 [cpp|go|rust|all|build] [args...]" >&2 + exit 2 + ;; +esac diff --git a/examples/33-seccomp-and-landlock/go/demo.sh b/examples/33-seccomp-and-landlock/go/demo.sh new file mode 100755 index 0000000..28606df --- /dev/null +++ b/examples/33-seccomp-and-landlock/go/demo.sh @@ -0,0 +1,23 @@ +#!/usr/bin/env bash +set -euo pipefail +cd "$(dirname "$0")" +REPO_ROOT="$(cd ../../.. && pwd)" +BIN=bin/app + +build() { + go build -o "$BIN" . +} + +run() { + if [[ -n "${TARGET:-}" ]]; then + exec "$REPO_ROOT/scripts/lab/deploy-to-vm.sh" "$TARGET" "$BIN" -- "$@" + fi + exec "./$BIN" "$@" +} + +case "${1:-}" in + build) build ;; + run) shift; run "$@" ;; + "") build; run ;; + *) echo "usage: $0 [build|run [args...]]" >&2; exit 2 ;; +esac diff --git a/examples/33-seccomp-and-landlock/go/go.mod b/examples/33-seccomp-and-landlock/go/go.mod new file mode 100644 index 0000000..51e0b0e --- /dev/null +++ b/examples/33-seccomp-and-landlock/go/go.mod @@ -0,0 +1,7 @@ +module github.com/patterncatalyst/linux-systems-programming/examples/33-seccomp-and-landlock/go + +go 1.26 + +toolchain go1.26.5 + +require golang.org/x/sys v0.47.0 diff --git a/examples/33-seccomp-and-landlock/go/go.sum b/examples/33-seccomp-and-landlock/go/go.sum new file mode 100644 index 0000000..37ee2d4 --- /dev/null +++ b/examples/33-seccomp-and-landlock/go/go.sum @@ -0,0 +1,2 @@ +golang.org/x/sys v0.47.0 h1:o7XGOvZQCADBQQ4Y7VNq2dRWQR7JmOUW8Kxx4ZsNgWs= +golang.org/x/sys v0.47.0/go.mod h1:4GL1E5IUh+htKOUEOaiffhrAeqysfVGipDYzABqnCmw= diff --git a/examples/33-seccomp-and-landlock/go/main.go b/examples/33-seccomp-and-landlock/go/main.go new file mode 100644 index 0000000..597e932 --- /dev/null +++ b/examples/33-seccomp-and-landlock/go/main.go @@ -0,0 +1,662 @@ +// fwatch v3 — the ch09 watcher, now sandboxed. +// +// `watch --sandbox DIR` applies two independent kernel access-control layers +// before the watch loop starts: +// +// Landlock — a ruleset restricting filesystem READS to DIR. Unprivileged, +// self-imposed, enforced by the kernel against this process and +// every descendant. +// seccomp — a syscall allowlist (a hand-assembled classic-BPF program, +// installed via the raw seccomp(2) syscall) covering only what +// the watch loop needs. Everything else returns EPERM. +// +// GO RUNTIME CAVEAT: both mechanisms are, by default, per-THREAD kernel +// state. The Go runtime schedules goroutines across OS threads it creates +// and destroys on its own schedule (GC workers, sysmon, threads parked in +// blocking syscalls) — applying either sandbox to only the calling thread +// would leave those other threads unrestricted, and a goroutine the runtime +// later migrates onto one of them would silently run outside the sandbox. +// This program avoids that two ways: +// +// - runtime.LockOSThread() pins main's goroutine to the OS thread that +// installs the sandbox, so the thread issuing the syscalls below never +// changes out from under it. +// - Both landlock_restrict_self and the seccomp(2) syscall are called with +// their respective TSYNC flag, which applies the new state to every +// thread in the process atomically — not just the calling one — so +// threads that already exist (and the ones any of them later clone) are +// covered too. +// +// Landlock has no x/sys/unix wrapper; every call goes through the raw +// syscall numbers x/sys/unix does export (SYS_LANDLOCK_*). The ABI is probed +// at runtime (LANDLOCK_CREATE_RULESET_VERSION), never assumed. +package main + +import ( + "errors" + "fmt" + "io/fs" + "os" + "os/signal" + "path/filepath" + "runtime" + "sort" + "strconv" + "strings" + "syscall" + "time" + "unsafe" + + "golang.org/x/sys/unix" +) + +const debounce = 100 * time.Millisecond + +// --------------------------------------------------------------------------- +// Landlock — raw syscalls (x/sys/unix has the numbers, not wrappers). +// --------------------------------------------------------------------------- + +const ( + landlockAccessFSReadFile = 1 << 2 + landlockAccessFSReadDir = 1 << 3 + + landlockCreateRulesetVersion = 1 << 0 + landlockRulePathBeneath = 1 + landlockRestrictSelfTSYNC = 1 << 3 +) + +// Mirrors struct landlock_ruleset_attr from . +type landlockRulesetAttr struct { + HandledAccessFS uint64 + HandledAccessNet uint64 + Scoped uint64 +} + +// Mirrors struct landlock_path_beneath_attr from . +type landlockPathBeneathAttr struct { + AllowedAccess uint64 + ParentFd int32 + _ [4]byte // struct padding to keep the layout syscall-compatible +} + +// landlockABI probes the running kernel's supported Landlock ABI version. +// Returns 0 if Landlock isn't built in / enabled at boot. +func landlockABI() int { + v, _, errno := unix.Syscall(unix.SYS_LANDLOCK_CREATE_RULESET, 0, 0, + uintptr(landlockCreateRulesetVersion)) + if errno != 0 { + return 0 + } + return int(v) +} + +// applyLandlock restricts filesystem reads to dir (READ_FILE + READ_DIR, +// both available since ABI v1) and enforces it with restrict_self+TSYNC so +// every OS thread the Go runtime is already running is covered too. +func applyLandlock(dir string) error { + attr := landlockRulesetAttr{ + HandledAccessFS: landlockAccessFSReadFile | landlockAccessFSReadDir, + } + rs, _, errno := unix.Syscall(unix.SYS_LANDLOCK_CREATE_RULESET, + uintptr(unsafe.Pointer(&attr)), unsafe.Sizeof(attr), 0) + if errno != 0 { + return fmt.Errorf("landlock_create_ruleset: %w", errno) + } + rulesetFd := int(rs) + defer unix.Close(rulesetFd) + + dirFd, err := unix.Open(dir, unix.O_PATH|unix.O_DIRECTORY|unix.O_CLOEXEC, 0) + if err != nil { + return fmt.Errorf("open %s: %w", dir, err) + } + pb := landlockPathBeneathAttr{ + AllowedAccess: landlockAccessFSReadFile | landlockAccessFSReadDir, + ParentFd: int32(dirFd), + } + _, _, errno = unix.Syscall6(unix.SYS_LANDLOCK_ADD_RULE, uintptr(rulesetFd), + uintptr(landlockRulePathBeneath), uintptr(unsafe.Pointer(&pb)), 0, 0, 0) + unix.Close(dirFd) + if errno != 0 { + return fmt.Errorf("landlock_add_rule: %w", errno) + } + + if err := unix.Prctl(unix.PR_SET_NO_NEW_PRIVS, 1, 0, 0, 0); err != nil { + return fmt.Errorf("prctl(PR_SET_NO_NEW_PRIVS): %w", err) + } + + // TSYNC first (syncs the domain to every thread already running); + // fall back to per-thread-only on a kernel too old to support it. + _, _, errno = unix.Syscall(unix.SYS_LANDLOCK_RESTRICT_SELF, uintptr(rulesetFd), + uintptr(landlockRestrictSelfTSYNC), 0) + if errno != 0 { + _, _, errno = unix.Syscall(unix.SYS_LANDLOCK_RESTRICT_SELF, uintptr(rulesetFd), 0, 0) + if errno != 0 { + return fmt.Errorf("landlock_restrict_self: %w", errno) + } + } + return nil +} + +// --------------------------------------------------------------------------- +// seccomp — a hand-assembled classic-BPF allowlist, installed via seccomp(2). +// --------------------------------------------------------------------------- + +const ( + bpfLD = 0x00 + bpfW = 0x00 + bpfABS = 0x20 + bpfJMP = 0x05 + bpfJEQ = 0x10 + bpfK = 0x00 + bpfRET = 0x06 + + // Offsets into struct seccomp_data (): { int nr; + // __u32 arch; __u64 instruction_pointer; __u64 args[6]; }. + seccompDataNrOff = 0 + seccompDataArchOff = 4 +) + +type sockFilter struct { + Code uint16 + Jt uint8 + Jf uint8 + K uint32 +} + +// Mirrors struct sock_fprog from ; the trailing padding +// keeps Filter's pointer naturally aligned on amd64. +type sockFprog struct { + Len uint16 + _ [6]byte + Filter *sockFilter +} + +func bpfStmt(code uint16, k uint32) sockFilter { return sockFilter{Code: code, K: k} } +func bpfJump(code uint16, k uint32, jt, jf uint8) sockFilter { + return sockFilter{Code: code, Jt: jt, Jf: jf, K: k} +} + +// buildAllowlist assembles: check arch == x86_64 (else kill), then one +// equality test per allowed syscall number that jumps to RET_ALLOW on match; +// falling off the end returns RET_ERRNO(denyErrno). +func buildAllowlist(allowed []uint32, denyErrno uint32) []sockFilter { + prog := []sockFilter{ + bpfStmt(bpfLD|bpfW|bpfABS, seccompDataArchOff), + bpfJump(bpfJMP|bpfJEQ|bpfK, unix.AUDIT_ARCH_X86_64, 1, 0), + bpfStmt(bpfRET, unix.SECCOMP_RET_KILL_PROCESS), + bpfStmt(bpfLD|bpfW|bpfABS, seccompDataNrOff), + } + for i, nr := range allowed { + remaining := len(allowed) - i - 1 + prog = append(prog, bpfJump(bpfJMP|bpfJEQ|bpfK, nr, uint8(remaining+1), 0)) + } + prog = append(prog, bpfStmt(bpfRET, unix.SECCOMP_RET_ERRNO|(denyErrno&unix.SECCOMP_RET_DATA))) + prog = append(prog, bpfStmt(bpfRET, unix.SECCOMP_RET_ALLOW)) + return prog +} + +// installSeccomp loads an allowlisting filter with SECCOMP_FILTER_FLAG_TSYNC +// so every OS thread already running in this process is covered, not just +// the calling one. Returns the number of syscalls admitted. +func installSeccomp(allowed []uint32, denyErrno uint32) (int, error) { + // seccomp(2) requires no_new_privs (or CAP_SYS_ADMIN), same as Landlock; + // harmless to set again if applyLandlock already did. + if err := unix.Prctl(unix.PR_SET_NO_NEW_PRIVS, 1, 0, 0, 0); err != nil { + return 0, fmt.Errorf("prctl(PR_SET_NO_NEW_PRIVS): %w", err) + } + prog := buildAllowlist(allowed, denyErrno) + fprog := sockFprog{Len: uint16(len(prog)), Filter: &prog[0]} + _, _, errno := unix.Syscall(unix.SYS_SECCOMP, uintptr(unix.SECCOMP_SET_MODE_FILTER), + uintptr(unix.SECCOMP_FILTER_FLAG_TSYNC), uintptr(unsafe.Pointer(&fprog))) + if errno != 0 { + return 0, errno + } + return len(allowed), nil +} + +// Syscalls the watch loop issues, empirically confirmed with `strace -f` +// across a full watch session (goroutine reading the inotify fd through the +// netpoller, signal.Notify, time.After/time.Timer) — plus the small, +// well-known set the Go runtime itself needs for scheduling, GC, and stack +// management on any goroutine (futex, mmap/munmap/mprotect, clone, sigaltstack, +// rt_sigaction). Anything else returns EPERM once the filter loads. +var watchSyscalls = []uint32{ + uint32(unix.SYS_READ), uint32(unix.SYS_WRITE), uint32(unix.SYS_CLOSE), + uint32(unix.SYS_EPOLL_CREATE1), uint32(unix.SYS_EPOLL_CTL), uint32(unix.SYS_EPOLL_PWAIT), + uint32(unix.SYS_EPOLL_WAIT), uint32(unix.SYS_INOTIFY_INIT1), uint32(unix.SYS_INOTIFY_ADD_WATCH), + uint32(unix.SYS_EVENTFD2), uint32(unix.SYS_FCNTL), + uint32(unix.SYS_RT_SIGACTION), uint32(unix.SYS_RT_SIGPROCMASK), uint32(unix.SYS_RT_SIGRETURN), + uint32(unix.SYS_SIGALTSTACK), uint32(unix.SYS_TGKILL), + uint32(unix.SYS_MMAP), uint32(unix.SYS_MUNMAP), uint32(unix.SYS_MPROTECT), uint32(unix.SYS_MADVISE), + uint32(unix.SYS_BRK), uint32(unix.SYS_FUTEX), uint32(unix.SYS_CLONE), uint32(unix.SYS_SCHED_YIELD), + uint32(unix.SYS_SCHED_GETAFFINITY), uint32(unix.SYS_GETPID), uint32(unix.SYS_GETTID), + uint32(unix.SYS_CLOCK_GETTIME), uint32(unix.SYS_CLOCK_NANOSLEEP), uint32(unix.SYS_NANOSLEEP), + uint32(unix.SYS_GETRANDOM), uint32(unix.SYS_OPENAT), uint32(unix.SYS_FSTAT), uint32(unix.SYS_NEWFSTATAT), + uint32(unix.SYS_PSELECT6), uint32(unix.SYS_SET_ROBUST_LIST), uint32(unix.SYS_GETRUSAGE), + uint32(unix.SYS_EXIT), uint32(unix.SYS_EXIT_GROUP), +} + +// --------------------------------------------------------------------------- +// v0: snapshot + diff +// --------------------------------------------------------------------------- + +func cmdSnapshot(dir string) error { + entries, err := os.ReadDir(dir) + if err != nil { + if errors.Is(err, fs.ErrNotExist) { + return fmt.Errorf("%s: no such directory", dir) + } + return fmt.Errorf("%s: %w", dir, err) + } + for _, e := range entries { + info, err := os.Lstat(filepath.Join(dir, e.Name())) + if err != nil { + continue + } + if !info.Mode().IsRegular() { + continue + } + st := info.Sys().(*syscall.Stat_t) + mtimeNs := int64(st.Mtim.Sec)*1_000_000_000 + int64(st.Mtim.Nsec) + fmt.Printf("%s\t%d\t%d\n", e.Name(), info.Size(), mtimeNs) + } + return nil +} + +func loadSnapshot(path string) (map[string]string, error) { + data, err := os.ReadFile(path) + if err != nil { + return nil, fmt.Errorf("%s: %w", path, err) + } + out := make(map[string]string) + for _, line := range strings.Split(string(data), "\n") { + tab2 := strings.LastIndexByte(line, '\t') + if tab2 <= 0 { + continue + } + tab1 := strings.LastIndexByte(line[:tab2], '\t') + if tab1 < 0 { + continue + } + out[line[:tab1]] = line[tab1+1:] + } + return out, nil +} + +func cmdDiff(oldPath, newPath string) error { + oldSnap, err := loadSnapshot(oldPath) + if err != nil { + return err + } + newSnap, err := loadSnapshot(newPath) + if err != nil { + return err + } + seen := make(map[string]bool, len(oldSnap)+len(newSnap)) + names := make([]string, 0, len(oldSnap)+len(newSnap)) + for name := range oldSnap { + seen[name] = true + names = append(names, name) + } + for name := range newSnap { + if !seen[name] { + names = append(names, name) + } + } + sort.Strings(names) + for _, name := range names { + o, inOld := oldSnap[name] + n, inNew := newSnap[name] + switch { + case !inOld: + fmt.Printf("created %s\n", name) + case !inNew: + fmt.Printf("deleted %s\n", name) + case o != n: + fmt.Printf("modified %s\n", name) + } + } + return nil +} + +// --------------------------------------------------------------------------- +// v1: watch — goroutine + channels; the runtime owns the epoll loop. +// --------------------------------------------------------------------------- + +type fsEvent struct { + kind string + name string +} + +func classify(mask uint32) (string, bool) { + switch { + case mask&(unix.IN_CREATE|unix.IN_MOVED_TO) != 0: + return "created", true + case mask&(unix.IN_DELETE|unix.IN_MOVED_FROM) != 0: + return "deleted", true + case mask&(unix.IN_MODIFY|unix.IN_ATTRIB) != 0: + return "modified", true + } + return "", false +} + +func mergeKinds(oldKind, newKind string) string { + switch { + case oldKind == "": + return newKind + case newKind == "deleted": + return "deleted" + case oldKind == "created": + return "created" + default: + return "modified" + } +} + +func readEvents(f *os.File, ch chan<- fsEvent) { + defer close(ch) + buf := make([]byte, 4096) + for { + n, err := f.Read(buf) + if err != nil { + return + } + for off := 0; off+unix.SizeofInotifyEvent <= n; { + raw := (*unix.InotifyEvent)(unsafe.Pointer(&buf[off])) + nameBytes := buf[off+unix.SizeofInotifyEvent : off+unix.SizeofInotifyEvent+int(raw.Len)] + off += unix.SizeofInotifyEvent + int(raw.Len) + name := strings.TrimRight(string(nameBytes), "\x00") + if name == "" { + continue + } + if kind, ok := classify(raw.Mask); ok { + ch <- fsEvent{kind: kind, name: name} + } + } + } +} + +type pendingEvent struct { + kind string + due time.Time +} + +func runWatchLoop(dir string, timeoutMs int) error { + ifd, err := unix.InotifyInit1(unix.IN_NONBLOCK | unix.IN_CLOEXEC) + if err != nil { + return fmt.Errorf("inotify_init1: %w", err) + } + watchMask := uint32(unix.IN_CREATE | unix.IN_MODIFY | unix.IN_ATTRIB | + unix.IN_DELETE | unix.IN_MOVED_FROM | unix.IN_MOVED_TO) + if _, err := unix.InotifyAddWatch(ifd, dir, watchMask); err != nil { + unix.Close(ifd) + if errors.Is(err, unix.ENOENT) { + return fmt.Errorf("%s: no such directory", dir) + } + return fmt.Errorf("%s: %w", dir, err) + } + inotifyFile := os.NewFile(uintptr(ifd), "inotify") + defer inotifyFile.Close() + + events := make(chan fsEvent, 64) + go readEvents(inotifyFile, events) + + sigs := make(chan os.Signal, 1) + signal.Notify(sigs, syscall.SIGINT, syscall.SIGTERM) + defer signal.Stop(sigs) + + fmt.Fprintf(os.Stderr, "fwatch: watching %s\n", dir) + timeout := time.After(time.Duration(timeoutMs) * time.Millisecond) + + pending := make(map[string]*pendingEvent) + flushTimer := time.NewTimer(time.Hour) + flushTimer.Stop() + defer flushTimer.Stop() + + flush := func(all bool) { + now := time.Now() + names := make([]string, 0, len(pending)) + for name, p := range pending { + if all || !p.due.After(now) { + names = append(names, name) + } + } + sort.Strings(names) + for _, name := range names { + fmt.Printf("event: %s %s\n", pending[name].kind, name) + delete(pending, name) + } + } + rearm := func() { + if len(pending) == 0 { + flushTimer.Stop() + return + } + earliest := time.Time{} + for _, p := range pending { + if earliest.IsZero() || p.due.Before(earliest) { + earliest = p.due + } + } + flushTimer.Reset(time.Until(earliest)) + } + + for { + select { + case ev, ok := <-events: + if !ok { + return errors.New("inotify reader stopped unexpectedly") + } + oldKind := "" + if p, exists := pending[ev.name]; exists { + oldKind = p.kind + } + pending[ev.name] = &pendingEvent{ + kind: mergeKinds(oldKind, ev.kind), + due: time.Now().Add(debounce), + } + rearm() + case <-flushTimer.C: + flush(false) + rearm() + case <-timeout: + flush(true) + fmt.Println("fwatch: exiting (timeout)") + return nil + case <-sigs: + flush(true) + fmt.Println("fwatch: exiting (signal)") + return nil + } + } +} + +func cmdWatch(dir string, timeoutMs int, sandbox bool) error { + if sandbox { + runtime.LockOSThread() // keep the sandbox-installing thread stable + + abi := landlockABI() + if abi <= 0 { + return errors.New("Landlock not supported by this kernel") + } + if err := applyLandlock(dir); err != nil { + return fmt.Errorf("landlock: %w", err) + } + fmt.Fprintf(os.Stderr, "fwatch: landlock ABI=%d enforced\n", abi) + + n, err := installSeccomp(watchSyscalls, uint32(unix.EPERM)) + if err != nil { + return fmt.Errorf("seccomp: %w", err) + } + fmt.Fprintf(os.Stderr, "fwatch: seccomp filter installed (%d syscalls allowed)\n", n) + } + return runWatchLoop(dir, timeoutMs) +} + +// --------------------------------------------------------------------------- +// probes — negative controls, one Landlock, one seccomp, exercised alone. +// --------------------------------------------------------------------------- + +// Exit codes distinct from the ordinary 0/1/2 contract: a verifier can tell +// "confirmed denied" (the PASSING case for a negative control) apart from +// "some other error" or "was NOT denied" (a sandbox bug). +const ( + probeDenied = 20 + probeNotDenied = 21 +) + +func cmdProbeOutside(sandboxDir, outsidePath string) int { + runtime.LockOSThread() + abi := landlockABI() + if abi <= 0 { + fmt.Fprintln(os.Stderr, "fwatch: probe: Landlock not supported by this kernel") + return 1 + } + if err := applyLandlock(sandboxDir); err != nil { + fmt.Fprintf(os.Stderr, "fwatch: probe: landlock: %v\n", err) + return 1 + } + fmt.Fprintf(os.Stderr, "fwatch: landlock ABI=%d enforced\n", abi) + + f, err := os.Open(outsidePath) + if err == nil { + f.Close() + fmt.Printf("fwatch: probe outside %s: opened (landlock did NOT block this)\n", outsidePath) + return probeNotDenied + } + if errors.Is(err, os.ErrPermission) { + fmt.Printf("fwatch: probe outside %s: EACCES (%v)\n", outsidePath, err) + return probeDenied + } + fmt.Printf("fwatch: probe outside %s: unexpected error: %v\n", outsidePath, err) + return 1 +} + +func cmdProbeForbiddenSyscall() int { + runtime.LockOSThread() + // socket(2) is deliberately absent from the allowlist. + n, err := installSeccomp(watchSyscalls, uint32(unix.EPERM)) + if err != nil { + fmt.Fprintf(os.Stderr, "fwatch: probe: seccomp: %v\n", err) + return 1 + } + fmt.Fprintf(os.Stderr, "fwatch: seccomp filter installed (%d syscalls allowed)\n", n) + + s, err := unix.Socket(unix.AF_INET, unix.SOCK_STREAM, 0) + if err == nil { + unix.Close(s) + fmt.Println("fwatch: probe forbidden-syscall: socket() unexpectedly succeeded") + return probeNotDenied + } + if errors.Is(err, unix.EPERM) { + fmt.Printf("fwatch: probe forbidden-syscall: EPERM (%v)\n", err) + return probeDenied + } + fmt.Printf("fwatch: probe forbidden-syscall: unexpected error: %v\n", err) + return 1 +} + +// --------------------------------------------------------------------------- +// CLI +// --------------------------------------------------------------------------- + +func usage() { + fmt.Fprintln(os.Stderr, "usage: fwatch ") + fmt.Fprintln(os.Stderr, " snapshot DIR one line per regular file") + fmt.Fprintln(os.Stderr, " diff OLD NEW compare two snapshots") + fmt.Fprintln(os.Stderr, " watch DIR [--timeout-ms T] unsandboxed watch") + fmt.Fprintln(os.Stderr, " watch --sandbox DIR [--timeout-ms T] Landlock+seccomp sandboxed watch") + fmt.Fprintln(os.Stderr, " probe --sandbox DIR --outside PATH negative control: open PATH (outside DIR) under Landlock") + fmt.Fprintln(os.Stderr, " probe --forbidden-syscall negative control: socket(2) under a seccomp allowlist that omits it") + os.Exit(2) +} + +func main() { + args := os.Args[1:] + if len(args) == 0 { + usage() + } + + switch args[0] { + case "snapshot": + if len(args) != 2 { + usage() + } + if err := cmdSnapshot(args[1]); err != nil { + fmt.Fprintf(os.Stderr, "fwatch: snapshot: %v\n", err) + os.Exit(1) + } + return + + case "diff": + if len(args) != 3 { + usage() + } + if err := cmdDiff(args[1], args[2]); err != nil { + fmt.Fprintf(os.Stderr, "fwatch: diff: %v\n", err) + os.Exit(1) + } + return + + case "watch": + sandbox := false + var dir string + timeoutMs := 2000 + for i := 1; i < len(args); i++ { + switch { + case args[i] == "--sandbox" && i+1 < len(args): + sandbox = true + i++ + dir = args[i] + case args[i] == "--timeout-ms" && i+1 < len(args): + i++ + v, err := strconv.Atoi(args[i]) + if err != nil || v <= 0 { + usage() + } + timeoutMs = v + case dir == "": + dir = args[i] + default: + usage() + } + } + if dir == "" { + usage() + } + if err := cmdWatch(dir, timeoutMs, sandbox); err != nil { + fmt.Fprintf(os.Stderr, "fwatch: watch: %v\n", err) + os.Exit(1) + } + return + + case "probe": + var sandboxDir, outside string + forbiddenSyscall := false + for i := 1; i < len(args); i++ { + switch { + case args[i] == "--sandbox" && i+1 < len(args): + i++ + sandboxDir = args[i] + case args[i] == "--outside" && i+1 < len(args): + i++ + outside = args[i] + case args[i] == "--forbidden-syscall": + forbiddenSyscall = true + default: + usage() + } + } + switch { + case forbiddenSyscall && sandboxDir == "" && outside == "": + os.Exit(cmdProbeForbiddenSyscall()) + case sandboxDir != "" && outside != "" && !forbiddenSyscall: + os.Exit(cmdProbeOutside(sandboxDir, outside)) + default: + usage() + } + + default: + usage() + } +} diff --git a/examples/33-seccomp-and-landlock/run-sandbox-checks.sh b/examples/33-seccomp-and-landlock/run-sandbox-checks.sh new file mode 100755 index 0000000..7e00350 --- /dev/null +++ b/examples/33-seccomp-and-landlock/run-sandbox-checks.sh @@ -0,0 +1,44 @@ +#!/usr/bin/env bash +# run-sandbox-checks.sh APP_BIN — staged on the lab guest and run as root +# (SUDO=1) by verify.lua. Drives all three behavioral assertions in one ssh +# round trip and prints a line per result the caller can pattern-match: +# +# forbidden-syscall-exit= (20 = confirmed EPERM, the passing case) +# outside-exit= (20 = confirmed EACCES, the passing case) +# watch-sandbox-exit= (0 = the positive control ran cleanly) +# +# Deliberately not `set -e`: a probe's nonzero exit is the expected result +# we're capturing, not a driver failure. +set -uo pipefail + +APP="${1:?usage: run-sandbox-checks.sh APP_BIN}" + +OUTSIDE_SANDBOX=/tmp/lsp33-outside-sandbox +OUTSIDE_TARGET_DIR=/tmp/lsp33-outside-target +OUTSIDE_TARGET="$OUTSIDE_TARGET_DIR/secret.txt" +WATCH_SANDBOX=/tmp/lsp33-watch-sandbox + +rm -rf "$OUTSIDE_SANDBOX" "$OUTSIDE_TARGET_DIR" "$WATCH_SANDBOX" +mkdir -p "$OUTSIDE_SANDBOX" "$OUTSIDE_TARGET_DIR" "$WATCH_SANDBOX" +echo "top secret" > "$OUTSIDE_TARGET" + +echo "=== forbidden-syscall ===" +"$APP" probe --forbidden-syscall +echo "forbidden-syscall-exit=$?" + +echo "=== outside ===" +"$APP" probe --sandbox "$OUTSIDE_SANDBOX" --outside "$OUTSIDE_TARGET" +echo "outside-exit=$?" + +echo "=== watch-sandbox (positive control: watching inside the tree still works) ===" +"$APP" watch --sandbox "$WATCH_SANDBOX" --timeout-ms 4000 & +WATCH_PID=$! +sleep 0.5 +echo hello > "$WATCH_SANDBOX/inside.txt" +echo more >> "$WATCH_SANDBOX/inside.txt" +rm -f "$WATCH_SANDBOX/inside.txt" +wait "$WATCH_PID" +echo "watch-sandbox-exit=$?" + +rm -rf "$OUTSIDE_SANDBOX" "$OUTSIDE_TARGET_DIR" "$WATCH_SANDBOX" +exit 0 diff --git a/examples/33-seccomp-and-landlock/rust/Cargo.lock b/examples/33-seccomp-and-landlock/rust/Cargo.lock new file mode 100644 index 0000000..831be5a --- /dev/null +++ b/examples/33-seccomp-and-landlock/rust/Cargo.lock @@ -0,0 +1,162 @@ +# This file is automatically @generated by Cargo. +# It is not intended for manual editing. +version = 4 + +[[package]] +name = "anyhow" +version = "1.0.104" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "330a5ed07fa54e4702c9d6c4174f74427fc0ef6e214bbd677ae50a5099946470" + +[[package]] +name = "bitflags" +version = "2.13.1" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "b588b76d00fde79687d7646a9b5bdf3cc0f655e0bbd080335a95d7e96f3587da" + +[[package]] +name = "cfg-if" +version = "1.0.4" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "9330f8b2ff13f34540b44e946ef35111825727b38d33286ef986142615121801" + +[[package]] +name = "cfg_aliases" +version = "0.2.2" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "f079e83a288787bcd14a6aea84cee5c87a67c5a3e660c30f557a3d24761b3527" + +[[package]] +name = "enumflags2" +version = "0.7.12" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "1027f7680c853e056ebcec683615fb6fbbc07dbaa13b4d5d9442b146ded4ecef" +dependencies = [ + "enumflags2_derive", +] + +[[package]] +name = "enumflags2_derive" +version = "0.7.12" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "67c78a4d8fdf9953a5c9d458f9efe940fd97a0cab0941c075a813ac594733827" +dependencies = [ + "proc-macro2", + "quote", + "syn 2.0.119", +] + +[[package]] +name = "fwatch-seccomp-landlock" +version = "0.1.0" +dependencies = [ + "anyhow", + "landlock", + "libc", + "nix", + "seccompiler", +] + +[[package]] +name = "landlock" +version = "0.4.5" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "635839550ae8b90d9fd2571460a6645dc0aec070225956ca7a2831ed31d2795d" +dependencies = [ + "enumflags2", + "libc", + "thiserror", +] + +[[package]] +name = "libc" +version = "0.2.186" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "68ab91017fe16c622486840e4c83c9a37afeff978bd239b5293d61ece587de66" + +[[package]] +name = "nix" +version = "0.30.1" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "74523f3a35e05aba87a1d978330aef40f67b0304ac79c1c00b294c9830543db6" +dependencies = [ + "bitflags", + "cfg-if", + "cfg_aliases", + "libc", +] + +[[package]] +name = "proc-macro2" +version = "1.0.107" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "985e7ec9bb745e6ce6535b544d84d6cd6f7ad8bd711c398938ae983b91a766d9" +dependencies = [ + "unicode-ident", +] + +[[package]] +name = "quote" +version = "1.0.47" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "1fbf4db142a473a8d80c26bbf18454ed458bf8d26c8219c331daecfdbd079001" +dependencies = [ + "proc-macro2", +] + +[[package]] +name = "seccompiler" +version = "0.5.0" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "a4ae55de56877481d112a559bbc12667635fdaf5e005712fd4e2b2fa50ffc884" +dependencies = [ + "libc", +] + +[[package]] +name = "syn" +version = "2.0.119" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "872831b642d1a07999a962a351ed35b955ea2cfc8f3862091e2a240a84f17297" +dependencies = [ + "proc-macro2", + "quote", + "unicode-ident", +] + +[[package]] +name = "syn" +version = "3.0.0" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "f2fac314a64dc9a36e61a9eb4261a5e9bbfbc922b27e518af97bc32b926cf967" +dependencies = [ + "proc-macro2", + "quote", + "unicode-ident", +] + +[[package]] +name = "thiserror" +version = "2.0.19" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "09a43598840e33d5b0331f38c5e30d13bb11c11210a4b58f0d9b18a5a5eefcd9" +dependencies = [ + "thiserror-impl", +] + +[[package]] +name = "thiserror-impl" +version = "2.0.19" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "43cbfe0cf76104d42a574802844187e84a305e531ed54455f11fbde0f10541cd" +dependencies = [ + "proc-macro2", + "quote", + "syn 3.0.0", +] + +[[package]] +name = "unicode-ident" +version = "1.0.24" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "e6e4313cd5fcd3dad5cafa179702e2b244f760991f45397d14d4ebf38247da75" diff --git a/examples/33-seccomp-and-landlock/rust/Cargo.toml b/examples/33-seccomp-and-landlock/rust/Cargo.toml new file mode 100644 index 0000000..3202c83 --- /dev/null +++ b/examples/33-seccomp-and-landlock/rust/Cargo.toml @@ -0,0 +1,15 @@ +[package] +name = "fwatch-seccomp-landlock" +version = "0.1.0" +edition = "2024" + +[[bin]] +name = "app" +path = "src/main.rs" + +[dependencies] +anyhow = "1" +nix = { version = "0.30", features = ["event", "inotify", "signal", "time"] } +landlock = "0.4" +seccompiler = "0.5" +libc = "0.2" diff --git a/examples/33-seccomp-and-landlock/rust/demo.sh b/examples/33-seccomp-and-landlock/rust/demo.sh new file mode 100755 index 0000000..220feea --- /dev/null +++ b/examples/33-seccomp-and-landlock/rust/demo.sh @@ -0,0 +1,26 @@ +#!/usr/bin/env bash +set -euo pipefail +cd "$(dirname "${BASH_SOURCE[0]}")" + +BIN=target/release/app + +build() { + cargo build --release +} + +run() { + if [[ -n "${TARGET:-}" ]]; then + local repo_root + repo_root=$(cd ../../.. && pwd) + "$repo_root/scripts/lab/deploy-to-vm.sh" "$TARGET" "$BIN" -- "$@" + else + "./$BIN" "$@" + fi +} + +case "${1:-}" in + build) build ;; + run) shift; run "$@" ;; + "") build; run ;; + *) echo "usage: $0 [build|run [args...]]" >&2; exit 2 ;; +esac diff --git a/examples/33-seccomp-and-landlock/rust/rust-toolchain.toml b/examples/33-seccomp-and-landlock/rust/rust-toolchain.toml new file mode 100644 index 0000000..4099f1a --- /dev/null +++ b/examples/33-seccomp-and-landlock/rust/rust-toolchain.toml @@ -0,0 +1,3 @@ +[toolchain] +channel = "1.97.1" +components = ["rustfmt", "clippy", "rust-src"] diff --git a/examples/33-seccomp-and-landlock/rust/src/main.rs b/examples/33-seccomp-and-landlock/rust/src/main.rs new file mode 100644 index 0000000..8a5e083 --- /dev/null +++ b/examples/33-seccomp-and-landlock/rust/src/main.rs @@ -0,0 +1,537 @@ +// fwatch v3 — the ch09 watcher, now sandboxed. +// +// `watch --sandbox DIR` applies two independent kernel access-control layers +// before the epoll loop ever runs: +// +// Landlock — a ruleset (the `landlock` crate) restricting filesystem READS +// to DIR. Unprivileged, self-imposed, enforced by the kernel +// against this process and every descendant. +// seccomp — a syscall allowlist (the `seccompiler` crate — Firecracker's +// BPF compiler) covering only what the watch loop needs. +// Everything else returns EPERM. +// +// Landlock's own syscall numbers have no libc wrapper; probing the running +// kernel's ABI (`landlock_abi()` below) goes through the raw syscall(2) +// numbers directly, mirroring the C++ and Go versions of this example. Those +// numbers are part of the kernel's frozen syscall table (once assigned they +// never change, unlike a struct field offset), so hardcoding them per +// architecture is the same thing glibc's own does — but the +// ABI *version* returned by the call is always probed at runtime, never +// assumed. + +use std::collections::{BTreeMap, BTreeSet}; +use std::os::linux::fs::MetadataExt; +use std::process::ExitCode; +use std::time::{Duration, Instant}; + +use anyhow::{Context, Result, bail}; +use landlock::{AccessFs, PathBeneath, PathFd, Ruleset, RulesetAttr, RulesetCreatedAttr, RulesetStatus}; +use nix::errno::Errno; +use nix::sys::epoll::{Epoll, EpollCreateFlags, EpollEvent, EpollFlags, EpollTimeout}; +use nix::sys::inotify::{AddWatchFlags, InitFlags, Inotify}; +use nix::sys::signal::{SigSet, Signal}; +use nix::sys::signalfd::{SfdFlags, SignalFd}; +use nix::sys::time::TimeSpec; +use nix::sys::timerfd::{ClockId, Expiration, TimerFd, TimerFlags, TimerSetTimeFlags}; +use seccompiler::{BpfProgram, SeccompAction, SeccompFilter, TargetArch}; + +const DEBOUNCE: Duration = Duration::from_millis(100); + +// --------------------------------------------------------------------------- +// Landlock ABI probe — raw syscall, no libc/crate wrapper exists for this. +// --------------------------------------------------------------------------- + +#[cfg(target_arch = "x86_64")] +const SYS_LANDLOCK_CREATE_RULESET: libc::c_long = 444; +const LANDLOCK_CREATE_RULESET_VERSION: libc::c_int = 1 << 0; + +/// Probes the running kernel's supported Landlock ABI version (0 if +/// Landlock isn't built in / enabled at boot). Never hardcoded. +fn landlock_abi() -> i32 { + // SAFETY: landlock_create_ruleset(NULL, 0, LANDLOCK_CREATE_RULESET_VERSION) + // is documented to only ever query the ABI version; it touches no memory. + let v = unsafe { + libc::syscall(SYS_LANDLOCK_CREATE_RULESET, std::ptr::null::(), 0usize, + LANDLOCK_CREATE_RULESET_VERSION) + }; + if v < 0 { 0 } else { v as i32 } +} + +/// Restricts filesystem reads to `dir` (ReadFile + ReadDir, both available +/// since ABI v1) via the `landlock` crate, and confirms the kernel actually +/// enforced it (not merely "best-effort, kernel too old"). +fn apply_landlock(dir: &str) -> Result<()> { + let access = AccessFs::ReadFile | AccessFs::ReadDir; + let status = Ruleset::default() + .handle_access(access) + .context("landlock: handle_access")? + .create() + .context("landlock: create ruleset")? + .add_rule(PathBeneath::new(PathFd::new(dir).context("landlock: open dir")?, access)) + .context("landlock: add_rule")? + .restrict_self() + .context("landlock: restrict_self")?; + if status.ruleset != RulesetStatus::FullyEnforced { + bail!("landlock ruleset not fully enforced by this kernel ({:?})", status.ruleset); + } + Ok(()) +} + +// --------------------------------------------------------------------------- +// seccomp — a `seccompiler` allowlist covering exactly what the watch loop +// needs (empirically confirmed with `strace -f` across a full watch +// session: create/modify/delete + debounce flush + timeout/signal exit). +// Anything else returns EPERM once the filter loads. +// --------------------------------------------------------------------------- + +fn watch_syscalls() -> Vec { + vec![ + libc::SYS_read, libc::SYS_write, libc::SYS_close, + libc::SYS_epoll_create1, libc::SYS_epoll_ctl, libc::SYS_epoll_wait, + libc::SYS_inotify_init1, libc::SYS_inotify_add_watch, + libc::SYS_timerfd_create, libc::SYS_timerfd_settime, libc::SYS_signalfd4, + libc::SYS_rt_sigprocmask, libc::SYS_rt_sigreturn, + libc::SYS_mmap, libc::SYS_munmap, libc::SYS_mremap, libc::SYS_brk, + libc::SYS_exit, libc::SYS_exit_group, + libc::SYS_fstat, libc::SYS_newfstatat, libc::SYS_lseek, + libc::SYS_clock_gettime, libc::SYS_clock_nanosleep, libc::SYS_nanosleep, + libc::SYS_getrandom, + ] +} + +/// Installs an allowlist of `allowed` syscalls; everything else returns +/// errno `deny_errno`. Returns the number of syscalls admitted. +fn install_seccomp(allowed: &[i64], deny_errno: i32) -> Result { + let rules = allowed.iter().map(|&nr| (nr, vec![])).collect(); + let filter = SeccompFilter::new( + rules, + SeccompAction::Errno(deny_errno as u32), + SeccompAction::Allow, + TargetArch::x86_64, + ) + .context("seccomp: build filter")?; + let program: BpfProgram = filter.try_into().context("seccomp: compile filter")?; + seccompiler::apply_filter(&program).context("seccomp: install filter")?; + Ok(allowed.len()) +} + +// --------------------------------------------------------------------------- +// v0: snapshot + diff +// --------------------------------------------------------------------------- + +fn cmd_snapshot(dir: &str) -> Result<()> { + let mut files: Vec<(String, u64, i64)> = Vec::new(); + let entries = std::fs::read_dir(dir).with_context(|| dir.to_string())?; + for entry in entries { + let entry = entry.with_context(|| dir.to_string())?; + let Ok(meta) = entry.metadata() else { + continue; // raced with a concurrent delete + }; + if !meta.file_type().is_file() { + continue; + } + let mtime_ns = meta.st_mtime() * 1_000_000_000 + meta.st_mtime_nsec(); + files.push((entry.file_name().to_string_lossy().into_owned(), meta.st_size(), mtime_ns)); + } + files.sort(); + for (name, size, mtime_ns) in files { + println!("{name}\t{size}\t{mtime_ns}"); + } + Ok(()) +} + +/// Parse "namesizemtime_ns" lines into name -> "sizemtime_ns"; +/// malformed lines are skipped. +fn load_snapshot(path: &str) -> Result> { + let text = std::fs::read_to_string(path).with_context(|| path.to_string())?; + let mut out = BTreeMap::new(); + for line in text.lines() { + let Some((rest, mtime)) = line.rsplit_once('\t') else { + continue; + }; + let Some((name, size)) = rest.rsplit_once('\t') else { + continue; + }; + if name.is_empty() { + continue; + } + out.insert(name.to_string(), format!("{size}\t{mtime}")); + } + Ok(out) +} + +fn cmd_diff(old_path: &str, new_path: &str) -> Result<()> { + let old_snap = load_snapshot(old_path)?; + let new_snap = load_snapshot(new_path)?; + let names: BTreeSet<&String> = old_snap.keys().chain(new_snap.keys()).collect(); + for name in names { + match (old_snap.get(name), new_snap.get(name)) { + (None, Some(_)) => println!("created {name}"), + (Some(_), None) => println!("deleted {name}"), + (Some(o), Some(n)) if o != n => println!("modified {name}"), + _ => {} + } + } + Ok(()) +} + +// --------------------------------------------------------------------------- +// v1: watch — one epoll loop, three fds, zero threads beyond main. +// --------------------------------------------------------------------------- + +#[derive(Clone, Copy, PartialEq, Eq)] +enum Kind { + Created, + Modified, + Deleted, +} + +impl Kind { + fn name(self) -> &'static str { + match self { + Kind::Created => "created", + Kind::Modified => "modified", + Kind::Deleted => "deleted", + } + } +} + +fn classify(mask: AddWatchFlags) -> Option { + if mask.intersects(AddWatchFlags::IN_CREATE | AddWatchFlags::IN_MOVED_TO) { + Some(Kind::Created) + } else if mask.intersects(AddWatchFlags::IN_DELETE | AddWatchFlags::IN_MOVED_FROM) { + Some(Kind::Deleted) + } else if mask.intersects(AddWatchFlags::IN_MODIFY | AddWatchFlags::IN_ATTRIB) { + Some(Kind::Modified) + } else { + None + } +} + +/// Coalescing rule shared by all three implementations: within one debounce +/// window, delete wins, a fresh creation stays "created" through later +/// writes, and a delete+recreate pair reads as "modified". +fn merge(old: Option, new: Kind) -> Kind { + match (old, new) { + (None, k) => k, + (_, Kind::Deleted) => Kind::Deleted, + (Some(Kind::Created), _) => Kind::Created, + _ => Kind::Modified, + } +} + +struct Pending { + kind: Kind, + due: Instant, +} + +const TOK_INOTIFY: u64 = 0; +const TOK_TIMER: u64 = 1; +const TOK_SIGNAL: u64 = 2; + +fn arm_timer(timer: &TimerFd, rel: Duration) -> Result<()> { + // A zero TimeSpec would disarm the timer; fire "immediately" instead. + let rel = rel.max(Duration::from_nanos(1)); + timer + .set(Expiration::OneShot(TimeSpec::from_duration(rel)), TimerSetTimeFlags::empty()) + .context("timerfd_settime")?; + Ok(()) +} + +fn flush(pending: &mut BTreeMap, now: Instant, all: bool) { + // BTreeMap iterates in name order, so a batch flush is deterministic. + let due: Vec = pending + .iter() + .filter(|(_, p)| all || p.due <= now) + .map(|(name, _)| name.clone()) + .collect(); + for name in due { + let p = pending.remove(&name).expect("collected from this map"); + println!("event: {} {}", p.kind.name(), name); + } +} + +/// The watch loop itself. Whatever sandboxing the caller applied (Landlock + +/// seccomp) has already happened before this runs; the loop's own logic +/// never needs to know. +fn run_watch_loop(dir: &str, timeout_ms: u64) -> Result<()> { + let inotify = + Inotify::init(InitFlags::IN_NONBLOCK | InitFlags::IN_CLOEXEC).context("inotify_init1")?; + let watch_mask = AddWatchFlags::IN_CREATE + | AddWatchFlags::IN_MODIFY + | AddWatchFlags::IN_ATTRIB + | AddWatchFlags::IN_DELETE + | AddWatchFlags::IN_MOVED_FROM + | AddWatchFlags::IN_MOVED_TO; + inotify.add_watch(dir, watch_mask).with_context(|| dir.to_string())?; + + let timer = TimerFd::new( + ClockId::CLOCK_MONOTONIC, + TimerFlags::TFD_NONBLOCK | TimerFlags::TFD_CLOEXEC, + ) + .context("timerfd_create")?; + + let mut sigset = SigSet::empty(); + sigset.add(Signal::SIGINT); + sigset.add(Signal::SIGTERM); + sigset.thread_block().context("sigprocmask")?; + let sigfd = SignalFd::with_flags(&sigset, SfdFlags::SFD_NONBLOCK | SfdFlags::SFD_CLOEXEC) + .context("signalfd")?; + + let epoll = Epoll::new(EpollCreateFlags::EPOLL_CLOEXEC).context("epoll_create1")?; + epoll.add(&inotify, EpollEvent::new(EpollFlags::EPOLLIN, TOK_INOTIFY)).context("epoll_ctl")?; + epoll.add(&timer, EpollEvent::new(EpollFlags::EPOLLIN, TOK_TIMER)).context("epoll_ctl")?; + epoll.add(&sigfd, EpollEvent::new(EpollFlags::EPOLLIN, TOK_SIGNAL)).context("epoll_ctl")?; + + eprintln!("fwatch: watching {dir}"); + let deadline = Instant::now() + Duration::from_millis(timeout_ms); + let mut pending: BTreeMap = BTreeMap::new(); + + loop { + // One timer covers both jobs: it is always armed to whichever comes + // first — the overall timeout or the earliest per-path debounce. + let next = pending.values().map(|p| p.due).min().map_or(deadline, |d| d.min(deadline)); + arm_timer(&timer, next.saturating_duration_since(Instant::now()))?; + + let mut events = [EpollEvent::empty(); 8]; + let n = match epoll.wait(&mut events, EpollTimeout::NONE) { + Ok(n) => n, + Err(Errno::EINTR) => continue, + Err(e) => return Err(e).context("epoll_wait"), + }; + + for event in &events[..n] { + match event.data() { + TOK_INOTIFY => loop { + let batch = match inotify.read_events() { + Ok(batch) => batch, + Err(Errno::EAGAIN) => break, // queue drained + Err(e) => return Err(e).context("inotify read"), + }; + let now = Instant::now(); + for ev in batch { + let Some(name) = ev.name else { + continue; // event on the directory itself + }; + let Some(kind) = classify(ev.mask) else { + continue; + }; + let name = name.to_string_lossy().into_owned(); + let old = pending.get(&name).map(|p| p.kind); + pending + .insert(name, Pending { kind: merge(old, kind), due: now + DEBOUNCE }); + } + }, + TOK_TIMER => { + let mut buf = [0u8; 8]; + let _ = nix::unistd::read(&timer, &mut buf); + let now = Instant::now(); + if now >= deadline { + flush(&mut pending, now, true); + println!("fwatch: exiting (timeout)"); + return Ok(()); + } + flush(&mut pending, now, false); + } + TOK_SIGNAL => { + let _ = sigfd.read_signal(); + flush(&mut pending, Instant::now(), true); + println!("fwatch: exiting (signal)"); + return Ok(()); + } + _ => {} + } + } + } +} + +fn cmd_watch(dir: &str, timeout_ms: u64, sandbox: bool) -> Result<()> { + if sandbox { + let abi = landlock_abi(); + if abi <= 0 { + bail!("Landlock not supported by this kernel"); + } + apply_landlock(dir)?; + eprintln!("fwatch: landlock ABI={abi} enforced"); + + let n = install_seccomp(&watch_syscalls(), libc::EPERM)?; + eprintln!("fwatch: seccomp filter installed ({n} syscalls allowed)"); + } + run_watch_loop(dir, timeout_ms) +} + +// --------------------------------------------------------------------------- +// probes — negative controls, one Landlock, one seccomp, exercised alone. +// --------------------------------------------------------------------------- + +// Exit codes distinct from the ordinary 0/1/2 contract: a verifier can tell +// "confirmed denied" (the PASSING case for a negative control) apart from +// "some other error" or "was NOT denied" (a sandbox bug). +const PROBE_DENIED: u8 = 20; +const PROBE_NOT_DENIED: u8 = 21; + +fn cmd_probe_outside(sandbox_dir: &str, outside_path: &str) -> ExitCode { + let abi = landlock_abi(); + if abi <= 0 { + eprintln!("fwatch: probe: Landlock not supported by this kernel"); + return ExitCode::from(1); + } + if let Err(e) = apply_landlock(sandbox_dir) { + eprintln!("fwatch: probe: landlock: {e:#}"); + return ExitCode::from(1); + } + eprintln!("fwatch: landlock ABI={abi} enforced"); + + match std::fs::File::open(outside_path) { + Ok(f) => { + drop(f); + println!("fwatch: probe outside {outside_path}: opened (landlock did NOT block this)"); + ExitCode::from(PROBE_NOT_DENIED) + } + Err(e) if e.kind() == std::io::ErrorKind::PermissionDenied => { + println!("fwatch: probe outside {outside_path}: EACCES ({e})"); + ExitCode::from(PROBE_DENIED) + } + Err(e) => { + println!("fwatch: probe outside {outside_path}: unexpected error: {e}"); + ExitCode::from(1) + } + } +} + +fn cmd_probe_forbidden_syscall() -> ExitCode { + // socket(2) is deliberately absent from the allowlist. + let n = match install_seccomp(&watch_syscalls(), libc::EPERM) { + Ok(n) => n, + Err(e) => { + eprintln!("fwatch: probe: seccomp: {e:#}"); + return ExitCode::from(1); + } + }; + eprintln!("fwatch: seccomp filter installed ({n} syscalls allowed)"); + + // SAFETY: socket(2) with these arguments has no memory-safety + // implications; we only inspect its return value. + let s = unsafe { libc::socket(libc::AF_INET, libc::SOCK_STREAM, 0) }; + if s >= 0 { + unsafe { libc::close(s) }; + println!("fwatch: probe forbidden-syscall: socket() unexpectedly succeeded"); + return ExitCode::from(PROBE_NOT_DENIED); + } + let err = std::io::Error::last_os_error(); + if err.raw_os_error() == Some(libc::EPERM) { + println!("fwatch: probe forbidden-syscall: EPERM ({err})"); + ExitCode::from(PROBE_DENIED) + } else { + println!("fwatch: probe forbidden-syscall: unexpected error: {err}"); + ExitCode::from(1) + } +} + +// --------------------------------------------------------------------------- +// CLI +// --------------------------------------------------------------------------- + +fn usage() -> ExitCode { + eprintln!("usage: fwatch "); + eprintln!(" snapshot DIR one line per regular file"); + eprintln!(" diff OLD NEW compare two snapshots"); + eprintln!(" watch DIR [--timeout-ms T] unsandboxed watch"); + eprintln!(" watch --sandbox DIR [--timeout-ms T] Landlock+seccomp sandboxed watch"); + eprintln!(" probe --sandbox DIR --outside PATH negative control: open PATH (outside DIR) under Landlock"); + eprintln!(" probe --forbidden-syscall negative control: socket(2) under a seccomp allowlist that omits it"); + ExitCode::from(2) +} + +fn parse_watch_args(args: &[&str]) -> Option<(String, u64, bool)> { + let mut sandbox = false; + let mut dir: Option = None; + let mut timeout_ms = 2000u64; + let mut i = 0; + while i < args.len() { + match args[i] { + "--sandbox" if i + 1 < args.len() => { + sandbox = true; + dir = Some(args[i + 1].to_string()); + i += 2; + } + "--timeout-ms" if i + 1 < args.len() => { + timeout_ms = args[i + 1].parse().ok().filter(|v| *v > 0)?; + i += 2; + } + other if dir.is_none() => { + dir = Some(other.to_string()); + i += 1; + } + _ => return None, + } + } + Some((dir?, timeout_ms, sandbox)) +} + +fn main() -> ExitCode { + let args: Vec = std::env::args().skip(1).collect(); + let argv: Vec<&str> = args.iter().map(String::as_str).collect(); + if argv.is_empty() { + return usage(); + } + + match argv[0] { + "snapshot" if argv.len() == 2 => match cmd_snapshot(argv[1]) { + Ok(()) => ExitCode::SUCCESS, + Err(e) => { + eprintln!("fwatch: snapshot: {e:#}"); + ExitCode::from(1) + } + }, + "diff" if argv.len() == 3 => match cmd_diff(argv[1], argv[2]) { + Ok(()) => ExitCode::SUCCESS, + Err(e) => { + eprintln!("fwatch: diff: {e:#}"); + ExitCode::from(1) + } + }, + "watch" => { + let Some((dir, timeout_ms, sandbox)) = parse_watch_args(&argv[1..]) else { + return usage(); + }; + match cmd_watch(&dir, timeout_ms, sandbox) { + Ok(()) => ExitCode::SUCCESS, + Err(e) => { + eprintln!("fwatch: watch: {e:#}"); + ExitCode::from(1) + } + } + } + "probe" => { + let mut sandbox_dir: Option<&str> = None; + let mut outside: Option<&str> = None; + let mut forbidden_syscall = false; + let rest = &argv[1..]; + let mut i = 0; + while i < rest.len() { + match rest[i] { + "--sandbox" if i + 1 < rest.len() => { + sandbox_dir = Some(rest[i + 1]); + i += 2; + } + "--outside" if i + 1 < rest.len() => { + outside = Some(rest[i + 1]); + i += 2; + } + "--forbidden-syscall" => { + forbidden_syscall = true; + i += 1; + } + _ => return usage(), + } + } + match (sandbox_dir, outside, forbidden_syscall) { + (None, None, true) => cmd_probe_forbidden_syscall(), + (Some(d), Some(p), false) => cmd_probe_outside(d, p), + _ => usage(), + } + } + _ => usage(), + } +} diff --git a/examples/33-seccomp-and-landlock/verify.lua b/examples/33-seccomp-and-landlock/verify.lua new file mode 100644 index 0000000..530f7fa --- /dev/null +++ b/examples/33-seccomp-and-landlock/verify.lua @@ -0,0 +1,120 @@ +-- Verify fwatch v3 (seccomp + Landlock) — a VM example. Behavioral +-- assertions, driven by run-sandbox-checks.sh staged and run as root +-- (SUDO=1) on the lab guest: +-- +-- 1. NEGATIVE CONTROL — `probe --forbidden-syscall` installs a seccomp +-- allowlist that omits socket(2), then calls it. The bare exit code +-- alone would prove nothing (a program that always exits nonzero would +-- "pass"); the assertion is that socket(2) specifically returned EPERM, +-- confirmed both by the process's own exit code (20, this program's +-- "confirmed denied" convention) AND the printed errno text. +-- 2. NEGATIVE CONTROL — `probe --sandbox DIR --outside PATH` restricts +-- Landlock to DIR, then opens a file under a DIFFERENT directory. +-- Same shape: exit 20 plus an explicit EACCES line. +-- 3. POSITIVE CONTROL — `watch --sandbox DIR` with both layers applied +-- still sees create/modify/delete events for a file created INSIDE +-- DIR and exits 0 on timeout. This is what makes 1 and 2 meaningful: +-- the sandbox denies specific things without breaking the watcher's +-- actual job. +-- +-- Runs under the runner's vm mode (TARGET=systems-target, deploy via a +-- custom root-run driver script — deploy-to-vm.sh only shells out a single +-- binary+args, and this needs mkdir/rm scaffolding around three invocations +-- of the same binary in one ssh round trip, same shape as example 30's +-- observe.sh). + +local script_dir = arg[0]:match("(.*/)") or "./" +local repo_root = os.getenv("REPO_ROOT") or (script_dir .. "../..") +package.path = package.path .. ";" .. repo_root .. "/scripts/lib/?.lua" +local checks = require("checks") + +local lang = os.getenv("LSP_LANG") +if lang ~= "cpp" and lang ~= "go" and lang ~= "rust" then + checks.skip("LSP_LANG must be cpp|go|rust (got " .. tostring(lang) .. ")") +end + +local target = os.getenv("TARGET") +if not target or target == "" then + checks.skip("no TARGET set — this is a vm example (run with --mode vm)") +end + +local example_dir = os.getenv("EXAMPLE_DIR") or "." +local home = os.getenv("HOME") or "/root" + +-- Local path of the freshly built binary for this language. +local bin_rel = ({ + cpp = "cpp/build/release/app", + go = "go/bin/app", + rust = "rust/target/release/app", +})[lang] +local local_bin = example_dir .. "/" .. bin_rel +local local_driver = example_dir .. "/run-sandbox-checks.sh" + +-- Resolve the guest IP (same helper deploy-to-vm.sh uses). +local ipf = io.popen(repo_root .. "/scripts/lab/vm-ip.sh '" .. target .. "' 2>/dev/null") +local ip = ipf and ipf:read("*l") or nil +if ipf then ipf:close() end +if not ip or ip == "" then + checks.skip("cannot resolve IP for VM '" .. target .. "' (lab down?)") +end + +local ssh_opts = + "-o StrictHostKeyChecking=accept-new -o UserKnownHostsFile=" .. home .. "/.ssh/known_hosts" + +-- Stage the binary under a per-language directory (so /proc comm is always +-- "app", the book-wide rule) alongside the driver script. +local remote_dir = "/tmp/lsp33-" .. lang +local remote_bin = remote_dir .. "/app" +local remote_driver = "/tmp/lsp33-run-sandbox-checks.sh" + +local stage_cmd = string.format( + "ssh %s 'fedora@%s' 'mkdir -p %s' && " .. + "scp %s '%s' 'fedora@%s:%s' && " .. + "scp %s '%s' 'fedora@%s:%s' && " .. + "ssh %s 'fedora@%s' 'chmod 755 %s %s'", + ssh_opts, ip, remote_dir, + ssh_opts, local_bin, ip, remote_bin, + ssh_opts, local_driver, ip, remote_driver, + ssh_opts, ip, remote_bin, remote_driver) +local st = checks.run(stage_cmd) +checks.expect_exit(st, 0, lang .. ": stage app binary + run-sandbox-checks.sh on guest") + +-- Run the driver as root; Landlock/seccomp are unprivileged mechanisms, but +-- this book's convention keeps every namespace/cgroup/seccomp/Landlock demo +-- on the VM path (SUDO=1) for consistency with its siblings. +local driven = checks.run(string.format( + "ssh %s 'fedora@%s' 'sudo bash %s %s'", + ssh_opts, ip, remote_driver, remote_bin)) +checks.expect_exit(driven, 0, lang .. ": run-sandbox-checks.sh exits 0") +if driven.exit ~= 0 then + print(driven.out) +end +local out = driven.out + +-- 1. seccomp negative control: socket(2) denied with EPERM. +checks.expect_match(out, "fwatch: seccomp filter installed %(%d+ syscalls allowed%)", + lang .. ": seccomp filter installed line printed") +checks.expect_match(out, "fwatch: probe forbidden%-syscall: EPERM", + lang .. ": forbidden syscall (socket) denied with EPERM") +checks.expect_match(out, "forbidden%-syscall%-exit=20", + lang .. ": probe --forbidden-syscall confirms denial (exit 20)") + +-- 2. Landlock negative control: a read outside the sandboxed tree denied +-- with EACCES. +checks.expect_match(out, "fwatch: landlock ABI=%d+ enforced", + lang .. ": landlock ABI enforced line printed") +checks.expect_match(out, "fwatch: probe outside /tmp/lsp33%-outside%-target/secret%.txt: EACCES", + lang .. ": read outside the Landlock tree denied with EACCES") +checks.expect_match(out, "outside%-exit=20", + lang .. ": probe --outside confirms denial (exit 20)") + +-- 3. Positive control: the sandboxed watch loop still sees events inside +-- the tree and exits cleanly on timeout. +checks.expect_match(out, "event: %a+ inside%.txt", + lang .. ": sandboxed watch still reports events for a file inside the tree") +checks.expect_match(out, "fwatch: exiting %(timeout%)", + lang .. ": sandboxed watch exits cleanly on timeout") +checks.expect_match(out, "watch%-sandbox%-exit=0", + lang .. ": watch --sandbox exits 0 (positive control passes)") + +checks.finish() diff --git a/examples/34-our-programs-in-containers/README.md b/examples/34-our-programs-in-containers/README.md new file mode 100644 index 0000000..b35017c --- /dev/null +++ b/examples/34-our-programs-in-containers/README.md @@ -0,0 +1,198 @@ +# 34 — our programs in containers + +`app`, a tiny PID-1 container entrypoint, implemented three times — C++23, +Go, and Rust — each built into its own multi-stage **Containerfile** (UBI10 +builder → UBI10-micro runtime) and run under **rootless podman**. It +demonstrates three things a program running as a container's entrypoint has +to get right that a normal program never has to think about: + +``` +usage: app serve|naive # C++, Go, Rust +usage: app serve|naive|worker|job # Go, Rust (self-reexec children; see below) +``` + +1. **pid-1 signal duties** — `serve` reaps every child (so short-lived jobs + never pile up as zombies) and forwards `SIGTERM` to the worker it + supervises before exiting. `naive` installs no signal handling at all, the + mistake this example exists to make visible. +2. **container-aware resource detection** — `serve`/`naive` read their own + cgroup v2 `cpu.max`/`memory.max` and print + `container: cpu.max= effective_parallelism= mem.max=`. Go's + `GOMAXPROCS` and Rust's `available_parallelism()` are cgroup-aware; C++'s + `hardware_concurrency()` is not — it reports the **host's** cpu count + however small `--cpus` is set. That mismatch is the trap. +3. **debugging across namespaces** — `podman exec` and a rootless + `podman unshare nsenter` from the host both reach the same PID 1 inside + the running container. + +## Layout + +``` +34-our-programs-in-containers/ +├── demo.sh # dispatcher: ./demo.sh [cpp|go|rust|all|build] [args...] +├── verify.lua # podman build/run/stop, behavioral checks +├── cpp/ # CMake preset build, demo.sh, Containerfile +├── go/ # go build, demo.sh, Containerfile +└── rust/ # cargo build, demo.sh, Containerfile +``` + +Each language directory's `Containerfile` builds a small (a few MB to a few +tens of MB) runtime image: + +| Stage | Base | Does | +|---|---|---| +| builder | `registry.access.redhat.com/ubi10/ubi:10.2` | installs the toolchain, compiles `app` | +| runtime | `registry.access.redhat.com/ubi10-micro:10.2` | copies just the binary, sets `ENTRYPOINT`/`CMD` | + +C++'s builder links `libstdc++`/`libgcc` statically (`libstdc++-static`), so +the runtime needs nothing beyond glibc. Go builds with `CGO_ENABLED=0` for a +fully static binary. Rust's builder installs the exact `rustc 1.97.1` pinned +by `rust-toolchain.toml` via `rustup` (UBI10's `dnf` package is 1.92.0); the +resulting binary needs only `libc`/`libgcc_s`, both already in +`ubi10-micro`. + +## Output contract (identical shape across all three languages) + +| Stream | Line | When | +|---|---|---| +| stdout | `container: cpu.max= effective_parallelism= mem.max=` | first line, `serve` and `naive` | +| stdout | `app: pid= ppid=` | second line, `serve` and `naive` | +| stdout | `app: worker started pid=` | `serve`, once | +| stdout | `app: worker pid= tick=` | `serve`, every 2s from the worker | +| stdout | `app: job pid= seq= done` | `serve`, every ~1s, from a short-lived job | +| stdout | `app: reaped pid= status=` | `serve`, immediately after each job/worker exit | +| stdout | `app: naive heartbeat tick=` | `naive`, every 1s, forever | +| stdout | `app: shutting down (SIGTERM\|SIGINT)` | `serve` only, right before it exits 0 | + +`naive` never prints a shutdown line — that absence is itself the evidence +the example verifies. + +`cpu.max` is printed as `max` (unconstrained) or `/` (derived +from the cgroup file's raw `" "` content); `mem.max` is +printed exactly as the cgroup reports it (`max` or a byte count). + +## The trap, reproduced for real + +Run with `podman run --cpus=2 --memory=128m`, all three correctly report the +container's `cpu.max=200000/100000` and `mem.max=134217728` (not the host's +`max`/`max`) — but `effective_parallelism` tells a different story per +language, captured from a real run on a 16-cpu host: + +``` +cpp: container: cpu.max=200000/100000 effective_parallelism=16 mem.max=134217728 +go: container: cpu.max=200000/100000 effective_parallelism=2 mem.max=134217728 +rust: container: cpu.max=200000/100000 effective_parallelism=2 mem.max=134217728 +``` + +C++'s `std::thread::hardware_concurrency()` is `sysconf(_SC_NPROCESSORS_ONLN)` +under the hood: it has no idea a cgroup exists. Go's `GOMAXPROCS` (default +container-aware as of the `go 1.26` language version) and Rust's +`available_parallelism()` both divide the cgroup's cpu quota by its period +and report `2` — the number that is actually true. + +## Per-language mechanics + +- **C++23** (`cpp/src/main.cpp`) — raw `fork(2)` for both the worker and each + job (no exec: they are just different functions in the same binary). This + is the one place a subtle, very real bug lived during development: the + forked worker **inherits the parent's blocked signal mask** + (`sigprocmask(SIG_BLOCK, ...)` for `SIGTERM`/`SIGINT`/`SIGCHLD`, set up so + `serve` can receive them via `signalfd(2)`). Left alone, a forwarded + `SIGTERM` to the worker would just sit *pending* forever instead of + running its default (terminate) action — the worker would be unkillable. + The fix is one `sigprocmask(SIG_SETMASK, &empty, ...)` in the child right + after `fork()`, the same principle as pmon's (ch. 12) reset-before-`execvp`, + just with no `exec` to do it for you. `reap_all()` drains + `waitpid(-1, WNOHANG)` from the ordinary `poll(2)` loop (never from signal + context) whenever the `signalfd` reports `SIGCHLD`. A `std::jthread` (with + a `stop_token`, RAII-joined) drives the periodic job spawns. +- **Go 1.26** (`go/main.go`) — no `fork(2)`-without-exec (Go's runtime is not + fork-safe once other goroutines/threads exist), so the worker and each job + are separate processes started by **re-executing the same binary** + (`os.Executable()` + `exec.Command(self, "worker"|"job", ...)`). No + `SIGCHLD`/`signalfd` anywhere: `Cmd.Wait()` calls `wait4(2)` for you, so one + goroutine parked in `Wait()` per spawned child *is* the reap loop. A real, + verified surprise from building this: a Go binary that never calls + `signal.Notify` still **dies immediately** on `SIGTERM`, even as PID 1 — + Go's runtime installs its own handler for fatal signals like `SIGTERM` at + startup regardless of user code (see `runtime`'s internal signal table), + so it never has an unclaimed `SIG_DFL` disposition for the kernel's PID-1 + special case to mask out. `naive` in Go is therefore not "unresponsive" the + way C++'s/Rust's are — it is abrupt instead: no reaping, no forwarding, no + shutdown message, just an instant, uncontrolled death. +- **Rust edition 2024** (`rust/src/main.rs`) — same self-reexec approach as + Go, via `std::env::current_exe()` + `std::process::Command`, for the same + reason (no fork-without-exec once threads exist). `signal_hook::iterator:: + Signals::new([SIGTERM, SIGINT])` turns both into a blocking iterator on the + main thread — no polling, no manual mask bookkeeping, and (unlike the + `sigprocmask`-based approach) nothing to reset before `exec`, since + `signal_hook` only installs handlers and never blocks the signals it + watches. Reaping is `std::thread::spawn` + `Child::wait()` per job, exactly + like Go's goroutine-per-child. + +## Try it + +```bash +# Build and run locally (unconstrained: cpu.max=max mem.max=max) +./demo.sh cpp run serve +./demo.sh go run naive + +# Build and run the actual container image, constrained +podman build -t lsp34-cpp -f cpp/Containerfile cpp +podman run --rm --cpus=2 --memory=128m lsp34-cpp serve + +# Compare the trap side by side (2 vCPUs, three languages) +for lang in cpp go rust; do + podman build -t lsp34-$lang -f $lang/Containerfile $lang >/dev/null + podman run --rm --cpus=2 lsp34-$lang serve --help >/dev/null 2>&1 || true + timeout 2 podman run --rm --cpus=2 lsp34-$lang serve | head -1 +done + +# Debugging across namespaces: podman exec, then a rootless nsenter from the host +podman run -d --name demo34 --cpus=2 --memory=128m lsp34-go serve +podman exec demo34 cat /proc/1/status | head -3 +HOSTPID=$(podman inspect --format '{{.State.Pid}}' demo34) +podman unshare nsenter --target "$HOSTPID" --pid --mount -- readlink /proc/1/exe +podman stop -t 3 demo34 # near-instant: serve forwards SIGTERM and exits cleanly +podman rm -f demo34; podman rmi -f lsp34-go +``` + +`nsenter` directly (without `podman unshare`) fails rootless with +`Operation not permitted` — the container's namespaces live inside podman's +own unprivileged user namespace, and `podman unshare` is what puts you inside +it first. + +## Verification (behavioral) + +`verify.lua` requires podman on the host (`skip()`, exit 77, if it is +missing) and, for every language, actually builds the image and drives real +containers rather than trusting exit codes: + +- **Resource detection under real constraints** — `podman run --cpus=2 + --memory=128m`, then the container's own logs must show the *constrained* + `cpu.max=200000/100000` and `mem.max=134217728` — never the host's `max`. + C++'s `effective_parallelism` must equal the **host's** live `nproc` (the + trap, reproduced); Go's and Rust's must equal `2` (the cgroup quota). The + host must report `nproc >= 3` or the test skips, since otherwise C++'s + "wrong" number would coincidentally equal the "right" one. +- **PID-1 identity and reaping** — `app: pid=1 ppid=0` in the logs; several + `app: reaped pid=... status=0` lines with no pile-up; a `podman exec` + zombie scan of `/proc/[0-9]*/stat` reports `zombies=0` while jobs are being + spawned and reaped. +- **Cross-namespace debugging** — `podman exec ... cat /proc/1/status` names + the process `app`; a rootless `podman unshare nsenter --target + --pid --mount -- readlink /proc/1/exe` from the host resolves to the same + `/usr/local/bin/app`. +- **`serve` vs `naive`, timed** — `podman stop -t 3` on `serve` completes in + well under a second (real runs: ~90-220ms) with a `shutting down (SIGTERM)` + line in the logs. `podman stop -t 2` on `naive`: for C++/Rust this takes + the **full grace period** (~2.1s) and podman's own log shows it fell back + to `SIGKILL`; for Go it stops fast anyway (~90ms) for the runtime reason + above, but in every language the shutdown line **never appears** in + `naive`'s logs, proving no graceful path ran regardless of how the process + actually died. +- Every container and the built image are removed at the end, whether the + checks passed or not. + +All three languages pass all 16 checks on the reference host (Fedora 44, +kernel 7.1.3-200.fc44, podman 5.8.4 rootless). diff --git a/examples/34-our-programs-in-containers/cpp/.clang-format b/examples/34-our-programs-in-containers/cpp/.clang-format new file mode 100644 index 0000000..2c8640f --- /dev/null +++ b/examples/34-our-programs-in-containers/cpp/.clang-format @@ -0,0 +1,8 @@ +--- +BasedOnStyle: LLVM +Language: Cpp +Standard: c++20 +IndentWidth: 4 +ColumnLimit: 100 +PointerAlignment: Left +AllowShortFunctionsOnASingleLine: Inline diff --git a/examples/34-our-programs-in-containers/cpp/.clang-tidy b/examples/34-our-programs-in-containers/cpp/.clang-tidy new file mode 100644 index 0000000..2f275ba --- /dev/null +++ b/examples/34-our-programs-in-containers/cpp/.clang-tidy @@ -0,0 +1,11 @@ +--- +Checks: > + bugprone-*, + modernize-*, + performance-*, + readability-*, + -modernize-use-trailing-return-type, + -readability-identifier-length +WarningsAsErrors: '' +HeaderFilterRegex: '.*' +FormatStyle: file diff --git a/examples/34-our-programs-in-containers/cpp/CMakeLists.txt b/examples/34-our-programs-in-containers/cpp/CMakeLists.txt new file mode 100644 index 0000000..33cdd90 --- /dev/null +++ b/examples/34-our-programs-in-containers/cpp/CMakeLists.txt @@ -0,0 +1,11 @@ +cmake_minimum_required(VERSION 3.25) +project(containers_app CXX) + +find_package(Threads REQUIRED) + +set(CMAKE_CXX_STANDARD 23) +set(CMAKE_CXX_STANDARD_REQUIRED ON) +set(CMAKE_CXX_EXTENSIONS OFF) + +add_executable(app src/main.cpp) +target_link_libraries(app PRIVATE Threads::Threads) diff --git a/examples/34-our-programs-in-containers/cpp/CMakePresets.json b/examples/34-our-programs-in-containers/cpp/CMakePresets.json new file mode 100644 index 0000000..e31bd2f --- /dev/null +++ b/examples/34-our-programs-in-containers/cpp/CMakePresets.json @@ -0,0 +1,62 @@ +{ + "version": 6, + "cmakeMinimumRequired": { "major": 3, "minor": 25, "patch": 0 }, + "configurePresets": [ + { + "name": "_base", + "hidden": true, + "generator": "Ninja", + "binaryDir": "${sourceDir}/build/${presetName}", + "cacheVariables": { + "CMAKE_EXPORT_COMPILE_COMMANDS": "ON" + } + }, + { + "name": "release", + "inherits": "_base", + "displayName": "Release (-O2 with debug info)", + "cacheVariables": { "CMAKE_BUILD_TYPE": "RelWithDebInfo" } + }, + { + "name": "debug", + "inherits": "_base", + "displayName": "Debug", + "cacheVariables": { "CMAKE_BUILD_TYPE": "Debug" } + }, + { + "name": "asan", + "inherits": "_base", + "displayName": "ASan + UBSan instrumented build", + "cacheVariables": { + "CMAKE_BUILD_TYPE": "Debug", + "CMAKE_CXX_FLAGS": "-O1 -g -fno-omit-frame-pointer -fsanitize=address,undefined", + "CMAKE_EXE_LINKER_FLAGS": "-fsanitize=address,undefined" + } + }, + { + "name": "release-clang", + "inherits": "release", + "displayName": "Release (clang)", + "cacheVariables": { + "CMAKE_C_COMPILER": "clang", + "CMAKE_CXX_COMPILER": "clang++" + } + }, + { + "name": "debug-clang", + "inherits": "debug", + "displayName": "Debug (clang)", + "cacheVariables": { + "CMAKE_C_COMPILER": "clang", + "CMAKE_CXX_COMPILER": "clang++" + } + } + ], + "buildPresets": [ + { "name": "release", "configurePreset": "release" }, + { "name": "debug", "configurePreset": "debug" }, + { "name": "asan", "configurePreset": "asan" }, + { "name": "release-clang", "configurePreset": "release-clang" }, + { "name": "debug-clang", "configurePreset": "debug-clang" } + ] +} diff --git a/examples/34-our-programs-in-containers/cpp/Containerfile b/examples/34-our-programs-in-containers/cpp/Containerfile new file mode 100644 index 0000000..243ad55 --- /dev/null +++ b/examples/34-our-programs-in-containers/cpp/Containerfile @@ -0,0 +1,22 @@ +# app (C++23) — multi-stage: UBI10 builder -> UBI10-micro runtime. +# +# podman build -t lsp34-cpp -f Containerfile . +# podman run --rm --cpus=2 --memory=128m lsp34-cpp serve +# +# libstdc++/libgcc are statically linked so the runtime stage needs nothing +# beyond glibc (already in ubi10-micro) -- no microdnf, no extra RPMs copied. + +FROM registry.access.redhat.com/ubi10/ubi:10.2 AS builder +RUN dnf -y install gcc-c++ cmake ninja-build libstdc++-static && dnf clean all +WORKDIR /src +COPY CMakeLists.txt CMakePresets.json ./ +COPY src ./src +RUN cmake --preset release \ + -DCMAKE_CXX_FLAGS="-static-libstdc++ -static-libgcc" \ + -DCMAKE_EXE_LINKER_FLAGS="-static-libstdc++ -static-libgcc" \ + && cmake --build --preset release + +FROM registry.access.redhat.com/ubi10-micro:10.2 AS runtime +COPY --from=builder /src/build/release/app /usr/local/bin/app +ENTRYPOINT ["/usr/local/bin/app"] +CMD ["serve"] diff --git a/examples/34-our-programs-in-containers/cpp/demo.sh b/examples/34-our-programs-in-containers/cpp/demo.sh new file mode 100755 index 0000000..6af6509 --- /dev/null +++ b/examples/34-our-programs-in-containers/cpp/demo.sh @@ -0,0 +1,40 @@ +#!/usr/bin/env bash +set -euo pipefail + +cd "$(dirname "${BASH_SOURCE[0]}")" +REPO_ROOT="$(cd ../../.. && pwd)" +BIN=build/release/app + +build() { + cmake --preset release + cmake --build --preset release +} + +run() { + if [[ ! -x "$BIN" ]]; then + build + fi + if [[ -n "${TARGET:-}" ]]; then + "$REPO_ROOT/scripts/lab/deploy-to-vm.sh" "$TARGET" "$BIN" -- "$@" + else + "./$BIN" "$@" + fi +} + +case "${1:-}" in + build) + build + ;; + run) + shift + run "$@" + ;; + "") + build + run + ;; + *) + echo "usage: $0 [build|run [args]]" >&2 + exit 2 + ;; +esac diff --git a/examples/34-our-programs-in-containers/cpp/src/main.cpp b/examples/34-our-programs-in-containers/cpp/src/main.cpp new file mode 100644 index 0000000..1ad9ab6 --- /dev/null +++ b/examples/34-our-programs-in-containers/cpp/src/main.cpp @@ -0,0 +1,316 @@ +// app — a tiny PID-1 container entrypoint (chapter 34: our programs in +// containers). +// +// app serve run as the container's PID 1: reap zombies, forward SIGTERM +// to the supervised worker, report the REAL container limits. +// app naive the trap: no signal handling installed at all. A process +// that is PID 1 of its own PID namespace (i.e. the container's +// entrypoint) is special-cased by the kernel: any signal for +// which it has installed no explicit handler is dropped +// instead of running that signal's default action. Only +// SIGKILL (and SIGSTOP) still work. `naive` never calls +// sigaction for anything, so `podman stop` cannot end it +// gracefully -- the engine has to wait out the stop timeout +// and fall back to SIGKILL. +// +// Container-aware resource detection: hardware_concurrency() is +// sysconf(_SC_NPROCESSORS_ONLN) under the hood -- the HOST's cpu count. It +// never reads a cgroup, so under `podman run --cpus=2` this still prints the +// host's full core count. That mismatch against Go's GOMAXPROCS and Rust's +// available_parallelism() (both cgroup-aware) is the whole point of this +// example: raw C++ threading primitives do not know they are in a +// container. + +#include +#include +#include +#include +#include +#include +#include +#include +#include +#include +#include +#include +#include +#include + +#include +#include +#include +#include +#include + +namespace { + +using namespace std::chrono_literals; + +[[nodiscard]] std::error_code errno_ec() { + return {errno, std::generic_category()}; +} + +[[noreturn]] void die(std::string_view what, std::error_code ec) { + std::println(stderr, "app: error: {}: {}", what, ec.message()); + std::exit(1); +} + +[[noreturn]] void usage() { + std::println(stderr, "usage: app serve|naive"); + std::exit(2); +} + +// Read a whole file into a string, trailing whitespace trimmed. Empty +// string if the file cannot be opened (never fatal: cgroup files may be +// absent on unusual hosts, and the caller falls back to "unknown"). +std::string read_trim(const std::string& path) { + std::ifstream f(path); + if (!f) { + return {}; + } + std::ostringstream ss; + ss << f.rdbuf(); + std::string s = ss.str(); + while (!s.empty() && + (s.back() == '\n' || s.back() == '\r' || s.back() == ' ')) { + s.pop_back(); + } + return s; +} + +// The cgroup v2 unified path this process is itself in: /proc/self/cgroup +// has exactly one line for the unified hierarchy, "0::". Falls back +// to root if the file is missing (e.g. a non-cgroup-v2 host). +std::string own_cgroup_path() { + std::ifstream f("/proc/self/cgroup"); + std::string line; + while (f && std::getline(f, line)) { + if (line.rfind("0::", 0) == 0) { + return line.substr(3); + } + } + return "/"; +} + +// Read a cgroup v2 controller file for THIS process's own cgroup. Inside a +// container the cgroup namespace already makes "/sys/fs/cgroup" the +// container's own slice, so the root-relative fallback covers that case too. +std::string read_own_cgroup_file(std::string_view name) { + static const std::string base = "/sys/fs/cgroup" + own_cgroup_path(); + std::string v = read_trim(base + "/" + std::string(name)); + if (!v.empty()) { + return v; + } + return read_trim("/sys/fs/cgroup/" + std::string(name)); +} + +// "max" or "/", derived from cpu.max's raw +// "max 100000" / "200000 100000" content. +std::string cpu_max_display() { + const std::string raw = read_own_cgroup_file("cpu.max"); + if (raw.empty()) { + return "unknown"; + } + const auto sp = raw.find(' '); + if (sp == std::string::npos) { + return raw; + } + const std::string quota = raw.substr(0, sp); + if (quota == "max") { + return "max"; + } + return quota + "/" + raw.substr(sp + 1); +} + +std::string mem_max_display() { + const std::string raw = read_own_cgroup_file("memory.max"); + return raw.empty() ? "unknown" : raw; +} + +void print_container_line() { + const unsigned n = std::thread::hardware_concurrency(); // THE TRAP: host cpus + std::println("container: cpu.max={} effective_parallelism={} mem.max={}", + cpu_max_display(), n, mem_max_display()); +} + +// Fork-safe: only async-signal-safe calls between fork() and _exit(). +void write_line(std::string_view s) { + [[maybe_unused]] auto n = ::write(STDOUT_FILENO, s.data(), s.size()); +} + +// A short-lived "job": forks, prints one line, exits immediately. Left +// unreaped it would sit as a zombie -- reap_all() in the serve loop is what +// keeps these from piling up. +void spawn_job(int seq) { + const pid_t pid = ::fork(); + if (pid < 0) { + return; // best-effort; a failed fork here is not fatal + } + if (pid == 0) { + char buf[64]; + const int len = + std::snprintf(buf, sizeof buf, "app: job pid=%d seq=%d done\n", + ::getpid(), seq); + write_line(std::string_view(buf, static_cast(len))); + ::_exit(0); + } +} + +// The persistent "worker": the long-running child that stands in for a +// real service. It installs no handlers of its own -- an ordinary +// (non-PID-1) process, so SIGTERM's default action (terminate) applies the +// moment `serve` forwards the signal to it. +[[nodiscard]] std::expected spawn_worker() { + const pid_t pid = ::fork(); + if (pid < 0) { + return std::unexpected(errno_ec()); + } + if (pid == 0) { + // Undo the parent's SIG_BLOCK: without this the child inherits + // TERM/INT/CHLD as *blocked*, so a forwarded SIGTERM would just sit + // pending forever instead of running its default (terminate) + // action -- the worker would be unkillable. Same fix pmon (ch. 12) + // applies before execvp; here there is no exec, so it happens + // directly. + sigset_t empty; + sigemptyset(&empty); + ::sigprocmask(SIG_SETMASK, &empty, nullptr); + for (int tick = 1;; ++tick) { + ::sleep(2); + char buf[64]; + const int len = std::snprintf( + buf, sizeof buf, "app: worker pid=%d tick=%d\n", ::getpid(), tick); + write_line(std::string_view(buf, static_cast(len))); + } + } + return pid; +} + +// Reap every exited child without blocking, reporting each one. Called from +// the ordinary poll(2) loop below (never from a signal handler), so +// std::println here is fine. +void reap_all() { + for (;;) { + int status = 0; + const pid_t pid = ::waitpid(-1, &status, WNOHANG); + if (pid <= 0) { + break; + } + if (WIFSIGNALED(status)) { + std::println("app: reaped pid={} signal={}", pid, WTERMSIG(status)); + } else { + std::println("app: reaped pid={} status={}", pid, WEXITSTATUS(status)); + } + } +} + +int serve() { + print_container_line(); + std::println("app: pid={} ppid={}", ::getpid(), ::getppid()); + + // Block SIGTERM/SIGINT/SIGCHLD and receive them as signalfd(2) reads, so + // no code ever runs in real signal-handler context (the pmon pattern + // from chapter 12). + sigset_t mask; + sigemptyset(&mask); + sigaddset(&mask, SIGTERM); + sigaddset(&mask, SIGINT); + sigaddset(&mask, SIGCHLD); + if (::sigprocmask(SIG_BLOCK, &mask, nullptr) < 0) { + die("sigprocmask", errno_ec()); + } + const int sfd = ::signalfd(-1, &mask, SFD_CLOEXEC); + if (sfd < 0) { + die("signalfd", errno_ec()); + } + + const auto worker = spawn_worker(); + if (!worker) { + die("fork worker", worker.error()); + } + const pid_t worker_pid = *worker; + std::println("app: worker started pid={}", worker_pid); + + // A std::jthread drives the periodic "job" spawns: RAII-managed, and its + // stop_token lets shutdown cancel it cooperatively instead of detaching + // or leaking a thread. + std::atomic next_seq{0}; + std::jthread job_spawner([&](const std::stop_token& stok) { + while (!stok.stop_requested()) { + for (int i = 0; i < 10 && !stok.stop_requested(); ++i) { + std::this_thread::sleep_for(100ms); + } + if (stok.stop_requested()) { + break; + } + spawn_job(next_seq.fetch_add(1) + 1); + } + }); + + for (;;) { + pollfd pfd{.fd = sfd, .events = POLLIN, .revents = 0}; + const int nready = ::poll(&pfd, 1, -1); + if (nready < 0) { + if (errno == EINTR) { + continue; + } + die("poll", errno_ec()); + } + + signalfd_siginfo si{}; + const ssize_t nread = ::read(sfd, &si, sizeof si); + if (nread != static_cast(sizeof si)) { + die("read signalfd", errno_ec()); + } + + if (si.ssi_signo == static_cast(SIGCHLD)) { + reap_all(); + continue; + } + + // SIGTERM or SIGINT: this is the pid-1 duty naive skips. Stop + // spawning new jobs, forward the signal to the worker, reap it, and + // only then exit -- so nothing is left behind when the container + // stops. + job_spawner.request_stop(); + if (::kill(worker_pid, SIGTERM) == 0 || errno == ESRCH) { + int status = 0; + ::waitpid(worker_pid, &status, 0); + } + std::println("app: shutting down ({})", + si.ssi_signo == static_cast(SIGTERM) ? "SIGTERM" + : "SIGINT"); + return 0; + } +} + +// THE OTHER HALF OF THE TRAP: naive installs no signal handling whatsoever. +// As this container's PID 1, the kernel will not run SIGTERM's default +// action (terminate) for it -- there is no handler, so the signal is simply +// dropped. `podman stop` has no way to ask it to leave; only SIGKILL ends it. +int naive() { + print_container_line(); + std::println("app: pid={} ppid={}", ::getpid(), ::getppid()); + for (int tick = 1;; ++tick) { + std::this_thread::sleep_for(1s); + std::println("app: naive heartbeat tick={}", tick); + } +} + +} // namespace + +int main(int argc, char** argv) { + std::setvbuf(stdout, nullptr, _IOLBF, 0); // line-buffered even into a pipe + + if (argc != 2) { + usage(); + } + const std::string_view sub = argv[1]; + if (sub == "serve") { + return serve(); + } + if (sub == "naive") { + return naive(); + } + usage(); +} diff --git a/examples/34-our-programs-in-containers/demo.sh b/examples/34-our-programs-in-containers/demo.sh new file mode 100755 index 0000000..afbbe4c --- /dev/null +++ b/examples/34-our-programs-in-containers/demo.sh @@ -0,0 +1,23 @@ +#!/usr/bin/env bash +set -euo pipefail +cd "$(dirname "$0")" + +langs=(cpp go rust) + +case "${1:-all}" in + cpp|go|rust) + lang="$1" + shift + exec "./$lang/demo.sh" "$@" + ;; + all) + for lang in "${langs[@]}"; do "./$lang/demo.sh"; done + ;; + build) + for lang in "${langs[@]}"; do "./$lang/demo.sh" build; done + ;; + *) + echo "usage: $0 [cpp|go|rust|all|build] [args...]" >&2 + exit 2 + ;; +esac diff --git a/examples/34-our-programs-in-containers/go/Containerfile b/examples/34-our-programs-in-containers/go/Containerfile new file mode 100644 index 0000000..05d1e8d --- /dev/null +++ b/examples/34-our-programs-in-containers/go/Containerfile @@ -0,0 +1,19 @@ +# app (Go 1.26) — multi-stage: UBI10 builder -> UBI10-micro runtime. +# +# podman build -t lsp34-go -f Containerfile . +# podman run --rm --cpus=2 --memory=128m lsp34-go serve +# +# CGO_ENABLED=0 gives a fully static binary, so the runtime stage needs +# nothing at all beyond the binary itself. + +FROM registry.access.redhat.com/ubi10/ubi:10.2 AS builder +RUN dnf -y install golang && dnf clean all +WORKDIR /src +COPY go.mod ./ +COPY main.go ./ +RUN CGO_ENABLED=0 go build -o /out/app . + +FROM registry.access.redhat.com/ubi10-micro:10.2 AS runtime +COPY --from=builder /out/app /usr/local/bin/app +ENTRYPOINT ["/usr/local/bin/app"] +CMD ["serve"] diff --git a/examples/34-our-programs-in-containers/go/demo.sh b/examples/34-our-programs-in-containers/go/demo.sh new file mode 100755 index 0000000..28606df --- /dev/null +++ b/examples/34-our-programs-in-containers/go/demo.sh @@ -0,0 +1,23 @@ +#!/usr/bin/env bash +set -euo pipefail +cd "$(dirname "$0")" +REPO_ROOT="$(cd ../../.. && pwd)" +BIN=bin/app + +build() { + go build -o "$BIN" . +} + +run() { + if [[ -n "${TARGET:-}" ]]; then + exec "$REPO_ROOT/scripts/lab/deploy-to-vm.sh" "$TARGET" "$BIN" -- "$@" + fi + exec "./$BIN" "$@" +} + +case "${1:-}" in + build) build ;; + run) shift; run "$@" ;; + "") build; run ;; + *) echo "usage: $0 [build|run [args...]]" >&2; exit 2 ;; +esac diff --git a/examples/34-our-programs-in-containers/go/go.mod b/examples/34-our-programs-in-containers/go/go.mod new file mode 100644 index 0000000..22730b2 --- /dev/null +++ b/examples/34-our-programs-in-containers/go/go.mod @@ -0,0 +1,5 @@ +module github.com/patterncatalyst/linux-systems-programming/examples/34-our-programs-in-containers/go + +go 1.26 + +toolchain go1.26.5 diff --git a/examples/34-our-programs-in-containers/go/go.sum b/examples/34-our-programs-in-containers/go/go.sum new file mode 100644 index 0000000..e69de29 diff --git a/examples/34-our-programs-in-containers/go/main.go b/examples/34-our-programs-in-containers/go/main.go new file mode 100644 index 0000000..edcc1a6 --- /dev/null +++ b/examples/34-our-programs-in-containers/go/main.go @@ -0,0 +1,243 @@ +// app — a tiny PID-1 container entrypoint (chapter 34: our programs in +// containers). +// +// app serve run as the container's PID 1: reap children, forward SIGTERM +// to the supervised worker, report the REAL container limits. +// app naive the trap: no signal handling installed at all. A process +// that is PID 1 of its own PID namespace (i.e. the container's +// entrypoint) is special-cased by the kernel: any signal for +// which it has installed no explicit handler is dropped +// instead of running that signal's default action. Only +// SIGKILL (and SIGSTOP) still work. `naive` never calls +// signal.Notify, so `podman stop` cannot end it gracefully -- +// the engine has to wait out the stop timeout and fall back to +// SIGKILL. +// app worker the long-running child `serve` supervises (self-reexec). +// app job a short-lived child `serve` spawns periodically (self-reexec). +// +// Reaping, the Go way: unlike the C++ and Rust versions there is no +// SIGCHLD/signalfd anywhere here. os/exec's Cmd.Wait() calls wait4(2) for +// you; parking one goroutine per spawned child in Wait() *is* the reap loop. +// +// Container-aware resource detection: runtime.GOMAXPROCS(0), as of the +// go.mod "go 1.26" language version, defaults to the average cgroup CPU +// throughput limit (cpu.max quota/period) when it is lower than the host's +// cpu count -- this is the one language of the three that gets this right +// automatically. Compare this program's number against the C++ version's +// (which reports the host's cpu count no matter what `--cpus` says). +package main + +import ( + "bufio" + "fmt" + "os" + "os/exec" + "os/signal" + "runtime" + "strconv" + "strings" + "syscall" + "time" +) + +func usage() { + fmt.Fprintln(os.Stderr, "usage: app serve|naive|worker|job") + os.Exit(2) +} + +func die(err error) { + fmt.Fprintf(os.Stderr, "app: error: %v\n", err) + os.Exit(1) +} + +// readTrim reads a whole file, trailing whitespace trimmed. Empty string +// (not an error) if the file cannot be opened -- cgroup files may be absent +// on unusual hosts, and callers fall back to "unknown". +func readTrim(path string) string { + b, err := os.ReadFile(path) + if err != nil { + return "" + } + return strings.TrimRight(string(b), "\n\r ") +} + +// ownCgroupPath is the cgroup v2 unified path this process is itself in: +// /proc/self/cgroup has exactly one "0::" line. Falls back to root if +// unreadable (e.g. a non-cgroup-v2 host). +func ownCgroupPath() string { + f, err := os.Open("/proc/self/cgroup") + if err != nil { + return "/" + } + defer f.Close() + sc := bufio.NewScanner(f) + for sc.Scan() { + if line := sc.Text(); strings.HasPrefix(line, "0::") { + return strings.TrimPrefix(line, "0::") + } + } + return "/" +} + +// readOwnCgroupFile reads a cgroup v2 controller file for THIS process's own +// cgroup. Inside a container the cgroup namespace already makes +// "/sys/fs/cgroup" the container's own slice, so the root-relative fallback +// covers that case too. +func readOwnCgroupFile(name string) string { + base := "/sys/fs/cgroup" + ownCgroupPath() + if v := readTrim(base + "/" + name); v != "" { + return v + } + return readTrim("/sys/fs/cgroup/" + name) +} + +// cpuMaxDisplay returns "max" or "/", derived from cpu.max's +// raw "max 100000" / "200000 100000" content. +func cpuMaxDisplay() string { + raw := readOwnCgroupFile("cpu.max") + if raw == "" { + return "unknown" + } + quota, period, ok := strings.Cut(raw, " ") + if !ok { + return raw + } + if quota == "max" { + return "max" + } + return quota + "/" + period +} + +func memMaxDisplay() string { + if v := readOwnCgroupFile("memory.max"); v != "" { + return v + } + return "unknown" +} + +func printContainerLine() { + fmt.Printf("container: cpu.max=%s effective_parallelism=%d mem.max=%s\n", + cpuMaxDisplay(), runtime.GOMAXPROCS(0), memMaxDisplay()) +} + +func workerRun() int { + for tick := 1; ; tick++ { + time.Sleep(2 * time.Second) + fmt.Printf("app: worker pid=%d tick=%d\n", os.Getpid(), tick) + } +} + +func jobRun(seq string) int { + fmt.Printf("app: job pid=%d seq=%s done\n", os.Getpid(), seq) + return 0 +} + +// selfPath resolves the running binary's own path so serve can re-exec it as +// "worker"/"job" -- there is no fork(2)-without-exec in Go, so this is the +// idiomatic way to spawn "more of this program" as a real child process. +func selfPath() string { + p, err := os.Executable() + if err != nil { + die(fmt.Errorf("resolve self path: %w", err)) + } + return p +} + +func serve() int { + printContainerLine() + fmt.Printf("app: pid=%d ppid=%d\n", os.Getpid(), os.Getppid()) + + sigCh := make(chan os.Signal, 4) + signal.Notify(sigCh, syscall.SIGTERM, syscall.SIGINT) + + self := selfPath() + + worker := exec.Command(self, "worker") + worker.Stdout, worker.Stderr = os.Stdout, os.Stderr + if err := worker.Start(); err != nil { + die(fmt.Errorf("start worker: %w", err)) + } + fmt.Printf("app: worker started pid=%d\n", worker.Process.Pid) + workerDone := make(chan struct{}) + go func() { + _ = worker.Wait() // reaps the worker when it eventually exits + close(workerDone) + }() + + // One goroutine parked in Wait() per spawned job: that goroutine IS the + // reaper for that job, no SIGCHLD involved. + seq := 0 + spawnJob := func() { + seq++ + job := exec.Command(self, "job", strconv.Itoa(seq)) + job.Stdout, job.Stderr = os.Stdout, os.Stderr + if err := job.Start(); err != nil { + fmt.Printf("app: job spawn failed: %v\n", err) + return + } + go func() { + _ = job.Wait() + status := job.ProcessState.ExitCode() + fmt.Printf("app: reaped pid=%d status=%d\n", job.Process.Pid, status) + }() + } + + ticker := time.NewTicker(time.Second) + defer ticker.Stop() + + for { + select { + case s := <-sigCh: + // This is the pid-1 duty naive skips: stop spawning new jobs, + // forward the signal to the worker, wait for it, and only then + // exit -- so nothing is left behind when the container stops. + ticker.Stop() + name := "SIGINT" + if s == syscall.SIGTERM { + name = "SIGTERM" + } + if err := worker.Process.Signal(syscall.SIGTERM); err == nil { + <-workerDone + } + fmt.Printf("app: shutting down (%s)\n", name) + return 0 + case <-ticker.C: + spawnJob() + } + } +} + +// THE OTHER HALF OF THE TRAP: naive installs no signal handling whatsoever. +// As this container's PID 1, the kernel will not run SIGTERM's default +// action (terminate) for it -- there is no handler, so the signal is simply +// dropped. `podman stop` has no way to ask it to leave; only SIGKILL ends it. +func naiveRun() int { + printContainerLine() + fmt.Printf("app: pid=%d ppid=%d\n", os.Getpid(), os.Getppid()) + for tick := 1; ; tick++ { + time.Sleep(time.Second) + fmt.Printf("app: naive heartbeat tick=%d\n", tick) + } +} + +func main() { + args := os.Args[1:] + if len(args) < 1 { + usage() + } + switch args[0] { + case "serve": + os.Exit(serve()) + case "naive": + os.Exit(naiveRun()) + case "worker": + os.Exit(workerRun()) + case "job": + if len(args) < 2 { + usage() + } + os.Exit(jobRun(args[1])) + default: + usage() + } +} diff --git a/examples/34-our-programs-in-containers/rust/Cargo.lock b/examples/34-our-programs-in-containers/rust/Cargo.lock new file mode 100644 index 0000000..3ca67ce --- /dev/null +++ b/examples/34-our-programs-in-containers/rust/Cargo.lock @@ -0,0 +1,99 @@ +# This file is automatically @generated by Cargo. +# It is not intended for manual editing. +version = 4 + +[[package]] +name = "anyhow" +version = "1.0.104" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "330a5ed07fa54e4702c9d6c4174f74427fc0ef6e214bbd677ae50a5099946470" + +[[package]] +name = "bitflags" +version = "2.13.1" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "b588b76d00fde79687d7646a9b5bdf3cc0f655e0bbd080335a95d7e96f3587da" + +[[package]] +name = "cfg-if" +version = "1.0.4" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "9330f8b2ff13f34540b44e946ef35111825727b38d33286ef986142615121801" + +[[package]] +name = "cfg_aliases" +version = "0.2.2" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "f079e83a288787bcd14a6aea84cee5c87a67c5a3e660c30f557a3d24761b3527" + +[[package]] +name = "containers-app" +version = "0.1.0" +dependencies = [ + "anyhow", + "nix", + "signal-hook", +] + +[[package]] +name = "errno" +version = "0.3.14" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "39cab71617ae0d63f51a36d69f866391735b51691dbda63cf6f96d042b63efeb" +dependencies = [ + "libc", + "windows-sys", +] + +[[package]] +name = "libc" +version = "0.2.186" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "68ab91017fe16c622486840e4c83c9a37afeff978bd239b5293d61ece587de66" + +[[package]] +name = "nix" +version = "0.31.3" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "cf20d2fde8ff38632c426f1165ed7436270b44f199fc55284c38276f9db47c3d" +dependencies = [ + "bitflags", + "cfg-if", + "cfg_aliases", + "libc", +] + +[[package]] +name = "signal-hook" +version = "0.4.4" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "b2a0c28ca5908dbdbcd52e6fdaa00358ab88637f8ab33e1f188dd510eb44b53d" +dependencies = [ + "libc", + "signal-hook-registry", +] + +[[package]] +name = "signal-hook-registry" +version = "1.4.8" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "c4db69cba1110affc0e9f7bcd48bbf87b3f4fc7c61fc9155afd4c469eb3d6c1b" +dependencies = [ + "errno", + "libc", +] + +[[package]] +name = "windows-link" +version = "0.2.1" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "f0805222e57f7521d6a62e36fa9163bc891acd422f971defe97d64e70d0a4fe5" + +[[package]] +name = "windows-sys" +version = "0.61.2" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "ae137229bcbd6cdf0f7b80a31df61766145077ddf49416a728b02cb3921ff3fc" +dependencies = [ + "windows-link", +] diff --git a/examples/34-our-programs-in-containers/rust/Cargo.toml b/examples/34-our-programs-in-containers/rust/Cargo.toml new file mode 100644 index 0000000..f03a69e --- /dev/null +++ b/examples/34-our-programs-in-containers/rust/Cargo.toml @@ -0,0 +1,13 @@ +[package] +name = "containers-app" +version = "0.1.0" +edition = "2024" + +[[bin]] +name = "app" +path = "src/main.rs" + +[dependencies] +anyhow = "1" +nix = { version = "0.31", features = ["signal"] } +signal-hook = { version = "0.4", features = ["iterator"] } diff --git a/examples/34-our-programs-in-containers/rust/Containerfile b/examples/34-our-programs-in-containers/rust/Containerfile new file mode 100644 index 0000000..a061061 --- /dev/null +++ b/examples/34-our-programs-in-containers/rust/Containerfile @@ -0,0 +1,26 @@ +# app (Rust edition 2024) — multi-stage: UBI10 builder -> UBI10-micro runtime. +# +# podman build -t lsp34-rust -f Containerfile . +# podman run --rm --cpus=2 --memory=128m lsp34-rust serve +# +# UBI10's dnf rust package (1.92.0) predates this book's pinned 1.97.1, so +# the builder installs the exact toolchain via rustup instead, matching +# rust-toolchain.toml. The resulting binary only needs libc + libgcc_s, both +# already present in ubi10-micro. + +FROM registry.access.redhat.com/ubi10/ubi:10.2 AS builder +RUN dnf -y install gcc-c++ curl && dnf clean all +COPY rust-toolchain.toml /tmp/rust-toolchain.toml +RUN curl --proto '=https' --tlsv1.2 -sSf https://sh.rustup.rs -o /tmp/rustup.sh \ + && sh /tmp/rustup.sh -y --profile minimal \ + --default-toolchain "$(sed -n 's/^channel *= *"\(.*\)"/\1/p' /tmp/rust-toolchain.toml)" +ENV PATH="/root/.cargo/bin:${PATH}" +WORKDIR /src +COPY Cargo.toml Cargo.lock ./ +COPY src ./src +RUN cargo build --release --locked + +FROM registry.access.redhat.com/ubi10-micro:10.2 AS runtime +COPY --from=builder /src/target/release/app /usr/local/bin/app +ENTRYPOINT ["/usr/local/bin/app"] +CMD ["serve"] diff --git a/examples/34-our-programs-in-containers/rust/demo.sh b/examples/34-our-programs-in-containers/rust/demo.sh new file mode 100755 index 0000000..220feea --- /dev/null +++ b/examples/34-our-programs-in-containers/rust/demo.sh @@ -0,0 +1,26 @@ +#!/usr/bin/env bash +set -euo pipefail +cd "$(dirname "${BASH_SOURCE[0]}")" + +BIN=target/release/app + +build() { + cargo build --release +} + +run() { + if [[ -n "${TARGET:-}" ]]; then + local repo_root + repo_root=$(cd ../../.. && pwd) + "$repo_root/scripts/lab/deploy-to-vm.sh" "$TARGET" "$BIN" -- "$@" + else + "./$BIN" "$@" + fi +} + +case "${1:-}" in + build) build ;; + run) shift; run "$@" ;; + "") build; run ;; + *) echo "usage: $0 [build|run [args...]]" >&2; exit 2 ;; +esac diff --git a/examples/34-our-programs-in-containers/rust/rust-toolchain.toml b/examples/34-our-programs-in-containers/rust/rust-toolchain.toml new file mode 100644 index 0000000..4099f1a --- /dev/null +++ b/examples/34-our-programs-in-containers/rust/rust-toolchain.toml @@ -0,0 +1,3 @@ +[toolchain] +channel = "1.97.1" +components = ["rustfmt", "clippy", "rust-src"] diff --git a/examples/34-our-programs-in-containers/rust/src/main.rs b/examples/34-our-programs-in-containers/rust/src/main.rs new file mode 100644 index 0000000..71520a6 --- /dev/null +++ b/examples/34-our-programs-in-containers/rust/src/main.rs @@ -0,0 +1,243 @@ +// app — a tiny PID-1 container entrypoint (chapter 34: our programs in +// containers). +// +// app serve run as the container's PID 1: reap children, forward SIGTERM +// to the supervised worker, report the REAL container limits. +// app naive the trap: no signal handling installed at all. A process +// that is PID 1 of its own PID namespace (i.e. the container's +// entrypoint) is special-cased by the kernel: any signal for +// which it has installed no explicit handler is dropped +// instead of running that signal's default action. Only +// SIGKILL (and SIGSTOP) still work. `naive` never registers a +// handler, so `podman stop` cannot end it gracefully -- the +// engine has to wait out the stop timeout and fall back to +// SIGKILL. +// app worker the long-running child `serve` supervises (self-reexec). +// app job a short-lived child `serve` spawns periodically (self-reexec). +// +// Reaping, the Rust way: like the Go version (and unlike the C++ one), there +// is no signalfd/SIGCHLD anywhere here. std::process::Child::wait() calls +// waitpid(2) for you; parking one std::thread per spawned child in wait() is +// the reap loop. signal_hook::iterator::Signals turns SIGTERM/SIGINT into a +// blocking iterator on the main thread -- ordinary code, nothing async- +// signal-unsafe runs in a handler. +// +// Container-aware resource detection: std::thread::available_parallelism() +// on Linux divides the cgroup's cpu.max quota/period when it is lower than +// the affinity mask count -- cgroup-aware, like Go's GOMAXPROCS. Compare +// this program's number against the C++ version's hardware_concurrency() +// (which reports the host's cpu count no matter what `--cpus` says). + +use std::fs; +use std::io::Read; +use std::process::{Child, Command}; +use std::sync::atomic::{AtomicBool, Ordering}; +use std::sync::Arc; +use std::thread; +use std::time::Duration; + +use anyhow::{Context, Result}; +use nix::sys::signal::{Signal, kill}; +use nix::unistd::Pid; +use signal_hook::consts::{SIGINT, SIGTERM}; +use signal_hook::iterator::Signals; + +fn usage() -> ! { + eprintln!("usage: app serve|naive|worker|job"); + std::process::exit(2); +} + +fn die(err: anyhow::Error) -> ! { + eprintln!("app: error: {err:#}"); + std::process::exit(1); +} + +/// Read a whole file, trailing whitespace trimmed. Empty string (not an +/// error) if the file cannot be opened -- cgroup files may be absent on +/// unusual hosts, and callers fall back to "unknown". +fn read_trim(path: &str) -> String { + let mut buf = String::new(); + match fs::File::open(path).and_then(|mut f| f.read_to_string(&mut buf)) { + Ok(_) => buf.trim_end_matches(['\n', '\r', ' ']).to_string(), + Err(_) => String::new(), + } +} + +/// The cgroup v2 unified path this process is itself in: /proc/self/cgroup +/// has exactly one "0::" line. Falls back to root if unreadable (e.g. +/// a non-cgroup-v2 host). +fn own_cgroup_path() -> String { + let content = read_trim("/proc/self/cgroup"); + for line in content.lines() { + if let Some(path) = line.strip_prefix("0::") { + return path.to_string(); + } + } + "/".to_string() +} + +/// Read a cgroup v2 controller file for THIS process's own cgroup. Inside a +/// container the cgroup namespace already makes "/sys/fs/cgroup" the +/// container's own slice, so the root-relative fallback covers that case +/// too. +fn read_own_cgroup_file(name: &str) -> String { + let base = format!("/sys/fs/cgroup{}", own_cgroup_path()); + let v = read_trim(&format!("{base}/{name}")); + if !v.is_empty() { + return v; + } + read_trim(&format!("/sys/fs/cgroup/{name}")) +} + +/// "max" or "/", derived from cpu.max's raw +/// "max 100000" / "200000 100000" content. +fn cpu_max_display() -> String { + let raw = read_own_cgroup_file("cpu.max"); + if raw.is_empty() { + return "unknown".to_string(); + } + match raw.split_once(' ') { + None => raw, + Some(("max", _)) => "max".to_string(), + Some((quota, period)) => format!("{quota}/{period}"), + } +} + +fn mem_max_display() -> String { + let raw = read_own_cgroup_file("memory.max"); + if raw.is_empty() { "unknown".to_string() } else { raw } +} + +fn print_container_line() { + // Cgroup-aware, unlike the C++ version's hardware_concurrency(). + let n = std::thread::available_parallelism().map(|n| n.get()).unwrap_or(0); + println!( + "container: cpu.max={} effective_parallelism={} mem.max={}", + cpu_max_display(), + n, + mem_max_display() + ); +} + +fn worker_run() -> i32 { + let mut tick = 1u64; + loop { + thread::sleep(Duration::from_secs(2)); + println!("app: worker pid={} tick={tick}", std::process::id()); + tick += 1; + } +} + +fn job_run(seq: &str) -> i32 { + println!("app: job pid={} seq={seq} done", std::process::id()); + 0 +} + +/// Resolve the running binary's own path so serve can re-exec it as +/// "worker"/"job" -- like Go, Rust avoids fork(2)-without-exec (the +/// allocator and runtime are not fork-safe once threads exist), so this is +/// the idiomatic way to spawn "more of this program" as a real child +/// process. +fn self_path() -> Result { + std::env::current_exe().context("resolve self path") +} + +fn serve() -> Result { + print_container_line(); + println!("app: pid={} ppid={}", std::process::id(), nix::unistd::getppid()); + + let mut signals = Signals::new([SIGTERM, SIGINT]).context("Signals::new")?; + let self_exe = self_path()?; + + let mut worker: Child = Command::new(&self_exe) + .arg("worker") + .spawn() + .context("start worker")?; + let worker_pid = worker.id(); + println!("app: worker started pid={worker_pid}"); + + // The job spawner: a background thread that periodically re-execs + // "app job " and, in its own thread per child, waits for it -- + // that wait() call IS the reap. `stop` lets shutdown cancel the loop + // cooperatively instead of detaching a thread that outlives serve(). + let stop = Arc::new(AtomicBool::new(false)); + let job_spawner = { + let stop = Arc::clone(&stop); + let self_exe = self_exe.clone(); + thread::spawn(move || { + let mut seq = 0u64; + while !stop.load(Ordering::SeqCst) { + for _ in 0..10 { + if stop.load(Ordering::SeqCst) { + return; + } + thread::sleep(Duration::from_millis(100)); + } + seq += 1; + match Command::new(&self_exe).args(["job", &seq.to_string()]).spawn() { + Ok(mut child) => { + thread::spawn(move || { + let pid = child.id(); + match child.wait() { + Ok(status) => println!( + "app: reaped pid={pid} status={}", + status.code().unwrap_or(-1) + ), + Err(e) => println!("app: reap failed pid={pid}: {e}"), + } + }); + } + Err(e) => println!("app: job spawn failed: {e}"), + } + } + }) + }; + + // Block on the signal iterator: this is the pid-1 duty naive skips. + let sig = signals.forever().next().expect("signal stream ended"); + stop.store(true, Ordering::SeqCst); + let _ = job_spawner.join(); + if kill(Pid::from_raw(worker_pid as i32), Signal::SIGTERM).is_ok() { + let _ = worker.wait(); + } + println!( + "app: shutting down ({})", + if sig == SIGTERM { "SIGTERM" } else { "SIGINT" } + ); + Ok(0) +} + +// THE OTHER HALF OF THE TRAP: naive installs no signal handling whatsoever. +// As this container's PID 1, the kernel will not run SIGTERM's default +// action (terminate) for it -- there is no handler, so the signal is simply +// dropped. `podman stop` has no way to ask it to leave; only SIGKILL ends it. +fn naive_run() -> i32 { + print_container_line(); + println!("app: pid={} ppid={}", std::process::id(), nix::unistd::getppid()); + let mut tick = 1u64; + loop { + thread::sleep(Duration::from_secs(1)); + println!("app: naive heartbeat tick={tick}"); + tick += 1; + } +} + +fn main() { + let args: Vec = std::env::args().skip(1).collect(); + if args.is_empty() { + usage(); + } + let code = match args[0].as_str() { + "serve" => serve().unwrap_or_else(|e| die(e)), + "naive" => naive_run(), + "worker" => worker_run(), + "job" => { + if args.len() < 2 { + usage(); + } + job_run(&args[1]) + } + _ => usage(), + }; + std::process::exit(code); +} diff --git a/examples/34-our-programs-in-containers/verify.lua b/examples/34-our-programs-in-containers/verify.lua new file mode 100644 index 0000000..e8730a2 --- /dev/null +++ b/examples/34-our-programs-in-containers/verify.lua @@ -0,0 +1,195 @@ +-- Verify 34-our-programs-in-containers for LSP_LANG. +-- +-- This example is LOCAL but requires podman: it builds the language's +-- Containerfile, runs the image rootless under real --cpus/--memory limits, +-- and asserts observable effects, not merely exit 0: +-- +-- 1. container-aware resource detection: the CONSTRAINED cpu.max/mem.max +-- are reported, not the host's -- and specifically that C++'s +-- effective_parallelism is the HOST cpu count (hardware_concurrency's +-- trap) while Go's and Rust's match the cgroup quota. +-- 2. this process really is PID 1 of its own namespace (pid=1 ppid=0, +-- /proc/1/status inside the container), and no zombie accumulates +-- while jobs are spawned and reaped. +-- 3. debugging across namespaces: podman exec AND a rootless +-- `podman unshare nsenter` from the host both see the same PID 1. +-- 4. pid-1 signal duties: `serve` reacts to SIGTERM almost instantly and +-- prints a graceful shutdown line; `naive` never prints that line +-- (proving no graceful path ran) and -- for C++/Rust, whose default +-- SIGTERM disposition is masked out by the kernel's PID-1 special +-- case -- takes the full stop grace period and is SIGKILLed. Go's +-- runtime is a real, verified exception: it installs its own handler +-- for SIGTERM at startup regardless of signal.Notify, so a "naive" Go +-- binary still dies promptly -- just abruptly, with no cleanup. +-- +-- Invoked from the example directory with LSP_LANG (cpp|go|rust) and +-- EXAMPLE_DIR set; the interpreter may be lua or luajit. Images and +-- containers are removed unconditionally at the end. + +local script_dir = arg[0]:match("(.*/)") or "./" +local repo_root = os.getenv("REPO_ROOT") or (script_dir .. "../..") +package.path = package.path .. ";" .. repo_root .. "/scripts/lib/?.lua" +local checks = require("checks") + +local lang = os.getenv("LSP_LANG") +if lang ~= "cpp" and lang ~= "go" and lang ~= "rust" then + checks.skip("LSP_LANG must be cpp|go|rust (got " .. tostring(lang) .. ")") +end + +if checks.run("command -v podman").exit ~= 0 then + checks.skip("podman not available on this host") +end + +local image = "lsp34-" .. lang +local serve_c = "lsp34-" .. lang .. "-serve-verify" +local naive_c = "lsp34-" .. lang .. "-naive-verify" + +-- Cleanup helper: safe to call at any point, never fails the run. +local function cleanup() + checks.run("podman rm -f " .. serve_c .. " " .. naive_c .. " >/dev/null 2>&1; true") +end +cleanup() -- in case a previous run left something behind + +-- --------------------------------------------------------------------------- +-- 0. host shape: the cpu-vs-host contrast needs a host with more than the +-- 2 cpus this test constrains the container to, or C++'s "wrong" number +-- would coincidentally match the "right" one. +-- --------------------------------------------------------------------------- + +local nproc = checks.run("nproc") +local host_nproc = tonumber((nproc.out:gsub("%s+", ""))) +if not host_nproc or host_nproc < 3 then + checks.skip("host has only " .. tostring(host_nproc) .. + " cpu(s); need >= 3 to demonstrate the cgroup-vs-host mismatch at --cpus=2") +end + +-- --------------------------------------------------------------------------- +-- 1. build the image +-- --------------------------------------------------------------------------- + +local build = checks.run("podman build -t " .. image .. " -f " .. lang .. "/Containerfile " .. lang) +checks.expect_exit(build, 0, lang .. ": podman build succeeds") + +-- --------------------------------------------------------------------------- +-- 2. serve, constrained: resource detection, pid-1 identity, reaping +-- --------------------------------------------------------------------------- + +local run_serve = checks.run( + "podman run -d --name " .. serve_c .. " --cpus=2 --memory=128m " .. image .. " serve") +checks.expect_exit(run_serve, 0, lang .. ": serve container starts") + +-- Let a handful of job-spawn/reap cycles happen before reading logs. +checks.run("sleep 5") +local logs = checks.run("podman logs " .. serve_c) + +checks.expect_match(logs.out, "container: cpu%.max=200000/100000", + lang .. ": reports the CONSTRAINED cpu.max (--cpus=2), not the host's") +checks.expect_match(logs.out, "mem%.max=134217728", + lang .. ": reports the CONSTRAINED mem.max (--memory=128m), not the host's") +checks.expect_match(logs.out, "app: pid=1 ppid=0", + lang .. ": this process really is PID 1 of its own namespace (ppid=0)") +checks.expect_match(logs.out, "app: worker started pid=%d+", + lang .. ": serve starts the supervised worker") + +local _, reap_count = logs.out:gsub("app: reaped pid=%d+ status=0", "") +checks.expect_match("reaps=" .. reap_count, "^reaps=[2-9]%d*$", + lang .. ": several jobs were spawned and reaped (no pile-up)") + +if lang == "cpp" then + checks.expect_match(logs.out, "effective_parallelism=" .. host_nproc, + "cpp: hardware_concurrency() reports the HOST cpu count (" .. host_nproc .. + ") even though the container is capped at 2 -- the trap") +else + checks.expect_match(logs.out, "effective_parallelism=2", + lang .. ": reports the cgroup-constrained parallelism (2), not the host's " .. + host_nproc) +end + +-- --------------------------------------------------------------------------- +-- 3. debugging across namespaces: podman exec + rootless podman-unshare nsenter +-- --------------------------------------------------------------------------- + +local status = checks.run("podman exec " .. serve_c .. " cat /proc/1/status") +checks.expect_match(status.out, "Name:%s*app", + lang .. ": podman exec sees this container's PID 1 as `app`") + +local zscript = table.concat({ + "podman exec " .. serve_c .. " bash -c '", + "z=0; for d in /proc/[0-9]*; do", + " line=$(cat \"$d/stat\" 2>/dev/null) || continue;", + " rest=${line##*) }; state=${rest%% *};", + ' [[ "$state" == "Z" ]] && z=$((z+1));', + " done; echo zombies=$z'", +}, "\n") +local zombies = checks.run(zscript) +checks.expect_match(zombies.out, "zombies=0", + lang .. ": no zombie processes accumulate inside the container") + +local hostpid_r = checks.run("podman inspect --format '{{.State.Pid}}' " .. serve_c) +local hostpid = (hostpid_r.out:gsub("%s+", "")) +local nsenter = checks.run( + "podman unshare nsenter --target " .. hostpid .. " --pid --mount -- readlink /proc/1/exe") +checks.expect_match(nsenter.out, "/usr/local/bin/app", + lang .. ": rootless `podman unshare nsenter` from the HOST sees the same PID 1") + +-- --------------------------------------------------------------------------- +-- 4a. serve: SIGTERM stops it almost instantly, with a graceful log line +-- --------------------------------------------------------------------------- + +local stop_serve = checks.run(table.concat({ + "t0=$(date +%s%N)", + "podman stop -t 3 " .. serve_c .. " >/dev/null", + "t1=$(date +%s%N)", + "echo elapsed_ms=$(( (t1 - t0) / 1000000 ))", +}, "; ")) +local ms_serve = tonumber(stop_serve.out:match("elapsed_ms=(%d+)")) +checks.expect_match("fast=" .. tostring(ms_serve ~= nil and ms_serve < 1500), + "^fast=true$", lang .. ": serve honors SIGTERM well within the 3s grace (" .. + tostring(ms_serve) .. "ms)") +local final_serve_logs = checks.run("podman logs " .. serve_c) +checks.expect_match(final_serve_logs.out, "app: shutting down %(SIGTERM%)", + lang .. ": serve prints its graceful shutdown line") + +-- --------------------------------------------------------------------------- +-- 4b. naive: no graceful path ever runs. C++/Rust additionally hang out the +-- full grace period (PID-1 default-action masking); Go's runtime kills +-- it promptly anyway (see the top-of-file note) but just as ungracefully. +-- --------------------------------------------------------------------------- + +local run_naive = checks.run("podman run -d --name " .. naive_c .. " " .. image .. " naive") +checks.expect_exit(run_naive, 0, lang .. ": naive container starts") +checks.run("sleep 1") + +local stop_naive = checks.run(table.concat({ + "t0=$(date +%s%N)", + "podman stop -t 2 " .. naive_c .. " >/dev/null 2>/dev/null", + "t1=$(date +%s%N)", + "echo elapsed_ms=$(( (t1 - t0) / 1000000 ))", +}, "; ")) +local ms_naive = tonumber(stop_naive.out:match("elapsed_ms=(%d+)")) + +if lang == "go" then + checks.expect_match("fast=" .. tostring(ms_naive ~= nil and ms_naive < 1000), + "^fast=true$", + "go: even naive dies promptly on SIGTERM -- the runtime installs its own " .. + "default handler regardless of signal.Notify (" .. tostring(ms_naive) .. "ms)") +else + checks.expect_match("slow=" .. tostring(ms_naive ~= nil and ms_naive >= 1700), + "^slow=true$", + lang .. ": naive ignores SIGTERM as PID 1 -- podman waits out the grace " .. + "period and falls back to SIGKILL (" .. tostring(ms_naive) .. "ms)") +end + +local naive_logs = checks.run("podman logs " .. naive_c .. " 2>&1; true") +checks.expect_match("shutdown_lines=" .. select(2, naive_logs.out:gsub("shutting down", "")), + "^shutdown_lines=0$", + lang .. ": naive never runs the graceful shutdown path (no cleanup, no message)") + +-- --------------------------------------------------------------------------- +-- cleanup: remove containers and the built image +-- --------------------------------------------------------------------------- + +cleanup() +checks.run("podman rmi -f " .. image .. " >/dev/null 2>&1; true") + +checks.finish() diff --git a/examples/35-virtualization-and-kvm/README.md b/examples/35-virtualization-and-kvm/README.md new file mode 100644 index 0000000..b968195 --- /dev/null +++ b/examples/35-virtualization-and-kvm/README.md @@ -0,0 +1,136 @@ +# 35-virtualization-and-kvm — overheadbench + +Chapter 35's example: `overheadbench`, three tiny host-local microbenchmarks +implemented identically in C++23, Go, and Rust. The example itself only +measures **wherever it is run** — it makes no VM/container comparison on its +own. The chapter's point is made by running the *same binary* in three +places (bare host, a lab KVM guest, and a rootless podman container) and +tabulating the numbers side by side. + +## The program + +``` +overheadbench [--bench syscall|mem|io|all] [--iters N] +``` + +Runs one or more of three benches and prints one line per bench: + +``` +bench=syscall metric=91.85 unit=ns/call +bench=mem metric=14.03 unit=GB/s +bench=io metric=1480.21 unit=ops/s +``` + +- **`syscall`** — a tight `getpid(2)` loop (a raw syscall each iteration, not + a libc-cached value). Reports **ns/call**: mean wall time per syscall. + This is the number the chapter contrasts against a VM's vmexit-trap cost + and a container's near-zero added cost (containers share the host kernel; + a syscall inside one *is* a host syscall). +- **`mem`** — allocates a 128 MiB buffer of `u64` words and reads it + sequentially, summing every word, for a configurable number of passes. + Reports **GB/s**: total bytes read across all passes divided by elapsed + time. The sum is consumed after the clock stops so the compiler can't + prove the scan is dead code and delete it. +- **`io`** — repeats a create → write(4096 bytes) → `fsync(2)` → unlink + cycle. Reports **ops/s**: complete cycles per second. The scratch + directory is created **relative to the current working directory** + (where `demo.sh` runs the binary from), not the system temp directory — + on many dev hosts `/tmp` is `tmpfs`, where `fsync` is nearly free and the + number stops meaning anything as "storage overhead." Comparing this + number meaningfully across host/VM/container requires each side to write + to a comparably real filesystem. + +`--bench` selects which bench(es) run (default `all`, in the fixed order +syscall → mem → io). `--iters` overrides the loop count for whichever +bench(es) run: it means "getpid calls" for `syscall`, "buffer passes" for +`mem`, and "create/write/fsync/unlink cycles" for `io`. Omitted, each bench +uses its own default (chosen so a run takes tens to a few hundred +milliseconds): `syscall` 200,000 calls, `mem` 16 passes, `io` 200 cycles. + +Bad usage (an unknown `--bench` value, a flag with a missing value, an +unknown flag, or a non-numeric/zero `--iters`) prints +`usage: overheadbench [--bench syscall|mem|io|all] [--iters N]` to stderr +and exits 2 — never a silently wrong number. + +## Layout + +``` +35-virtualization-and-kvm/ +├── demo.sh # dispatcher: ./demo.sh [cpp|go|rust|all|build] [args...] +├── verify.lua # automated check driven by CI (host-local sanity checks) +├── cpp/ # CMake preset build, demo.sh +├── go/ # go build, demo.sh +└── rust/ # cargo build, demo.sh +``` + +## The demo contract + +Every language directory has a `demo.sh` with the same interface, identical +across the whole book: + +- `./demo.sh` — build then run (`--bench all`) +- `./demo.sh build` — build only +- `./demo.sh run [args]` — run the built binary with the given arguments +- With env `TARGET` set, `run` deploys the binary to that lab VM via + `scripts/lab/deploy-to-vm.sh "$TARGET" -- [args]` instead of + running locally — this is how the chapter gets the VM-side numbers. + +## Verification + +`verify.lua` runs from this directory with `LSP_LANG` set to one language at +a time. It asserts, per language: + +- the built binary exists; +- a default run (`--bench all`) exits 0 and prints exactly the three rows, + in order `syscall`, `mem`, `io`, each parsing as `bench= + metric= unit=` with the metric inside a generous but real + sanity window (e.g. `syscall` between 1 ns and 5 ms per call, `mem` + between 10 MB/s and 1 TB/s, `io` between 0.1 and 5,000,000 ops/s) — wide + enough to tolerate a loaded CI host or unusual hardware, narrow enough to + catch a zeroed-out, negative, or nonsensical measurement; +- selecting a single bench with `--bench` prints only that bench's row; +- `--iters` with a small explicit value (e.g. 10 or 500) still produces a + sane, parseable row — guarding against a divide-by-zero at low iteration + counts; +- bad usage (unknown `--bench` value, missing flag value, unknown flag, + non-numeric or zero `--iters`) exits 2 and prints the usage line. + +Real host numbers from this machine (Fedora 44, kernel 7.1.3-200.fc44, +`examples/` on an NVMe-backed btrfs filesystem — not `/tmp`): + +| lang | syscall (ns/call) | mem (GB/s) | io (ops/s) | +|------|-------------------:|-----------:|-----------:| +| cpp | 89–120 | 24–29 | 1,480–1,590 | +| go | 90–159 | 12–14 | 1,450–1,670 | +| rust | 87–96 | 16–19 | 1,260–1,580 | + +The exact numbers vary run to run — that's expected of a microbenchmark; +`verify.lua` checks shape and sane range, not a fixed value. + +## What the chapter does with it (not asserted by `verify.lua`) + +1. Run `./demo.sh run` on the bare host (the numbers above). +2. Deploy the same binary to `systems-target` + (`TARGET=systems-target ./demo.sh run`) and record the VM numbers — + expect a real `syscall` gap (vmexit/vmenter round trip on every trapped + instruction) and comparable `mem`/`io` numbers once the guest is warmed + up, modulo whatever the host's virtio/virtualization layer costs for + storage. +3. Build the chapter-34 UBI 10 Containerfile for the same language, run the + binary with `podman run --rm ` on the host, and record the + container numbers — expect `syscall` and `mem` close to bare-host (a + container's process runs directly on the host kernel and CPU), with `io` + depending on the container's storage driver/overlay filesystem. +4. Tabulate host vs VM vs container for the pros/cons discussion: containers + trade isolation strength for near-zero syscall/CPU overhead; a KVM guest + pays a real (if small, with modern hardware virtualization) tax per + trapped instruction in exchange for a separate kernel and stronger + isolation. + +## Mapping to a chapter + +This is a copy of `examples/_template` (`scripts/new-example.sh +35-virtualization-and-kvm`) with the program body replaced by the +`overheadbench` benches described above. Keep the demo contract and +`verify.lua` shape intact so CI and the manifest (`examples/manifest.yaml`) +keep working. diff --git a/examples/35-virtualization-and-kvm/cpp/.clang-format b/examples/35-virtualization-and-kvm/cpp/.clang-format new file mode 100644 index 0000000..2c8640f --- /dev/null +++ b/examples/35-virtualization-and-kvm/cpp/.clang-format @@ -0,0 +1,8 @@ +--- +BasedOnStyle: LLVM +Language: Cpp +Standard: c++20 +IndentWidth: 4 +ColumnLimit: 100 +PointerAlignment: Left +AllowShortFunctionsOnASingleLine: Inline diff --git a/examples/35-virtualization-and-kvm/cpp/.clang-tidy b/examples/35-virtualization-and-kvm/cpp/.clang-tidy new file mode 100644 index 0000000..2f275ba --- /dev/null +++ b/examples/35-virtualization-and-kvm/cpp/.clang-tidy @@ -0,0 +1,11 @@ +--- +Checks: > + bugprone-*, + modernize-*, + performance-*, + readability-*, + -modernize-use-trailing-return-type, + -readability-identifier-length +WarningsAsErrors: '' +HeaderFilterRegex: '.*' +FormatStyle: file diff --git a/examples/35-virtualization-and-kvm/cpp/CMakeLists.txt b/examples/35-virtualization-and-kvm/cpp/CMakeLists.txt new file mode 100644 index 0000000..a167c39 --- /dev/null +++ b/examples/35-virtualization-and-kvm/cpp/CMakeLists.txt @@ -0,0 +1,8 @@ +cmake_minimum_required(VERSION 3.25) +project(overheadbench CXX) + +set(CMAKE_CXX_STANDARD 23) +set(CMAKE_CXX_STANDARD_REQUIRED ON) +set(CMAKE_CXX_EXTENSIONS OFF) + +add_executable(app src/main.cpp) diff --git a/examples/35-virtualization-and-kvm/cpp/CMakePresets.json b/examples/35-virtualization-and-kvm/cpp/CMakePresets.json new file mode 100644 index 0000000..e31bd2f --- /dev/null +++ b/examples/35-virtualization-and-kvm/cpp/CMakePresets.json @@ -0,0 +1,62 @@ +{ + "version": 6, + "cmakeMinimumRequired": { "major": 3, "minor": 25, "patch": 0 }, + "configurePresets": [ + { + "name": "_base", + "hidden": true, + "generator": "Ninja", + "binaryDir": "${sourceDir}/build/${presetName}", + "cacheVariables": { + "CMAKE_EXPORT_COMPILE_COMMANDS": "ON" + } + }, + { + "name": "release", + "inherits": "_base", + "displayName": "Release (-O2 with debug info)", + "cacheVariables": { "CMAKE_BUILD_TYPE": "RelWithDebInfo" } + }, + { + "name": "debug", + "inherits": "_base", + "displayName": "Debug", + "cacheVariables": { "CMAKE_BUILD_TYPE": "Debug" } + }, + { + "name": "asan", + "inherits": "_base", + "displayName": "ASan + UBSan instrumented build", + "cacheVariables": { + "CMAKE_BUILD_TYPE": "Debug", + "CMAKE_CXX_FLAGS": "-O1 -g -fno-omit-frame-pointer -fsanitize=address,undefined", + "CMAKE_EXE_LINKER_FLAGS": "-fsanitize=address,undefined" + } + }, + { + "name": "release-clang", + "inherits": "release", + "displayName": "Release (clang)", + "cacheVariables": { + "CMAKE_C_COMPILER": "clang", + "CMAKE_CXX_COMPILER": "clang++" + } + }, + { + "name": "debug-clang", + "inherits": "debug", + "displayName": "Debug (clang)", + "cacheVariables": { + "CMAKE_C_COMPILER": "clang", + "CMAKE_CXX_COMPILER": "clang++" + } + } + ], + "buildPresets": [ + { "name": "release", "configurePreset": "release" }, + { "name": "debug", "configurePreset": "debug" }, + { "name": "asan", "configurePreset": "asan" }, + { "name": "release-clang", "configurePreset": "release-clang" }, + { "name": "debug-clang", "configurePreset": "debug-clang" } + ] +} diff --git a/examples/35-virtualization-and-kvm/cpp/demo.sh b/examples/35-virtualization-and-kvm/cpp/demo.sh new file mode 100755 index 0000000..6af6509 --- /dev/null +++ b/examples/35-virtualization-and-kvm/cpp/demo.sh @@ -0,0 +1,40 @@ +#!/usr/bin/env bash +set -euo pipefail + +cd "$(dirname "${BASH_SOURCE[0]}")" +REPO_ROOT="$(cd ../../.. && pwd)" +BIN=build/release/app + +build() { + cmake --preset release + cmake --build --preset release +} + +run() { + if [[ ! -x "$BIN" ]]; then + build + fi + if [[ -n "${TARGET:-}" ]]; then + "$REPO_ROOT/scripts/lab/deploy-to-vm.sh" "$TARGET" "$BIN" -- "$@" + else + "./$BIN" "$@" + fi +} + +case "${1:-}" in + build) + build + ;; + run) + shift + run "$@" + ;; + "") + build + run + ;; + *) + echo "usage: $0 [build|run [args]]" >&2 + exit 2 + ;; +esac diff --git a/examples/35-virtualization-and-kvm/cpp/src/main.cpp b/examples/35-virtualization-and-kvm/cpp/src/main.cpp new file mode 100644 index 0000000..ec0b62b --- /dev/null +++ b/examples/35-virtualization-and-kvm/cpp/src/main.cpp @@ -0,0 +1,243 @@ +// overheadbench: three host-local microbenchmarks used to talk about +// virtualization/container overhead in chapter 35 — +// (a) syscall latency: a tight getpid(2) loop, ns/call +// (b) memory bandwidth: sequential read of a big buffer, GB/s +// (c) small-file IO: create/write/fsync/unlink loop, ops/s +// +// overheadbench [--bench syscall|mem|io|all] [--iters N] +// +// Prints one line per bench: "bench= metric= unit=". +// This binary always measures wherever it runs (host, VM, or container) — +// the chapter runs the *same* binary in all three places and tabulates the +// numbers; this example itself only asserts the host numbers are sane. + +#include +#include +#include +#include +#include +#include +#include +#include +#include +#include +#include + +#include +#include +#include + +namespace { + +using Clock = std::chrono::steady_clock; + +enum class Bench { Syscall, Mem, Io, All }; + +struct Config { + Bench bench = Bench::All; + std::optional iters; +}; + +constexpr std::string_view kUsage = + "usage: overheadbench [--bench syscall|mem|io|all] [--iters N]"; + +// Defaults are chosen so each bench runs for roughly tens to a few hundred +// milliseconds on a modern host; identical across all three languages. +constexpr std::uint64_t kSyscallDefaultIters = 200'000; +constexpr std::uint64_t kMemDefaultPasses = 16; +constexpr std::size_t kMemBufBytes = 128ull * 1024 * 1024; // 128 MiB +constexpr std::uint64_t kIoDefaultIters = 200; +constexpr std::size_t kIoFileBytes = 4096; + +[[nodiscard]] std::expected parse_uint(std::string_view s) { + if (s.empty()) { + return std::unexpected("--iters value must not be empty"); + } + std::uint64_t v = 0; + for (char c : s) { + if (c < '0' || c > '9') { + return std::unexpected("--iters value must be a positive integer: " + std::string(s)); + } + v = v * 10 + static_cast(c - '0'); + } + if (v == 0) { + return std::unexpected("--iters value must be >= 1"); + } + return v; +} + +[[nodiscard]] std::expected parse_args(int argc, char** argv) { + Config cfg; + for (int i = 1; i < argc; ++i) { + const std::string_view arg = argv[i]; + if (arg == "--bench") { + if (++i >= argc) { + return std::unexpected("--bench requires a value"); + } + const std::string_view v = argv[i]; + if (v == "syscall") { + cfg.bench = Bench::Syscall; + } else if (v == "mem") { + cfg.bench = Bench::Mem; + } else if (v == "io") { + cfg.bench = Bench::Io; + } else if (v == "all") { + cfg.bench = Bench::All; + } else { + return std::unexpected("unknown --bench value: " + std::string(v)); + } + } else if (arg == "--iters") { + if (++i >= argc) { + return std::unexpected("--iters requires a value"); + } + auto n = parse_uint(argv[i]); + if (!n) { + return std::unexpected(n.error()); + } + cfg.iters = *n; + } else { + return std::unexpected("unknown argument: " + std::string(arg)); + } + } + return cfg; +} + +// --- syscall bench: tight getpid(2) loop, ns/call. -------------------------- +double bench_syscall(std::uint64_t iters) { + const auto start = Clock::now(); + for (std::uint64_t i = 0; i < iters; ++i) { + // A raw syscall (not the libc wrapper) so every iteration really + // crosses into the kernel — this is the syscall-boundary cost the + // chapter contrasts against VM (vmexit trap) and container + // (near-zero extra) overhead. + ::syscall(SYS_getpid); + } + const auto elapsed = Clock::now() - start; + double ns = std::chrono::duration_cast>(elapsed).count(); + if (ns <= 0.0) { + ns = 1.0; + } + return ns / static_cast(iters); +} + +// --- mem bench: sequential read bandwidth over a big buffer, GB/s. --------- +double bench_mem(std::uint64_t passes) { + const std::size_t words = kMemBufBytes / sizeof(std::uint64_t); + std::vector buf(words); + for (std::size_t i = 0; i < words; ++i) { + buf[i] = static_cast(i) * 2654435761ull; + } + + std::uint64_t sum = 0; + const auto start = Clock::now(); + for (std::uint64_t p = 0; p < passes; ++p) { + for (std::size_t i = 0; i < words; ++i) { + sum += buf[i]; + } + } + const auto elapsed = Clock::now() - start; + double secs = std::chrono::duration_cast>(elapsed).count(); + if (secs <= 0.0) { + secs = 1e-9; + } + + // Consume `sum` after the clock stops so the compiler cannot prove the + // whole scan is dead and elide it; this doesn't affect the measured time. + volatile std::uint64_t sink = sum; + (void)sink; + + const double bytes = static_cast(kMemBufBytes) * static_cast(passes); + return bytes / secs / 1e9; +} + +// --- io bench: create/write/fsync/unlink loop, ops/s. ----------------------- +[[nodiscard]] std::expected bench_io(std::uint64_t iters) { + // Relative to the CWD (the demo.sh working directory), not /tmp: on many + // dev hosts /tmp is tmpfs, where fsync is nearly free and the number + // stops meaning anything as "disk IO overhead". The example directory + // itself is normally on a real filesystem, which is the point of + // comparing this number across host/VM/container. + char dir_template[] = "overheadbench-io-XXXXXX"; + const char* dir = ::mkdtemp(dir_template); + if (dir == nullptr) { + return std::unexpected(std::string("mkdtemp: ") + std::strerror(errno)); + } + const std::string path = std::string(dir) + "/probe"; + const std::vector payload(kIoFileBytes, 'x'); + + auto cleanup = [&] { + ::unlink(path.c_str()); + ::rmdir(dir); + }; + + const auto start = Clock::now(); + for (std::uint64_t i = 0; i < iters; ++i) { + int fd = ::open(path.c_str(), O_CREAT | O_WRONLY | O_TRUNC, 0600); + if (fd < 0) { + cleanup(); + return std::unexpected(std::string("open: ") + std::strerror(errno)); + } + const ssize_t w = ::write(fd, payload.data(), payload.size()); + if (w != static_cast(payload.size())) { + ::close(fd); + cleanup(); + return std::unexpected("short write"); + } + if (::fsync(fd) != 0) { + ::close(fd); + cleanup(); + return std::unexpected(std::string("fsync: ") + std::strerror(errno)); + } + ::close(fd); + ::unlink(path.c_str()); + } + const auto elapsed = Clock::now() - start; + double secs = std::chrono::duration_cast>(elapsed).count(); + if (secs <= 0.0) { + secs = 1e-9; + } + ::rmdir(dir); + return static_cast(iters) / secs; +} + +struct Row { + std::string_view name; + std::string_view unit; + double value; +}; + +} // namespace + +int main(int argc, char** argv) { + const auto cfg = parse_args(argc, argv); + if (!cfg) { + std::println(stderr, "{}", cfg.error()); + std::println(stderr, "{}", kUsage); + return 2; + } + + const bool want_syscall = cfg->bench == Bench::Syscall || cfg->bench == Bench::All; + const bool want_mem = cfg->bench == Bench::Mem || cfg->bench == Bench::All; + const bool want_io = cfg->bench == Bench::Io || cfg->bench == Bench::All; + + std::vector rows; + if (want_syscall) { + rows.push_back({"syscall", "ns/call", bench_syscall(cfg->iters.value_or(kSyscallDefaultIters))}); + } + if (want_mem) { + rows.push_back({"mem", "GB/s", bench_mem(cfg->iters.value_or(kMemDefaultPasses))}); + } + if (want_io) { + auto r = bench_io(cfg->iters.value_or(kIoDefaultIters)); + if (!r) { + std::println(stderr, "io bench failed: {}", r.error()); + return 1; + } + rows.push_back({"io", "ops/s", *r}); + } + + for (const auto& row : rows) { + std::println("bench={} metric={:.2f} unit={}", row.name, row.value, row.unit); + } + return 0; +} diff --git a/examples/35-virtualization-and-kvm/demo.sh b/examples/35-virtualization-and-kvm/demo.sh new file mode 100755 index 0000000..afbbe4c --- /dev/null +++ b/examples/35-virtualization-and-kvm/demo.sh @@ -0,0 +1,23 @@ +#!/usr/bin/env bash +set -euo pipefail +cd "$(dirname "$0")" + +langs=(cpp go rust) + +case "${1:-all}" in + cpp|go|rust) + lang="$1" + shift + exec "./$lang/demo.sh" "$@" + ;; + all) + for lang in "${langs[@]}"; do "./$lang/demo.sh"; done + ;; + build) + for lang in "${langs[@]}"; do "./$lang/demo.sh" build; done + ;; + *) + echo "usage: $0 [cpp|go|rust|all|build] [args...]" >&2 + exit 2 + ;; +esac diff --git a/examples/35-virtualization-and-kvm/go/Containerfile b/examples/35-virtualization-and-kvm/go/Containerfile new file mode 100644 index 0000000..779ca6b --- /dev/null +++ b/examples/35-virtualization-and-kvm/go/Containerfile @@ -0,0 +1,24 @@ +# Multi-stage build for overheadbench (chapter 35). Build stage compiles a +# fully static binary against UBI 10's go-toolset (go1.26.5 — the same +# toolchain pin the host uses); the runtime stage is ubi10/ubi-micro, which +# carries no shell and no package manager, so the final image is just the +# static binary plus the handful of files ubi-micro itself ships. +FROM registry.access.redhat.com/ubi10/go-toolset:latest AS build +# Stay in the s2i image's own /opt/app-root/src: it is already writable by +# the image's arbitrary non-root uid (1001, gid 0); a fresh WORKDIR would be +# created root-owned and rootless podman's build user could not write to it. +COPY go.mod go.sum ./ +RUN go mod download +COPY main.go ./ +# CGO_ENABLED=0: unix.Getpid() is a raw syscall, no libc needed, so a fully +# static binary can run unmodified on ubi-micro's minimal userspace. +RUN CGO_ENABLED=0 go build -o overheadbench . + +FROM registry.access.redhat.com/ubi10/ubi-micro:latest +COPY --from=build /opt/app-root/src/overheadbench /usr/local/bin/overheadbench +# Rootless podman default: container runs as the invoking user's mapped uid, +# not root, and writes its scratch io-bench files into WORKDIR (set at run +# time via -w so the io bench lands on the mounted volume, not container +# storage). +ENTRYPOINT ["/usr/local/bin/overheadbench"] +CMD ["--bench", "all"] diff --git a/examples/35-virtualization-and-kvm/go/demo.sh b/examples/35-virtualization-and-kvm/go/demo.sh new file mode 100755 index 0000000..28606df --- /dev/null +++ b/examples/35-virtualization-and-kvm/go/demo.sh @@ -0,0 +1,23 @@ +#!/usr/bin/env bash +set -euo pipefail +cd "$(dirname "$0")" +REPO_ROOT="$(cd ../../.. && pwd)" +BIN=bin/app + +build() { + go build -o "$BIN" . +} + +run() { + if [[ -n "${TARGET:-}" ]]; then + exec "$REPO_ROOT/scripts/lab/deploy-to-vm.sh" "$TARGET" "$BIN" -- "$@" + fi + exec "./$BIN" "$@" +} + +case "${1:-}" in + build) build ;; + run) shift; run "$@" ;; + "") build; run ;; + *) echo "usage: $0 [build|run [args...]]" >&2; exit 2 ;; +esac diff --git a/examples/35-virtualization-and-kvm/go/go.mod b/examples/35-virtualization-and-kvm/go/go.mod new file mode 100644 index 0000000..3af61b8 --- /dev/null +++ b/examples/35-virtualization-and-kvm/go/go.mod @@ -0,0 +1,7 @@ +module github.com/patterncatalyst/linux-systems-programming/examples/35-virtualization-and-kvm/go + +go 1.26 + +toolchain go1.26.5 + +require golang.org/x/sys v0.47.0 diff --git a/examples/35-virtualization-and-kvm/go/go.sum b/examples/35-virtualization-and-kvm/go/go.sum new file mode 100644 index 0000000..37ee2d4 --- /dev/null +++ b/examples/35-virtualization-and-kvm/go/go.sum @@ -0,0 +1,2 @@ +golang.org/x/sys v0.47.0 h1:o7XGOvZQCADBQQ4Y7VNq2dRWQR7JmOUW8Kxx4ZsNgWs= +golang.org/x/sys v0.47.0/go.mod h1:4GL1E5IUh+htKOUEOaiffhrAeqysfVGipDYzABqnCmw= diff --git a/examples/35-virtualization-and-kvm/go/main.go b/examples/35-virtualization-and-kvm/go/main.go new file mode 100644 index 0000000..578b425 --- /dev/null +++ b/examples/35-virtualization-and-kvm/go/main.go @@ -0,0 +1,240 @@ +// overheadbench: three host-local microbenchmarks used to talk about +// virtualization/container overhead in chapter 35 — +// +// (a) syscall latency: a tight getpid(2) loop, ns/call +// (b) memory bandwidth: sequential read of a big buffer, GB/s +// (c) small-file IO: create/write/fsync/unlink loop, ops/s +// +// overheadbench [--bench syscall|mem|io|all] [--iters N] +// +// Prints one line per bench: "bench= metric= unit=". +// This binary always measures wherever it runs (host, VM, or container) — +// the chapter runs the *same* binary in all three places and tabulates the +// numbers; this example itself only asserts the host numbers are sane. +package main + +import ( + "fmt" + "os" + "strconv" + "time" + + "golang.org/x/sys/unix" +) + +const usage = "usage: overheadbench [--bench syscall|mem|io|all] [--iters N]" + +// Defaults are chosen so each bench runs for roughly tens to a few hundred +// milliseconds on a modern host; identical across all three languages. +const ( + syscallDefaultIters = uint64(200_000) + memDefaultPasses = uint64(16) + memBufBytes = uint64(128) * 1024 * 1024 // 128 MiB + ioDefaultIters = uint64(200) + ioFileBytes = 4096 +) + +type bench int + +const ( + benchSyscall bench = iota + benchMem + benchIo + benchAll +) + +type config struct { + bench bench + iters *uint64 +} + +func parseArgs(args []string) (config, error) { + cfg := config{bench: benchAll} + for i := 0; i < len(args); i++ { + switch args[i] { + case "--bench": + i++ + if i >= len(args) { + return config{}, fmt.Errorf("--bench requires a value") + } + switch args[i] { + case "syscall": + cfg.bench = benchSyscall + case "mem": + cfg.bench = benchMem + case "io": + cfg.bench = benchIo + case "all": + cfg.bench = benchAll + default: + return config{}, fmt.Errorf("unknown --bench value: %s", args[i]) + } + case "--iters": + i++ + if i >= len(args) { + return config{}, fmt.Errorf("--iters requires a value") + } + n, err := strconv.ParseUint(args[i], 10, 64) + if err != nil || n == 0 { + return config{}, fmt.Errorf("--iters value must be a positive integer: %s", args[i]) + } + cfg.iters = &n + default: + return config{}, fmt.Errorf("unknown argument: %s", args[i]) + } + } + return cfg, nil +} + +// benchSyscallRun: tight getpid(2) loop, ns/call. +func benchSyscallRun(iters uint64) float64 { + start := time.Now() + for i := uint64(0); i < iters; i++ { + // unix.Getpid issues the raw getpid(2) syscall on every call (the Go + // runtime does not cache it) — every iteration really crosses into + // the kernel, the syscall-boundary cost this chapter contrasts + // against VM (vmexit trap) and container (near-zero extra) overhead. + unix.Getpid() + } + ns := float64(time.Since(start).Nanoseconds()) + if ns <= 0 { + ns = 1 + } + return ns / float64(iters) +} + +// benchMemRun: sequential read bandwidth over a big buffer, GB/s. +func benchMemRun(passes uint64) float64 { + words := memBufBytes / 8 + buf := make([]uint64, words) + for i := range buf { + buf[i] = uint64(i) * 2654435761 + } + + var sum uint64 + start := time.Now() + for p := uint64(0); p < passes; p++ { + for _, w := range buf { + sum += w + } + } + secs := time.Since(start).Seconds() + if secs <= 0 { + secs = 1e-9 + } + + // Consume `sum` after the clock stops so the compiler cannot prove the + // whole scan is dead and discard it; this doesn't affect the measured time. + if sum == 0xFFFFFFFFFFFFFFFF { + fmt.Fprintln(os.Stderr, "unreachable sink:", sum) + } + + bytes := float64(memBufBytes) * float64(passes) + return bytes / secs / 1e9 +} + +// benchIoRun: create/write/fsync/unlink loop, ops/s. +func benchIoRun(iters uint64) (float64, error) { + // Relative to the CWD (the demo.sh working directory), not the system + // temp dir: on many dev hosts that's tmpfs, where fsync is nearly free + // and the number stops meaning anything as "disk IO overhead". The + // example directory itself is normally on a real filesystem, which is + // the point of comparing this number across host/VM/container. + dir, err := os.MkdirTemp(".", "overheadbench-io-*") + if err != nil { + return 0, fmt.Errorf("mkdirtemp: %w", err) + } + defer os.RemoveAll(dir) + + path := dir + "/probe" + payload := make([]byte, ioFileBytes) + for i := range payload { + payload[i] = 'x' + } + + start := time.Now() + for i := uint64(0); i < iters; i++ { + f, err := os.OpenFile(path, os.O_CREATE|os.O_WRONLY|os.O_TRUNC, 0o600) + if err != nil { + return 0, fmt.Errorf("open: %w", err) + } + n, err := f.Write(payload) + if err != nil || n != len(payload) { + f.Close() + return 0, fmt.Errorf("write: %w", err) + } + if err := f.Sync(); err != nil { + f.Close() + return 0, fmt.Errorf("fsync: %w", err) + } + if err := f.Close(); err != nil { + return 0, fmt.Errorf("close: %w", err) + } + if err := os.Remove(path); err != nil { + return 0, fmt.Errorf("unlink: %w", err) + } + } + secs := time.Since(start).Seconds() + if secs <= 0 { + secs = 1e-9 + } + return float64(iters) / secs, nil +} + +type row struct { + name string + unit string + value float64 +} + +func runBenches(cfg config) ([]row, error) { + wantSyscall := cfg.bench == benchSyscall || cfg.bench == benchAll + wantMem := cfg.bench == benchMem || cfg.bench == benchAll + wantIo := cfg.bench == benchIo || cfg.bench == benchAll + + var rows []row + if wantSyscall { + iters := syscallDefaultIters + if cfg.iters != nil { + iters = *cfg.iters + } + rows = append(rows, row{"syscall", "ns/call", benchSyscallRun(iters)}) + } + if wantMem { + passes := memDefaultPasses + if cfg.iters != nil { + passes = *cfg.iters + } + rows = append(rows, row{"mem", "GB/s", benchMemRun(passes)}) + } + if wantIo { + iters := ioDefaultIters + if cfg.iters != nil { + iters = *cfg.iters + } + v, err := benchIoRun(iters) + if err != nil { + return nil, fmt.Errorf("io bench failed: %w", err) + } + rows = append(rows, row{"io", "ops/s", v}) + } + return rows, nil +} + +func main() { + cfg, err := parseArgs(os.Args[1:]) + if err != nil { + fmt.Fprintln(os.Stderr, err) + fmt.Fprintln(os.Stderr, usage) + os.Exit(2) + } + + rows, err := runBenches(cfg) + if err != nil { + fmt.Fprintln(os.Stderr, err) + os.Exit(1) + } + for _, r := range rows { + fmt.Printf("bench=%s metric=%.2f unit=%s\n", r.name, r.value, r.unit) + } +} diff --git a/examples/35-virtualization-and-kvm/rust/Cargo.lock b/examples/35-virtualization-and-kvm/rust/Cargo.lock new file mode 100644 index 0000000..6852ea5 --- /dev/null +++ b/examples/35-virtualization-and-kvm/rust/Cargo.lock @@ -0,0 +1,66 @@ +# This file is automatically @generated by Cargo. +# It is not intended for manual editing. +version = 4 + +[[package]] +name = "bitflags" +version = "2.13.1" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "b588b76d00fde79687d7646a9b5bdf3cc0f655e0bbd080335a95d7e96f3587da" + +[[package]] +name = "errno" +version = "0.3.14" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "39cab71617ae0d63f51a36d69f866391735b51691dbda63cf6f96d042b63efeb" +dependencies = [ + "libc", + "windows-sys", +] + +[[package]] +name = "libc" +version = "0.2.186" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "68ab91017fe16c622486840e4c83c9a37afeff978bd239b5293d61ece587de66" + +[[package]] +name = "linux-raw-sys" +version = "0.12.1" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "32a66949e030da00e8c7d4434b251670a91556f4144941d37452769c25d58a53" + +[[package]] +name = "overheadbench" +version = "0.1.0" +dependencies = [ + "rustix", +] + +[[package]] +name = "rustix" +version = "1.1.4" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "b6fe4565b9518b83ef4f91bb47ce29620ca828bd32cb7e408f0062e9930ba190" +dependencies = [ + "bitflags", + "errno", + "libc", + "linux-raw-sys", + "windows-sys", +] + +[[package]] +name = "windows-link" +version = "0.2.1" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "f0805222e57f7521d6a62e36fa9163bc891acd422f971defe97d64e70d0a4fe5" + +[[package]] +name = "windows-sys" +version = "0.61.2" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "ae137229bcbd6cdf0f7b80a31df61766145077ddf49416a728b02cb3921ff3fc" +dependencies = [ + "windows-link", +] diff --git a/examples/35-virtualization-and-kvm/rust/Cargo.toml b/examples/35-virtualization-and-kvm/rust/Cargo.toml new file mode 100644 index 0000000..8643e4d --- /dev/null +++ b/examples/35-virtualization-and-kvm/rust/Cargo.toml @@ -0,0 +1,11 @@ +[package] +name = "overheadbench" +version = "0.1.0" +edition = "2024" + +[[bin]] +name = "app" +path = "src/main.rs" + +[dependencies] +rustix = { version = "1", features = ["fs", "process", "time"] } diff --git a/examples/35-virtualization-and-kvm/rust/demo.sh b/examples/35-virtualization-and-kvm/rust/demo.sh new file mode 100755 index 0000000..220feea --- /dev/null +++ b/examples/35-virtualization-and-kvm/rust/demo.sh @@ -0,0 +1,26 @@ +#!/usr/bin/env bash +set -euo pipefail +cd "$(dirname "${BASH_SOURCE[0]}")" + +BIN=target/release/app + +build() { + cargo build --release +} + +run() { + if [[ -n "${TARGET:-}" ]]; then + local repo_root + repo_root=$(cd ../../.. && pwd) + "$repo_root/scripts/lab/deploy-to-vm.sh" "$TARGET" "$BIN" -- "$@" + else + "./$BIN" "$@" + fi +} + +case "${1:-}" in + build) build ;; + run) shift; run "$@" ;; + "") build; run ;; + *) echo "usage: $0 [build|run [args...]]" >&2; exit 2 ;; +esac diff --git a/examples/35-virtualization-and-kvm/rust/rust-toolchain.toml b/examples/35-virtualization-and-kvm/rust/rust-toolchain.toml new file mode 100644 index 0000000..4099f1a --- /dev/null +++ b/examples/35-virtualization-and-kvm/rust/rust-toolchain.toml @@ -0,0 +1,3 @@ +[toolchain] +channel = "1.97.1" +components = ["rustfmt", "clippy", "rust-src"] diff --git a/examples/35-virtualization-and-kvm/rust/src/main.rs b/examples/35-virtualization-and-kvm/rust/src/main.rs new file mode 100644 index 0000000..59f1cba --- /dev/null +++ b/examples/35-virtualization-and-kvm/rust/src/main.rs @@ -0,0 +1,257 @@ +// overheadbench: three host-local microbenchmarks used to talk about +// virtualization/container overhead in chapter 35 — +// (a) syscall latency: a tight getpid(2) loop, ns/call +// (b) memory bandwidth: sequential read of a big buffer, GB/s +// (c) small-file IO: create/write/fsync/unlink loop, ops/s +// +// overheadbench [--bench syscall|mem|io|all] [--iters N] +// +// Prints one line per bench: "bench= metric= unit=". +// This binary always measures wherever it runs (host, VM, or container) — +// the chapter runs the *same* binary in all three places and tabulates the +// numbers; this example itself only asserts the host numbers are sane. + +use std::fmt; +use std::fs::{self, OpenOptions}; +use std::hint::black_box; +use std::io::Write as _; +use std::time::Instant; + +const USAGE: &str = "usage: overheadbench [--bench syscall|mem|io|all] [--iters N]"; + +// Defaults are chosen so each bench runs for roughly tens to a few hundred +// milliseconds on a modern host; identical across all three languages. +const SYSCALL_DEFAULT_ITERS: u64 = 200_000; +const MEM_DEFAULT_PASSES: u64 = 16; +const MEM_BUF_BYTES: u64 = 128 * 1024 * 1024; // 128 MiB +const IO_DEFAULT_ITERS: u64 = 200; +const IO_FILE_BYTES: usize = 4096; + +#[derive(Clone, Copy, PartialEq)] +enum Bench { + Syscall, + Mem, + Io, + All, +} + +struct Config { + bench: Bench, + iters: Option, +} + +#[derive(Debug)] +struct ArgError(String); + +impl fmt::Display for ArgError { + fn fmt(&self, f: &mut fmt::Formatter<'_>) -> fmt::Result { + write!(f, "{}", self.0) + } +} + +fn parse_args(args: &[String]) -> Result { + let mut cfg = Config { + bench: Bench::All, + iters: None, + }; + let mut i = 0; + while i < args.len() { + match args[i].as_str() { + "--bench" => { + i += 1; + let v = args + .get(i) + .ok_or_else(|| ArgError("--bench requires a value".into()))?; + cfg.bench = match v.as_str() { + "syscall" => Bench::Syscall, + "mem" => Bench::Mem, + "io" => Bench::Io, + "all" => Bench::All, + other => return Err(ArgError(format!("unknown --bench value: {other}"))), + }; + } + "--iters" => { + i += 1; + let v = args + .get(i) + .ok_or_else(|| ArgError("--iters requires a value".into()))?; + let n: u64 = v.parse().map_err(|_| { + ArgError(format!("--iters value must be a positive integer: {v}")) + })?; + if n == 0 { + return Err(ArgError("--iters value must be >= 1".into())); + } + cfg.iters = Some(n); + } + other => return Err(ArgError(format!("unknown argument: {other}"))), + } + i += 1; + } + Ok(cfg) +} + +/// Tight getpid(2) loop, ns/call. +fn bench_syscall(iters: u64) -> f64 { + let start = Instant::now(); + for _ in 0..iters { + // rustix::process::getpid issues the raw getpid(2) syscall every + // call — every iteration really crosses into the kernel, the + // syscall-boundary cost this chapter contrasts against VM (vmexit + // trap) and container (near-zero extra) overhead. + black_box(rustix::process::getpid()); + } + let mut ns = start.elapsed().as_nanos() as f64; + if ns <= 0.0 { + ns = 1.0; + } + ns / iters as f64 +} + +/// Sequential read bandwidth over a big buffer, GB/s. +fn bench_mem(passes: u64) -> f64 { + let words = (MEM_BUF_BYTES / 8) as usize; + let mut buf = vec![0u64; words]; + for (i, w) in buf.iter_mut().enumerate() { + *w = (i as u64).wrapping_mul(2654435761); + } + + let mut sum: u64 = 0; + let start = Instant::now(); + for _ in 0..passes { + for &w in &buf { + sum = sum.wrapping_add(w); + } + } + let mut secs = start.elapsed().as_secs_f64(); + if secs <= 0.0 { + secs = 1e-9; + } + + // Consume `sum` after the clock stops so the compiler cannot prove the + // whole scan is dead and elide it; this doesn't affect the measured time. + black_box(sum); + + let bytes = MEM_BUF_BYTES as f64 * passes as f64; + bytes / secs / 1e9 +} + +/// Create/write/fsync/unlink loop, ops/s. +fn bench_io(iters: u64) -> Result { + // Relative to the CWD (the demo.sh working directory), not the system + // temp dir: on many dev hosts that's tmpfs, where fsync is nearly free + // and the number stops meaning anything as "disk IO overhead". The + // example directory itself is normally on a real filesystem, which is + // the point of comparing this number across host/VM/container. + let dir = std::path::PathBuf::from(format!("overheadbench-io-{}", std::process::id())); + fs::create_dir_all(&dir).map_err(|e| format!("mkdir: {e}"))?; + let path = dir.join("probe"); + let payload = vec![b'x'; IO_FILE_BYTES]; + + let cleanup = |p: &std::path::Path, d: &std::path::Path| { + let _ = fs::remove_file(p); + let _ = fs::remove_dir(d); + }; + + let start = Instant::now(); + for _ in 0..iters { + let mut f = match OpenOptions::new() + .create(true) + .write(true) + .truncate(true) + .open(&path) + { + Ok(f) => f, + Err(e) => { + cleanup(&path, &dir); + return Err(format!("open: {e}")); + } + }; + if let Err(e) = f.write_all(&payload) { + cleanup(&path, &dir); + return Err(format!("write: {e}")); + } + if let Err(e) = f.sync_all() { + cleanup(&path, &dir); + return Err(format!("fsync: {e}")); + } + drop(f); + if let Err(e) = fs::remove_file(&path) { + cleanup(&path, &dir); + return Err(format!("unlink: {e}")); + } + } + let mut secs = start.elapsed().as_secs_f64(); + if secs <= 0.0 { + secs = 1e-9; + } + let _ = fs::remove_dir(&dir); + Ok(iters as f64 / secs) +} + +struct Row { + name: &'static str, + unit: &'static str, + value: f64, +} + +fn run_benches(cfg: &Config) -> Result, String> { + let want_syscall = cfg.bench == Bench::Syscall || cfg.bench == Bench::All; + let want_mem = cfg.bench == Bench::Mem || cfg.bench == Bench::All; + let want_io = cfg.bench == Bench::Io || cfg.bench == Bench::All; + + let mut rows = Vec::new(); + if want_syscall { + let iters = cfg.iters.unwrap_or(SYSCALL_DEFAULT_ITERS); + rows.push(Row { + name: "syscall", + unit: "ns/call", + value: bench_syscall(iters), + }); + } + if want_mem { + let passes = cfg.iters.unwrap_or(MEM_DEFAULT_PASSES); + rows.push(Row { + name: "mem", + unit: "GB/s", + value: bench_mem(passes), + }); + } + if want_io { + let iters = cfg.iters.unwrap_or(IO_DEFAULT_ITERS); + let value = bench_io(iters).map_err(|e| format!("io bench failed: {e}"))?; + rows.push(Row { + name: "io", + unit: "ops/s", + value, + }); + } + Ok(rows) +} + +fn main() { + let args: Vec = std::env::args().skip(1).collect(); + + let cfg = match parse_args(&args) { + Ok(c) => c, + Err(e) => { + eprintln!("{e}"); + eprintln!("{USAGE}"); + std::process::exit(2); + } + }; + + let rows = match run_benches(&cfg) { + Ok(r) => r, + Err(e) => { + eprintln!("{e}"); + std::process::exit(1); + } + }; + + for row in &rows { + println!( + "bench={} metric={:.2} unit={}", + row.name, row.value, row.unit + ); + } +} diff --git a/examples/35-virtualization-and-kvm/verify.lua b/examples/35-virtualization-and-kvm/verify.lua new file mode 100644 index 0000000..f7445c0 --- /dev/null +++ b/examples/35-virtualization-and-kvm/verify.lua @@ -0,0 +1,147 @@ +-- Verify overheadbench (chapter 35: virtualization and KVM). This example +-- itself is host-local: it asserts the three microbenchmarks actually run +-- on THIS host and emit parseable, sane-range numeric rows. +-- +-- * "--bench all" (the default) prints exactly the three rows, in order +-- syscall/mem/io, each "bench= metric= unit=" with +-- parsing as a finite non-negative number and the expected unit. +-- * Each metric lands in a generous but real sanity window — wide enough +-- to tolerate a loaded CI host, narrow enough to catch "printed zero" +-- or "printed a bogus huge/negative number" bugs. +-- * Selecting a single bench via --bench prints only that bench's row. +-- * --iters overrides the loop count for whichever bench(es) run, and a +-- small explicit value still produces a sane row (guards against a +-- divide-by-zero / elapsed==0 edge case at low iteration counts). +-- * Bad usage (unknown --bench value, missing flag value, unknown flag, +-- non-numeric --iters) exits 2 and prints the usage line — never a +-- silent wrong answer. +-- +-- The chapter separately deploys this same binary to a lab VM +-- (systems-target) and into a rootless podman container on the host, and +-- tabulates host vs VM vs container numbers for the overhead discussion; +-- that comparison is authored by hand, not asserted here. +-- +-- Invoked from the example directory with LSP_LANG (cpp|go|rust) and +-- EXAMPLE_DIR set. Plain Lua 5.1+ (works under luajit). + +local script_dir = arg[0]:match("(.*/)") or "./" +local repo_root = os.getenv("REPO_ROOT") or (script_dir .. "../..") +package.path = package.path .. ";" .. repo_root .. "/scripts/lib/?.lua" +local checks = require("checks") + +local lang = os.getenv("LSP_LANG") +if lang ~= "cpp" and lang ~= "go" and lang ~= "rust" then + checks.skip("LSP_LANG must be cpp|go|rust (got " .. tostring(lang) .. ")") +end + +local bins = { + cpp = "cpp/build/release/app", + go = "go/bin/app", + rust = "rust/target/release/app", +} +local bin = bins[lang] + +local function expect_true(cond, label) + if cond then + checks.expect_match("yes", "yes", label) + else + checks.expect_match("no", "yes", label) + end +end + +-- Sane ranges: wide enough for a loaded CI host or older hardware, narrow +-- enough to catch a broken/zero/negative measurement. +local RANGES = { + syscall = { unit = "ns/call", lo = 1, hi = 5e6 }, -- 1ns .. 5ms/call + mem = { unit = "GB/s", lo = 0.01, hi = 1000 }, -- 10MB/s .. 1TB/s + io = { unit = "ops/s", lo = 0.1, hi = 5e6 }, -- down to one op per 10s +} + +local function check_row(out, name, label_prefix) + local pattern = "bench=" .. name .. " metric=(%-?%d+%.%d+) unit=(%S+)" + local metric_s, unit = out:match(pattern) + expect_true(metric_s ~= nil, label_prefix .. ": " .. name .. " row parses (bench=" .. name .. " metric= unit=)") + if metric_s == nil then return end + local metric = tonumber(metric_s) + local r = RANGES[name] + expect_true(unit == r.unit, label_prefix .. ": " .. name .. " unit is " .. r.unit .. " (got " .. tostring(unit) .. ")") + expect_true(metric ~= nil and metric >= r.lo and metric <= r.hi, + label_prefix .. ": " .. name .. " metric " .. tostring(metric_s) .. " " .. tostring(unit) .. + " is in sane range [" .. r.lo .. ", " .. r.hi .. "]") +end + +-- --------------------------------------------------------------------------- +-- 1. Built binary exists (the orchestrator builds before verifying). +-- --------------------------------------------------------------------------- +local r = checks.run("test -x " .. bin) +checks.expect_exit(r, 0, lang .. ": built binary " .. bin .. " exists") + +-- --------------------------------------------------------------------------- +-- 2. Default run (--bench all): three rows, syscall/mem/io, in order, each +-- parseable and in a sane range. +-- --------------------------------------------------------------------------- +r = checks.run("./demo.sh " .. lang .. " run") +checks.expect_exit(r, 0, lang .. ": default run (--bench all) exits 0") + +local si = r.out:find("bench=syscall") +local mi = r.out:find("bench=mem") +local ii = r.out:find("bench=io") +expect_true(si ~= nil and mi ~= nil and ii ~= nil, lang .. ": all three bench rows are present") +expect_true(si ~= nil and mi ~= nil and ii ~= nil and si < mi and mi < ii, + lang .. ": rows print in order syscall, mem, io") + +check_row(r.out, "syscall", lang) +check_row(r.out, "mem", lang) +check_row(r.out, "io", lang) + +-- --------------------------------------------------------------------------- +-- 3. Selecting a single bench prints only that row. +-- --------------------------------------------------------------------------- +for _, name in ipairs({ "syscall", "mem", "io" }) do + r = checks.run("./demo.sh " .. lang .. " run --bench " .. name) + checks.expect_exit(r, 0, lang .. ": --bench " .. name .. " exits 0") + check_row(r.out, name, lang) + for _, other in ipairs({ "syscall", "mem", "io" }) do + if other ~= name then + expect_true(not r.out:find("bench=" .. other), lang .. ": --bench " .. name .. " does not print the " .. other .. " row") + end + end +end + +-- --------------------------------------------------------------------------- +-- 4. --iters overrides the loop count; a small explicit value still yields +-- a sane, parseable row (no divide-by-zero at low iteration counts). +-- --------------------------------------------------------------------------- +r = checks.run("./demo.sh " .. lang .. " run --bench syscall --iters 500") +checks.expect_exit(r, 0, lang .. ": --bench syscall --iters 500 exits 0") +check_row(r.out, "syscall", lang) + +r = checks.run("./demo.sh " .. lang .. " run --bench io --iters 10") +checks.expect_exit(r, 0, lang .. ": --bench io --iters 10 exits 0") +check_row(r.out, "io", lang) + +-- --------------------------------------------------------------------------- +-- 5. Bad usage: unknown --bench value, missing flag value, unknown flag, +-- non-numeric --iters all exit 2 with the usage line -- never a silent +-- wrong answer. +-- --------------------------------------------------------------------------- +r = checks.run("./demo.sh " .. lang .. " run --bench bogus") +checks.expect_exit(r, 2, lang .. ": unknown --bench value exits 2") +checks.expect_match(r.out, "usage: overheadbench", lang .. ": unknown --bench value prints usage") + +r = checks.run("./demo.sh " .. lang .. " run --bench") +checks.expect_exit(r, 2, lang .. ": --bench with no value exits 2") +checks.expect_match(r.out, "usage: overheadbench", lang .. ": --bench with no value prints usage") + +r = checks.run("./demo.sh " .. lang .. " run --iters abc") +checks.expect_exit(r, 2, lang .. ": non-numeric --iters exits 2") +checks.expect_match(r.out, "usage: overheadbench", lang .. ": non-numeric --iters prints usage") + +r = checks.run("./demo.sh " .. lang .. " run --iters 0") +checks.expect_exit(r, 2, lang .. ": --iters 0 exits 2") + +r = checks.run("./demo.sh " .. lang .. " run --nonsense") +checks.expect_exit(r, 2, lang .. ": unknown flag exits 2") +checks.expect_match(r.out, "usage: overheadbench", lang .. ": unknown flag prints usage") + +checks.finish() diff --git a/examples/manifest.yaml b/examples/manifest.yaml index 654c931..e816357 100644 --- a/examples/manifest.yaml +++ b/examples/manifest.yaml @@ -129,3 +129,20 @@ examples: langs: [cpp, go, rust] mode: local timeout: 360 + - id: 32-namespaces-and-cgroups + langs: [cpp, go, rust] + mode: vm + timeout: 360 + - id: 33-seccomp-and-landlock + langs: [cpp, go, rust] + mode: vm + timeout: 360 + - id: 34-our-programs-in-containers + langs: [cpp, go, rust] + mode: local + requires: [podman] + timeout: 420 + - id: 35-virtualization-and-kvm + langs: [cpp, go, rust] + mode: local + timeout: 360 diff --git a/scripts/test-all-examples.py b/scripts/test-all-examples.py index 5d7949b..15ed6ab 100755 --- a/scripts/test-all-examples.py +++ b/scripts/test-all-examples.py @@ -230,6 +230,20 @@ def lgtm_up() -> bool: return _lgtm_up +_podman_up: bool | None = None + + +def podman_up() -> bool: + global _podman_up + if _podman_up is None: + try: + r = subprocess.run(["podman", "info"], capture_output=True, timeout=15) + _podman_up = r.returncode == 0 + except (FileNotFoundError, subprocess.SubprocessError, OSError): + _podman_up = False + return _podman_up + + def revert_target() -> bool: p = subprocess.run( [str(REPO_ROOT / "scripts" / "lab" / "revert-vm.sh"), TARGET_VM, "lab-ready"], @@ -392,6 +406,12 @@ def main(argv: list[str]) -> int: print(f" {ex['id']}: SKIP (lgtm down)") continue + if "podman" in ex["requires"] and not podman_up(): + for l in langs: + results[(ex["id"], l)].mark("SKIP", "podman unavailable") + print(f" {ex['id']}: SKIP (podman unavailable)") + continue + if ex["mode"] in ("vm", "vm-peer"): needed = [TARGET_VM] + ([PEER_VM] if ex["mode"] == "vm-peer" else []) if not all(vm_reachable(vm) for vm in needed):