Skip to content

Start on kernels without any cgroup device gate - #1

Merged
aroxu merged 1 commit into
aroxu:mainfrom
MadeByAlpha:fix/cgroup-unavailable-devices
Sep 12, 2026
Merged

aroxu merged 1 commit into
aroxu:mainfrom
MadeByAlpha:fix/cgroup-unavailable-devices

Conversation

@AlphaKR93

Copy link
Copy Markdown

Important

Disclaimer: This PR was created by Claude Opus 5.

  • Verify operation on an actual device
  • Confirm code quality
  • Close Claude's open questions
    1. direct and exclusive never used the gate. Should they also be allowed on
      these kernels, or is refusing them the intended conservative stance? The
      patch currently refuses.
    2. Should CAP_MKNOD be dropped from the bounding set in this tier, mirroring
      the existing CAP_SYS_BOOT drop? That would close the mknod path. It is
      left out here because it also affects Docker device creation, which seems
      like your call.
    3. A kernel ≥ 4.15 with CONFIG_CGROUP_BPF=n but usable cgroup2 is the mirror
      case (v2 delegation fine, gate missing). Adding a v2-nodev tier would need
      health to distinguish an attached program from an absent one, which is not
      observable from inside the container; a launcher-written marker would be
      required. Out of scope here.

Start on kernels without any cgroup device gate

Summary

On kernels that provide neither cgroup device policy backend, start fails
during namespace setup and Debian never boots, even with USB passthrough off.
This adds a last legacy tier that delegates only the name=systemd hierarchy
(cgroup_mode=v1-nodev), allowed only while USB passthrough is off, and makes
the USB log lines state whether major 189 is actually denied.

Symptom

debian-lifecycle.log, start from the app button:

BFU_DEBIAN_STAGE cgroup_v2_controllers_processed enabled=1
BFU_DEBIAN_LAUNCHER_FAILED stage=cgroup_v2_device_bpf_load errno=22 error=Invalid argument
BFU_DEBIAN_COMPAT cgroup_requested=auto cgroup_v2_available=false errno=22 fallback=v1
BFU_DEBIAN_LAUNCHER_FAILED stage=cgroup_v1_devices_mount errno=2 error=No such file or directory
ANDROID_RESULT operation=START ... output=BFU_DEBIAN_START_FAILED stage=namespace_setup exit=91

Host diagnostics from the same run:

#subsys_name	hierarchy	num_cgroups	enabled
cpuset	3	10	1
cpu	2	13	1
cpuacct	0	203	1
schedtune	4	8	1
blkio	1	2	1
memory	0	203	1
freezer	5	5	1

Kernel configuration on the affected device:

# CONFIG_CGROUP_DEVICE is not set
CONFIG_CGROUP_BPF=y

Device: Samsung Exynos 9610, Linux 4.14.194 (Mint Beta 1413), host USB mode off.

Root cause

Both device policy backends are unavailable, and each failure is currently
fatal.

  • v2 path. BPF_PROG_TYPE_CGROUP_DEVICE was added in Linux 4.15. On 4.14
    the program type does not exist, so bpf(BPF_PROG_LOAD) returns -EINVAL
    from find_prog_type() regardless of CONFIG_CGROUP_BPF:

    static int find_prog_type(enum bpf_prog_type type, struct bpf_prog *prog)
    {
    	if (type >= ARRAY_SIZE(bpf_prog_types) || !bpf_prog_types[type])
    		return -EINVAL;
  • v1 fallback. The kernel is built without CONFIG_CGROUP_DEVICE, so there
    is no devices row in /proc/cgroups, and mounting the controller fails with
    ENOENT rather than EINVAL, because cgroup v1 option parsing reports an
    unknown subsystem name that way (4.14 parse_cgroupfs_options()):

    		if (i == CGROUP_SUBSYS_COUNT)
    			return -ENOENT;

A relevant detail: the device gate is used by exactly one policy today. Both
apply_host_usb_v2_policy() and prepare_devices_cgroup_mount() install a deny
rule only when the policy is off, and return early with
action=allow_parent_policy for direct and exclusive. So on these kernels
the launcher refuses to start for a capability that the requested policy may not
even use.

Change

  • Add CGROUP_MODE_V1_NO_DEVICES (v1-nodev): the name=systemd hierarchy is
    delegated as before, the devices hierarchy is skipped in mount, move, view
    and PID 1 setup.
  • Probe /proc/cgroups for the devices controller instead of inferring
    availability from a mount errno, so a known kernel limit is not reported as a
    mount fault.
  • Allow the new tier only when host USB mode is off. direct and exclusive
    now fail with stage=cgroup_devices_controller error=device_gate_unavailable_for_usb_passthrough (exit 120) instead of a
    bare mount errno.
  • Report enforcement honestly: the USB log lines carry
    cgroup_major_189_denied= and future_hotplug_cgroup_enforced= as true
    only when a device gate is actually active. /dev/bus/usb keeps the empty
    read-only overmount in every case.
  • Extend the health probe with a v1-nodev branch that reports
    cgroup_delegation=delegated devices_cgroup=unavailable, plus a new
    systemd_path= field for diagnosis. The app's health requirement
    (cgroup_delegation=delegated) is unchanged.
  • Keep systemd on the legacy cgroup command line
    (systemd.unified_cgroup_hierarchy=0, SYSTEMD_CGROUP_ENABLE_LEGACY_FORCE=1)
    for the new mode.
  • Document the tier in docs/architecture.md and docs/architecture.ko.md.

The v2 path is untouched: a kernel that cannot load the device BPF program still
does not resolve to v2, so devices_cgroup=bpf never becomes a false claim.

Security considerations

In v1-nodev the blocked USB policy loses its defence in depth. What remains is
the empty read-only overmount on /dev/bus/usb; what is gone is the major 189
deny rule, so a root process inside Debian could recreate a usbfs node with
mknod under the shared /dev and open it. The patch therefore keeps this tier
limited to the policy that requests no USB access, and states the weaker
guarantee in the log and the docs rather than implying enforcement.

Compatibility

  • The supervisor state file keeps format=6. Only a new cgroup_mode value is
    introduced; an older binary reading a v1-nodev state resolves it to unknown
    and asks for a restart, which is the existing behaviour for unrecognised
    modes.
  • LauncherState.cgroup_mode grows from 8 to 16 bytes to hold the new name. The
    state file is text, so no layout compatibility is involved.
  • The probe path (enter_debian_probe) passes CGROUP_MODE_UNKNOWN, so its USB
    log line now reads future_hotplug_cgroup_enforced=false. That path never
    sets up a device cgroup, so the previous true was inaccurate.

Testing

  • gcc -fsyntax-only -Wall -Wextra -std=c11 app/src/main/cpp/bfu_namespace_probe.c
    is clean (host headers; the NDK build runs in CI).
  • Device verification pending on the affected kernel: start from the app, then
    ANDROID_HEALTH ... cgroup_mode=v1-nodev cgroup_delegation=delegated devices_cgroup=unavailable, SSH login, stop, restart.
  • Not yet exercised: a kernel with the devices controller present (the v1 path
    should be unchanged), and Docker inside the new tier.

Linux added BPF_PROG_TYPE_CGROUP_DEVICE in 4.15, and a kernel built
without CONFIG_CGROUP_DEVICE has no cgroup v1 `devices` controller.
On such kernels the v2 probe fails at BPF_PROG_LOAD with EINVAL and the
v1 fallback fails at mount(2) with ENOENT, because cgroup v1 rejects
unknown subsystem names, so the launcher cannot start Debian at all:

  BFU_DEBIAN_LAUNCHER_FAILED stage=cgroup_v2_device_bpf_load errno=22
  BFU_DEBIAN_LAUNCHER_FAILED stage=cgroup_v1_devices_mount errno=2
  BFU_DEBIAN_START_FAILED stage=namespace_setup exit=91

Add a last legacy tier that delegates only the `name=systemd`
hierarchy, reported as cgroup_mode=v1-nodev. The device gate is used
solely by the blocked USB policy, so the new tier starts only while USB
passthrough is off; direct and exclusive passthrough still require a
device gate and now fail with a named stage instead of a bare mount
errno. /dev/bus/usb keeps its empty read-only overmount, and the USB log
lines report whether major 189 is actually denied instead of asserting
it unconditionally.

Probe /proc/cgroups for the controller rather than inferring it from a
mount errno, extend the health probe with a v1-nodev branch (and a
systemd_path field), and keep systemd on the legacy cgroup command line
for the new mode.
@aroxu
aroxu marked this pull request as ready for review September 12, 2026 10:59
@aroxu
aroxu merged commit cff151d into aroxu:main Sep 12, 2026
@AlphaKR93
AlphaKR93 deleted the fix/cgroup-unavailable-devices branch September 13, 2026 05:25
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants