bug(proxy): EMFILE stops the sandbox proxy while phase remains Ready

[jyaunches]

Agent Diagnostic

• Skills loaded: debug-openshell-cluster, openshell-cli, and create-github-issue.
• OpenShell version reproduced: 0.0.72 with the Docker driver.
• Latest release checked: 0.0.85 (released 2026-07-16). The affected accept-loop implementation is still present on current main.
• Known fixes reviewed: release history and current source for crates/openshell-supervisor-network/src/proxy.rs, the Docker sandbox lifecycle, and the exec relay.
• Possible duplicates reviewed: searched open and closed issues for Too many open files, EMFILE, Broken pipe, exec relay, proxy FD leaks, and sandboxes remaining Ready. fix: K8s driver watch stream crashes with FD leak when non-gateway Sandbox resources exist #2211 is related only at the symptom level: it is a Kubernetes gateway watch-stream leak, while this report concerns the network proxy inside a Docker sandbox supervisor.
• Findings: on an accept error the proxy emits Proxy accept error: ... and immediately breaks out of the accept loop. The proxy task ending is not reflected in sandbox phase/readiness, so the sandbox can continue to report Ready after its policy proxy is permanently gone. Because the proxy and SSH relay share the supervisor process's descriptor table, descriptor exhaustion also breaks subsequent exec relays.
• Latest-release runtime note: the production sandbox was pinned to OpenShell 0.0.72 by its supported NemoClaw stack, so it was not destructively upgraded solely for retesting. Source inspection confirms the same terminal break behavior on 0.0.85/main.

Description

Actual behavior: When the sandbox network proxy's listener reaches EMFILE, it logs Proxy accept error: Too many open files (os error 24) and exits its accept loop permanently. Subsequent host-side execs fail with SSH relay pipe/EOF errors, but openshell sandbox list continues to show the sandbox as Ready. Reducing descriptor pressure does not restart the proxy; recreating the sandbox is required.

Expected behavior: A failed proxy accept loop should either recover with bounded backoff once descriptor pressure clears, or terminate/degrade the sandbox supervisor so the control plane no longer reports Ready. Descriptor pressure and the stopped proxy task should be observable through health/status before every exec relay is unusable.

The important invariant is that Ready must imply the sandbox's required policy proxy and exec relay are still operational.

Reproduction Steps

Use a disposable Docker-driver sandbox. The low RLIMIT_NOFILE below accelerates the production failure and should not be used on a sandbox containing data.

1. Create a sandbox and allow a harmless HTTPS target:

   openshell sandbox create --name qa-fd --no-auto-providers
   # Exit the interactive shell, leaving the sandbox running.
   openshell policy update qa-fd \
     --add-endpoint api.github.com:443:read-only:rest:enforce \
     --binary /usr/bin/curl \
     --wait

2. In terminal A, connect before lowering the limit:

   openshell sandbox connect qa-fd

3. In terminal B, find the disposable sandbox container and lower PID 1's open-file limit to only eight descriptors above its current use:

   container="$(docker ps --filter name=openshell-qa-fd --format '{{.Names}}' | head -1)"
   docker exec --user 0 "$container" python -c \
     'import os,resource; n=len(os.listdir("/proc/1/fd")); lim=n+8; resource.prlimit(1, resource.RLIMIT_NOFILE, (lim,lim)); print(f"nofile={lim}")'

4. Back in terminal A, create concurrent proxied connections:

   for i in $(seq 1 64); do
     curl -fsS https://api.github.com/zen >/dev/null &
   done
   wait

5. In terminal B, inspect logs and phase, then try a fresh exec:

   openshell logs qa-fd --since 5m
   openshell sandbox list
   openshell sandbox exec --name qa-fd -- true

6. Let the concurrent requests finish and retry the exec. The proxy accept task does not restart even after descriptors become available.

If the published base image does not include Python's resource.prlimit, run the same fault injection with prlimit --pid 1 --nofile <limit>:<limit> from a disposable base image containing util-linux.

Environment

• OS: Ubuntu Linux, x86_64
• Compute driver: Docker
• OpenShell reproduced: 0.0.72
• Latest release checked: 0.0.85; affected source path remains unchanged
• Downstream managed stack at observation: NemoClaw 0.0.81, OpenClaw 2026.6.10
• Possible duplicates checked: yes, across open and closed issues

Logs

[sandbox] Proxy accept error: Too many open files (os error 24)
[sandbox] ssh relay bridge failed (...): write to target failed: Broken pipe (os error 32)
Error: exec relay closed before the command reported an exit status

$ openshell sandbox list
# qa-fd still reports Phase: Ready

Suggested Acceptance Coverage

• Fault-inject EMFILE into the proxy listener and assert the accept task does not silently disappear.
• Assert the sandbox leaves Ready (or the listener retries successfully) when a required proxy task exits.
• Assert status exposes descriptor pressure or a stopped proxy subsystem.
• Assert a recovered descriptor budget permits new proxied connections and exec relays without sandbox recreation.

Related Downstream Recovery Gap

NVIDIA/NemoClaw#7062 tracks the managed-MCP rebuild dead end exposed when this OpenShell failure makes a registered sandbox unreachable.

Agent-First Checklist

• I pointed my agent at the repository and had it investigate this issue.
• I loaded the relevant OpenShell skills.
• I checked the latest release and explained why the pinned production sandbox was not upgraded in place.
• I searched existing issues for possible duplicates.
• The agent could not resolve this through configuration; the proxy accept task exits and health remains stale.