fix: check native subagent spawn before fan-out

CHANGELOG.md

@@ -2,6 +2,7 @@
 
 ## Unreleased — Codex run coordination improvements
 
+- Added a first-standby native spawn preflight, missing-thread diagnostics that stop per-role retries, and sequential fallback guidance that preserves persistence preferences and keeps native-only failures explicit ([#1](https://github.com/revfactory/codex-harness/issues/1)).
 - Added v2 run state, refreshed task context with input fingerprints and decisions, native agent ID/lifecycle records, and selective resume invalidation.
 - Added completion checks for required result evidence, artifacts, and freshness, separate from native runtime observations.
 - Added explicit concurrent communication logs under `_workspace/communications/`, with correlated questions, answers, delivery outcomes, acknowledgments, and optional Markdown export.

docs/compatibility.md

@@ -56,7 +56,23 @@ The Codex plugin manifest packages the skill. Installing it makes that workflow
 
 Sandbox declarations express the intended agent policy. Parent session runtime overrides and platform restrictions can affect the effective sandbox. Ownership instructions do not create per-file access controls. Check the active client's effective policy when verifying write boundaries. [OpenAI subagent reference](https://learn.chatgpt.com/docs/agent-configuration/subagents).
 
-If custom roles are absent, verify the working project root, trust/configuration loading, and TOML validity, then start a new thread. If native subagent tools are unavailable, the parent can still follow the workflow sequentially; it should report that execution mode explicitly.
+If custom roles are absent, verify the working project root, trust/configuration loading, and TOML validity, then start a new thread. An available built-in role can cover a suitable task with explicit instructions; report that substitution. Role discovery and exposed subagent tools do not establish that native spawning works in the current session.
+
+## Native spawn preflight and ephemeral sessions
+
+Before wider fan-out, use the first planned agent as a compatibility check:
+
+1. Inspect the active native tool schema, available roles, and actual capacity. If no native tool is exposed, proceed with the sequential fallback below.
+2. Spawn one agent with a standby-only instruction: wait for the refreshed task packet and do not read or write product files. Inspect the actual tool result before spawning more agents.
+3. Only when a real child ID is returned, register it with `run.py start` if using a run plan, then dispatch the refreshed packet through the actual follow-up tool. Reuse this child for its planned task and count it against capacity; no separate throwaway probe is needed.
+
+If the result contains `collab spawn failed: no thread with id`, stop further spawn attempts in that session. Preserve the full error and client/version as evidence. Retrying each role or switching to a built-in role does not address this missing-thread failure. Without a returned child ID, do not call `run.py start` or invent an agent ID. Capacity errors are a separate case: inspect actual session states and use supported lifecycle controls before a justified retry.
+
+`codex exec --ephemeral` avoids persisting session rollout files to disk. This is the documented flag meaning and was confirmed by installed **Codex CLI 0.154.0** help on **2026-09-14**; it is not a guarantee of native subagent compatibility. The [CLI reference](https://learn.chatgpt.com/docs/developer-commands?surface=cli) and [subagent reference](https://learn.chatgpt.com/docs/agent-configuration/subagents) do not establish that combination's compatibility. Known ephemeral mode is a reason to check the first spawn carefully; the Python helpers do not detect the parent CLI mode.
+
+The [verification record](verification.md) contains the historical **0.153.4** ephemeral failure and ordinary-session success, plus a bounded **0.154.0** ephemeral probe that reported success without independent spawn metadata. These observations neither establish universal incompatibility nor identify a fixed version.
+
+If native spawning is unavailable or fails, the parent can complete suitable substantive work sequentially in the existing session and report that mode. Preserve the user's persistence preference; do not silently restart without `--ephemeral`. An ordinary invocation is an option when persistent sessions are acceptable; see the [quickstart](quickstart.md#2-open-the-project-in-a-new-session). Keep native tasks that never started pending and record parent work separately, without synthetic IDs or native results. Native-only smoke checks remain `failed` for an observed failure or `not_run` for unexecuted checks; sequential work does not make that smoke pass.
 
 ## Local helpers and native runtime actions
 
@@ -70,4 +86,4 @@ Read-only agents remain read-only and ask the parent to record important message
 
 Keep the output of `python3 scripts/validate.py --project .` and the automated test suite separate from a live smoke result. For runtime reports, include client/version, project root, effective subagent settings, discovered roles, observed concurrent sessions, and the parent integration result. Follow the [quickstart live smoke procedure](quickstart.md#live-multi-agent-smoke-test).
 
-The [verification record](verification.md) documents a successful two-custom-agent read workflow on Codex CLI 0.153.4 and a failed ephemeral-session probe. There is no maintained cross-version matrix or performance benchmark. Static validation alone still does not imply a live runtime pass.
+The [verification record](verification.md) distinguishes historical live results from the more limited current probe. There is no maintained cross-version matrix or performance benchmark. Static validation alone still does not imply a live runtime pass.

docs/multi-agent.md

@@ -64,13 +64,13 @@ Read scope can overlap. Write ownership must be disjoint while tasks execute con
 
 1. Audit the existing harness and clarify only information needed to proceed.
 2. Establish a compact dependency graph and interface contracts.
-3. Spawn independent tasks up to the available subagent capacity while continuing useful parent work.
+3. Check native compatibility with the first planned standby agent and inspect its actual spawn result before wider fan-out. With a returned child ID, register and reuse that agent, then spawn further independent tasks within actual capacity while continuing useful parent work.
 4. Reuse an agent for related follow-ups with refreshed inputs, decisions, and skill paths. Register the actual native agent ID and its observed lifecycle separately from task status. Close unnecessary sessions with the actual runtime tool and record the observed outcome.
 5. Inspect each result, reconcile interfaces, and integrate before dispatching dependent tasks.
 6. Review and test the integrated change. Route confirmed defects back to the owning worker, then recheck the affected behavior.
 7. Report completed work, actual evidence, and remaining limitations.
 
-The project requests `max_concurrent_threads_per_session = 3` under `[agents]`; these are subagent threads coordinated by the parent session. The effective runtime can impose a lower limit. If a spawn fails due to capacity, inspect actual session states, collect pending results, and use the runtime's supported lifecycle tools before retrying. Waiting or marking a local task complete does not itself release native capacity. If tools are unavailable, use sequential parent execution and report that limitation. Do not abandon required work or manufacture tool names.
+The project requests `max_concurrent_threads_per_session = 3` under `[agents]`; these are subagent threads coordinated by the parent session. The effective runtime can impose a lower limit. If a spawn fails due to capacity, inspect actual session states, collect pending results, and use the runtime's supported lifecycle tools before retrying. Waiting or marking a local task complete does not itself release native capacity. Exposed tools and discovered roles do not guarantee successful spawning. For `collab spawn failed: no thread with id`, stop further spawn attempts in that session, including retries with different roles. Follow the [compatibility preflight](compatibility.md#native-spawn-preflight-and-ephemeral-sessions). Do not abandon required substantive work or manufacture tool names.
 
 Five role definitions do not mean five simultaneous agents. Spawn only concrete tasks that gain something from independent execution. Workers do not spawn their own agents. Hierarchical decomposition stays in the parent's dependency graph, with leaf tasks scheduled within the same bounded pool.
 
@@ -80,7 +80,7 @@ Use the [runtime guide](../skills/harness/references/runtime-guide.md) to initia
 
 Independent tasks receive small, self-contained packets. Related follow-ups reread the changed packet, files, and decisions. An independent reviewer can use fresh context when supported; do not assume a full conversation fork or a particular fork flag exists.
 
-The parent creates new native agents with a standby-only instruction, registers their actual IDs through `start` in `agents.json`, then dispatches the refreshed packet through the actual follow-up tool. A failed local registration prevents task dispatch. `run.py result` records task evidence but does not mark the native agent idle. Confirm actual idleness or a stop acknowledgment before reassigning write ownership; `stop_requested` alone is insufficient. Use the native follow-up tool to resume work and the native lifecycle tool to stop or close a session. The local CLI records observations; it does not perform those native actions.
+The parent creates new native agents with a standby-only instruction to wait for the refreshed packet without reading or writing product files, registers their actual IDs through `start` in `agents.json`, then dispatches the packet through the actual follow-up tool. Without a returned native child ID, do not call `run.py start`; a synthetic ID or the parent's ID cannot stand in for a child. A failed local registration prevents task dispatch. `run.py result` records task evidence but does not mark the native agent idle. Confirm actual idleness or a stop acknowledgment before reassigning write ownership; `stop_requested` alone is insufficient. Use the native follow-up tool to resume work and the native lifecycle tool to stop or close a session. The local CLI records observations; it does not perform those native actions.
 
 `run.py resume` creates a new run, preserves prior records, and invalidates changed inputs/results plus affected descendants. `validate.py --project . --run ID --complete` checks required result evidence, artifacts, and freshness before the parent reports completion. Neither operation proves native execution.
 
@@ -96,6 +96,8 @@ Events append under `_workspace/communications/<run-id>.jsonl` through the helpe
 
 A blocked child returns the exact dependency or failure and any useful partial findings. The parent resolves the dependency, narrows the task, or finishes it locally. Retry only when the input or environment has changed enough to justify another attempt. A child saying “complete” does not satisfy acceptance criteria without evidence.
 
+If native tools are unavailable or the first spawn fails, the parent can complete suitable substantive work sequentially in the current session and report the limitation. Keep native tasks that never started pending and record parent work separately; do not use invented IDs or submit parent work as native child results to make completion validation pass. Required native-only checks remain `failed` or `not_run` according to the evidence, so this fallback cannot pass a native smoke test. Preserve an explicit ephemeral-session preference; ordinary persistent execution is an option only when that persistence is acceptable. The [compatibility guide](compatibility.md#native-spawn-preflight-and-ephemeral-sessions) separates historical ephemeral failures from current evidence.
+
 When concurrent edits collide, stop assigning overlapping writes, inspect the combined diff, and preserve both contributors' intent. The parent owns reconciliation. Do not reset shared files to a prior version or overwrite the workspace to recover one task's result.
 
 ## Boundaries and verification

docs/quickstart.md

@@ -27,6 +27,14 @@ Open the project you want to configure, rather than the downloaded marketplace d
 codex -C /absolute/path/to/project
 ```
 
+For a non-interactive ordinary session:
+
+```bash
+codex exec -C /absolute/path/to/project '$harness Build a harness for this project. Use subagents for independent tasks after checking the first native spawn.'
+```
+
+These examples keep your configured model and permissions. Single quotes preserve the literal `$harness` prompt in the shell. If you chose `--ephemeral` to avoid persistent session files, keep that choice and follow the [native spawn preflight](compatibility.md#native-spawn-preflight-and-ephemeral-sessions); do not silently switch to an ordinary session.
+
 Start a new session after installation, then send:
 
 ```text
@@ -75,6 +83,7 @@ If you already have this checkout, run only the last two commands from its root.
 - **Marketplace or plugin not found:** run `codex plugin marketplace list` and `codex plugin list --marketplace codex-harness --available --json`. Confirm the source includes `.agents/plugins/marketplace.json`; use the local-checkout route for unpublished changes.
 - **`harness` is missing:** confirm the plugin is installed and enabled, then start a new session in the target project. In the desktop app, use `@` to select the plugin or bundled skill. [Official plugin usage](https://learn.chatgpt.com/docs/plugins).
 - **`.agents/skills/harness/scripts/...` is missing:** those project-local paths come from the optional project installer. Ask the plugin to use its installed helper paths, or install the project scaffold below.
+- **`collab spawn failed: no thread with id`:** stop native fan-out and per-role retries in that session. Record the error and CLI version; no returned child ID means no `run.py start`. Follow the [preflight and fallback guidance](compatibility.md#native-spawn-preflight-and-ephemeral-sessions), preserving any explicit `--ephemeral` preference.
 
 ## Use this checkout
 
@@ -133,7 +142,9 @@ python3 .agents/skills/harness/scripts/run.py --project . ready --run project-v1
 python3 .agents/skills/harness/scripts/run.py --project . status --run project-v1
 ```
 
-The parent first creates a native agent with a standby-only instruction, uses its real ID for `start`, then sends the refreshed packet through the actual follow-up tool. It registers actual results with `result`, and separately records confirmed session idleness or termination with `agent`. Important questions, answers, findings, and handoffs are explicitly recorded under `_workspace/communications/`; native delivery remains a separate tool call. Read-only roles ask the parent to log and route messages.
+The parent first creates one native agent with a standby-only instruction and inspects the returned result before wider fan-out. Only a successful spawn with a real child ID permits `start`, followed by delivery of the refreshed packet through the actual follow-up tool. Reuse that agent for its planned task within actual capacity. A missing-thread error stops retries; leave unstarted native tasks pending and record sequential parent work separately. See the [preflight](compatibility.md#native-spawn-preflight-and-ephemeral-sessions).
+
+The parent registers actual results with `result`, and separately records confirmed session idleness or termination with `agent`. Important questions, answers, findings, and handoffs are explicitly recorded under `_workspace/communications/`; native delivery remains a separate tool call. Read-only roles ask the parent to log and route messages.
 
 ```bash
 python3 .agents/skills/harness/scripts/communication.py --project . --run project-v1 view \
@@ -145,21 +156,28 @@ A later input change can start a new run through `run.py resume --run project-v1
 
 ## Live multi-agent smoke test
 
-Static validation cannot establish runtime discovery or parallel execution. In a new trusted local Codex session, ask:
+Static validation and exposed tools cannot establish native spawning or parallel execution. In a new trusted local Codex session, ask:
 
 ```text
-Use harness_explorer and harness_architect as two separate subagents in parallel.
+First spawn one harness_explorer with a standby-only instruction: wait for the
+task packet and do not read or write product files. Inspect its actual result
+before starting another agent. If no child ID is returned, report the error and
+stop this native smoke. For "collab spawn failed: no thread with id", do not retry
+other roles in this session.
+On success, reuse that child for the Explorer task below, then start a separate
+harness_architect for the Architect task within actual capacity. Dispatch both
+independent tasks before waiting for their results.
 Explorer: inspect the harness entry points and cite the relevant files.
 Architect: inspect the task ownership protocol and return a small improvement plan
 in your response only. Both tasks are read-only and independent.
 Have the parent combine the two results. Do not modify files.
 ```
 
-Check that two native subagent sessions start, each completes its assigned task, and the parent combines their results. Record the Codex client/version, effective configuration, discovered role names, observed overlap, returned evidence, and any errors. An instruction to run in parallel is not proof that the runtime did so.
+Check that the first spawn returns a real child ID, two native subagent sessions start, each completes its assigned task, and the parent combines their results. Record the Codex client/version, session mode if known, effective configuration, discovered role names, observed overlap, returned evidence, and any errors. An instruction to run in parallel is not proof that the runtime did so.
 
 For write-path verification, use a disposable project and explicitly assign separate temporary files to two workers. Verify both expected contents, unchanged sentinel files outside ownership, parent integration, and reviewer/QA results. Task packets and a completion checklist are in [multi-agent.md](multi-agent.md).
 
-If roles are missing, confirm the project root and trust settings, validate the files, then start a new thread. If subagent tools are unavailable or capacity is exhausted, have the parent complete the work sequentially and report the reduced execution mode.
+If roles are missing, confirm the project root and trust settings, validate the files, then start a new thread. Distinguish missing roles, unavailable tools, capacity limits, and the missing-thread error using the [compatibility guide](compatibility.md#native-spawn-preflight-and-ephemeral-sessions). The parent can perform suitable substantive work sequentially in the existing session, preserving the user's persistence preference. Native-only checks stay `failed` or `not_run` as observed; sequential fallback is not a successful native smoke.
 
 ## Repository checks