FEATURE-ci-v3-runtime-green: tolerate borrowed-handle closure and enforce drain deadlines

Author: etr

ChangeLog

@@ -1,4 +1,8 @@
 Version 3.0.0 - 2026-10-10
+	Keep the HTTP/1 connection watchdog armed at a pending server drain
+		deadline so a stalled close handoff cannot outlive the drain budget.
+	Tolerate a concurrently closed borrowed registration in the Windows
+		reference host poll helper instead of failing the whole WSAPoll set.
 	Register each native executable once in Automake test evidence inventories.
 	Run Windows allocation-failure injection against the installed static
 		library and require ordinary readiness cases in both linkage lanes.

docs/external-loop-contract.md

@@ -121,6 +121,15 @@ the resulting protocol/transport outcome. Duplicate or spurious notifications,
 including events with all flags false, are permitted and do not imply additional
 completions. Empty batches are permitted for timers and pending library work.
 
+A registration borrowed by a snapshot can be closed by library work at any
+moment, because retaining a snapshot never extends a handle lifetime. A POSIX
+host observes this as a per-entry `POLLNVAL` readiness (translated to `error`
+and ignored as stale). Windows `WSAPoll` instead rejects the entire call with
+`WSAENOTSOCK`; a host must treat that failure like an empty batch — dispatch,
+acquire a fresh snapshot, and wait again — rather than stopping, because the
+closed registration is absent from every later snapshot. The reference
+`examples/host_poll.hpp` helper implements this tolerance.
+
 Obtain an initial snapshot before waiting. After **every successful dispatch**,
 obtain and reconcile a fresh snapshot before waiting again. A failed dispatch
 may have consumed part of a batch unless the documented rejection rule says

examples/host_poll.hpp

@@ -102,6 +102,13 @@ inline std::vector<srv::readiness_event> wait(
     const auto ready = ::WSAPoll(fds.data(), static_cast<ULONG>(fds.size()), timeout(deadline));
     if (ready < 0) {
         const int error = ::WSAGetLastError();
+        // Snapshot handles are borrowed: the library may close one between
+        // snapshot acquisition and this call. POSIX poll reports that
+        // per entry as POLLNVAL; WSAPoll rejects the whole set with
+        // WSAENOTSOCK. An empty batch keeps the contract's spurious-
+        // notification allowance: the caller dispatches, re-acquires
+        // interests, and the closed registration is already gone.
+        if (error == WSAENOTSOCK) return {};
         if (error != WSAEINTR) throw std::runtime_error("WSAPoll failed (WSA error " + std::to_string(error) + ")");
     }
 #else

specs/tasks/FEATURE-ci-v3-runtime-green.md

@@ -128,3 +128,14 @@ The duplicated names are http_cookie, consumer_v3_cookie, consumer_v3_cookie_wir
 Evidence: primary `.debug/v3-ci-native-repairs-PJFEkn/native-inventory/`. Autoreconf succeeded with existing AC_OUTPUT/.PHONY warnings. Initial config.status invocation used the source directory instead of its configured build directory and failed before any native execution; rechecking/regenerating in the correct build directory fixed setup. No unrelated source or generated tracked file changed.
 
 Current9c92 Windows package job122646921257 fails earlier at the public shared external consumer: exit6, captured stderr WSAPoll error10038. The selected820-byte process receipt was verified within a closed2,581,790-byte range assessment; no full artifact or original program ran. It contains no failed-call slot/state values. The earlier approved external4 capture remains CLOSED/PASS4/52/nonreproduction, and does not authorize running the distinct public consumer. No guessed polling or shutdown suppression was applied. A fresh consumer-specific bounded capture proposal is needed if further native diagnosis is required.
+
+## Stage 7: Windows WSAPoll borrowed-closure tolerance and HTTP/1 drain-deadline watchdog (2026-10-30)
+
+- [x] Correlate all retained Windows failures (external four-test suite twice, public shared consumer exit6) with the approved DIAG capture: the host loop dies on "WSAPoll failed (WSA error 10038)" while dispatch never failed and stopping was false.
+- [x] Root-cause by contract analysis, not a new capture: `interest_snapshot` handles are borrowed and "retaining a snapshot does not extend any handle lifetime"; the library closes registrations under its registry mutex and never republishes dead records, so WSAENOTSOCK can only mean a snapshot handle closed between snapshot and WSAPoll entry (handle-value reuse makes it WSAENOTSOCK on busy runners and POLLNVAL on idle probes, reconciling the closed local probes).
+- [x] Repair `examples/host_poll.hpp`: WSAENOTSOCK returns an empty batch (the contract's spurious-notification allowance); every reference caller re-acquires interests after each dispatch, and dead records are pruned from the next snapshot, so the loop provably progresses. All other WSAPoll failures still throw; POSIX POLLNVAL handling is unchanged. Document the host obligation in `docs/external-loop-contract.md`.
+- [x] Root-cause the 2026-10-28 macOS/Darwin external4 drain stall (idle keep-alive connection never closed after `begin_drain`; host silent for the full 10s budget; remaining 1): `watchdog_terminal_locked()` treated the quiesce close mark as terminal, but HTTP/1 enforces the server drain deadline only through the watchdog (`enforce_timeout` is the sole HTTP/1 `scope_.expire_if_due` caller), so a stalled close handoff had no deadline enforcement left.
+- [x] Repair: the close mark no longer retires the watchdog while a server drain deadline is pending; the deadline joins `plan_watchdog_locked` candidates through the existing `add_protocol_deadlines_locked` entry and fires through the unchanged global-expiry path. Deadline-less closes (request_close, response-Connection: close) stay terminal as before.
+- [x] RED/GREEN regression `quiesce_with_drain_deadline_keeps_watchdog_armed` (unstarted engine, deterministic plan sample; deadline-less case pins unchanged terminal behavior) plus behavioral `quiesce_with_drain_deadline_still_closes_promptly`. RED verified against unfixed source, GREEN against the repair, five consecutive clean runs.
+- [x] Linux focused gates: connection_engine 42/0 (x5), external_http1_e2e 4/0, websocket_drain 3/0, http1_websocket_upgrade 17/0, native_http1_e2e 62/0, external_readiness_adapter 28/0, examples/external_event_loop builds and serves; changed-file cpplint and `git diff --check` pass.
+- [ ] Full exact-source CI on the working branch (mingw64/macOS/FreeBSD lanes remain nonlocal and deferred to CI).

src/detail/connection_engine_websocket.cpp

@@ -227,7 +227,13 @@ connection_engine::io_posture connection_engine::writer_posture(
     return io_posture::park;
 }
 bool connection_engine::watchdog_terminal_locked() const {
-    return shutdown_ || phase_ == stream_phase::terminal || (close_after_drain_ && !websocket_);
+    // The close mark alone must not retire the watchdog while a server
+    // drain deadline is pending: HTTP/1 enforces that deadline's hard
+    // stop only here (enforce_timeout's scope expiry), so a stalled
+    // close handoff may not outlive the drain budget. The transport's
+    // release still ends this loop with connection_closed.
+    return shutdown_ || phase_ == stream_phase::terminal
+        || (close_after_drain_ && !websocket_ && !server_drain_deadline_);
 }
 bool connection_engine::transport_output_pending_locked() const {
     if (outbox_.queued_bytes() > 0) return true;

test/unit/connection_engine_test.cpp

@@ -68,6 +68,7 @@
 #include <parity/response_frame.hpp>
 
 #include "./io_loopback.hpp"
+#include "./websocket_engine_helpers.hpp"
 #include "./littletest.hpp"
 
 namespace {
@@ -301,7 +302,7 @@ struct scenario {
     explicit scenario(srv::timeout_options timeouts_in)
         : scenario(srv::budget_limits(), timeouts_in) { }
 
-    bool start_engine() {
+    bool create_engine() {
         if (!pair.ok()) return false;
         backend.adopt_connection(kConnId, pair.detach_local());
         engine = std::make_shared<connection_engine>(backend, pool, registry,
@@ -310,6 +311,11 @@ struct scenario {
                                                     [this] {
                                                         stopped.store(true);
                                                     });
+        return true;
+    }
+
+    bool start_engine() {
+        if (!create_engine()) return false;
         engine->start();
         return true;
     }
@@ -1215,6 +1221,45 @@ LT_BEGIN_AUTO_TEST(connection_engine_suite, quiesce_idle_connection_closes_promp
     LT_CHECK(reaches_eof(s.pair.peer()));
 LT_END_AUTO_TEST(quiesce_idle_connection_closes_promptly)
 
+// TASK-110 pin: a quiesce that carries a server drain deadline keeps the
+// watchdog armed at that deadline. HTTP/1 enforces the drain budget's
+// hard stop only through the watchdog (enforce_timeout's scope expiry),
+// so the close mark alone must not retire it while the deadline is
+// pending -- a stalled close handoff may not outlive the drain budget.
+// Loops stay unstarted so the plan is sampled before any close path can
+// complete; a deadline-less quiesce stays terminal exactly as before.
+LT_BEGIN_AUTO_TEST(connection_engine_suite, quiesce_with_drain_deadline_keeps_watchdog_armed)
+    srv::timeout_options timeouts;
+    timeouts.header = std::chrono::milliseconds(10000);
+    scenario s(timeouts);
+    LT_CHECK(s.create_engine());
+    const auto drain_deadline = std::chrono::steady_clock::now() + std::chrono::seconds(2);
+    s.engine->quiesce(drain_deadline);
+    const auto armed = httpserver::detail::connection_engine_test_access::deadline(*s.engine);
+    LT_CHECK(armed.has_value());
+    if (armed) LT_CHECK(*armed <= drain_deadline);
+    scenario plain;
+    LT_CHECK(plain.create_engine());
+    plain.engine->quiesce();
+    LT_CHECK(!httpserver::detail::connection_engine_test_access::deadline(*plain.engine).has_value());
+LT_END_AUTO_TEST(quiesce_with_drain_deadline_keeps_watchdog_armed)
+
+// The armed drain deadline must not delay the ordinary close chain: a
+// started engine quiesced with a far drain budget still ends (and the
+// transport closes) promptly, as in the deadline-less case above.
+LT_BEGIN_AUTO_TEST(connection_engine_suite, quiesce_with_drain_deadline_still_closes_promptly)
+    srv::timeout_options timeouts;
+    timeouts.header = std::chrono::milliseconds(10000);
+    scenario s(timeouts);
+    LT_CHECK(s.start_engine());
+    const auto began = std::chrono::steady_clock::now();
+    s.engine->quiesce(std::chrono::steady_clock::now() + std::chrono::seconds(10));
+    LT_CHECK(wait_until([&s] { return s.stopped.load(); }));
+    const auto elapsed = std::chrono::steady_clock::now() - began;
+    LT_CHECK(elapsed < std::chrono::milliseconds(5000));
+    LT_CHECK(reaches_eof(s.pair.peer()));
+LT_END_AUTO_TEST(quiesce_with_drain_deadline_still_closes_promptly)
+
 // TASK-110: the counted units bracket the exchange -- the handler runs
 // between the connection unit and its own exchange unit (2 live), and
 // the count returns to zero once the engine finalized.