Repository navigation
proactor, tcp, tls: a stop ends what waits on the proactor, so a parked handler unwinds, and a call from another thread after it returns instead of waiting forever (#29) - #53
Merged
Conversation
…ed handler unwinds, and a call from another thread after it returns instead of waiting forever (#29) The teardown marks each live connection closed before its socket goes, which wakes a parked read. A flush, which in kronos is the kernel's the moment it parks, ends once IORING_REGISTER_SYNC_CANCEL by its tag says the kernel is done with the send, a second at most; one it cannot vouch for stays parked, its slab kept. A RingTimer wait still parked completes with -ECANCELED. What the last pass prepared goes to the kernel first, while every socket it names is open. A post from another thread is refused once the stop begins, and the ones taken run in the teardown. Each handoff of kronos's own answers a refusal as a closed connection would: a read returns closed, a flush drops what was staged, a pipe's read and flush return completed, a RingTimer wait returns ECANCELED, and submitTimeout throws. Proactor.post drops the task. The stop lets go of each live connection, freeing it once its handler has let go too. A handler that lets go later, from another thread, frees it then: at the teardown's end, or where it is once the teardown is over.
…ingTimer wait, and to calls from another thread after it (#29) Each handler runs its finally and its connection is freed; a read, a flush, a pipe's read and flush, a RingTimer wait, a raw submitTimeout and a post made from another thread after the stop each return, throw or drop as their docs say; and a handler that lets go after the stop frees its connection. TlsStopParkedTest makes the calls through TlsDuplexPipe. All 8 fail on main.
…r by a handler it wakes, reaches no other socket, and a stop no longer frees live connections (#29)
…ore a socket closes and sends what is queued first, and posts close last, with no gate on a post from another thread (#29) A send a handler woken by the stop queued on a connection still open stayed in the SQ when its socket closed, and a later submit (a growing op table) sent it to whatever had taken the number. Every connection is now marked closed first, then the SQ is submitted, then the sockets close. Posts close at the end of the teardown with a volatile flag. A post from another thread that finds it set drains the queue itself, one task taken at a time under a lock only ever taken then: a Handoff (a read, a flush or a wait its caller waits on) runs and answers as on a closed connection, and any other post is dropped. The gate's compare-and-set and decrement on every post from another thread are gone, and so is the freeing of live connections at stop, which keep their memory.
…one a handler still holds, and a recycle into a full pool frees the connection right there (#29)
…to the pool, which its last step frees (#29) Once every wake is done, each live connection whose handler has let go goes back to the pool as a recycle would: its own buffer ring unregistered while the ring is open, cleared, and freed right there if the pool is full. The teardown's freeMemory then frees the pool. One a handler still holds, or whose send the kernel may still be reading, keeps its memory: a write into it would land in freed memory, and clear() would reopen it to a read from another thread.
MDA2AV
added a commit
that referenced
this pull request
Nov 11, 2026
…ptions (#54) count their services, so the last to stop frees the context With #54 every proactor started with one TlsOptions instance shares its context, so the first proactor to stop could no longer free it under the rest. TlsOptions also keeps the ALPN list the context's callback reads and a count of the services on it, under the lock #54 builds it under: start raises the count, free lowers it, and the one that reaches zero frees the context and the list and clears them, so a start after that builds another. Each service still frees its own kTLS handoff buffer, and teardown frees the services after #53's drain, last of all. Tests: TlsResumptionTest runs one-proactor servers on one port through a stop, a later start, and a start after the last stopped; the heap test runs two proactors sharing one options instance.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Closes #29.
A stop now ends what was waiting on the proactor, instead of leaving it parked forever. This follows MDA2AV/ioxide#555, ioxide's fix for the same bug, except where kron's code needs something different.
What was wrong
Teardown closed the sockets of live connections and left the connections themselves alone. Nothing resumed a handler parked in
read(),flush()or aRingTimerwait, so itsfinallynever ran. After the stop, a call from another thread posted to a queue nothing would drain again, and waited forever.What changed
submitFlushclaims it in the same call, so marking the connection closed releases none of them. Each one ends onceIORING_REGISTER_SYNC_CANCEL, keyed by the send's tag, says the kernel is done with the send.completeFlush(), notfinishSend(). WithcloseAfterSendset,finishSend()would recycle the connection and close its socket number a second time.RingTimerwaits still parked complete with-ECANCELED(RingTimer.ECANCELED). The kernel copies a timeout's deadline when it takes the SQE, so these need no cancel, unlike ioxide's ring ops that carry buffers. A handler that starts another wait from there stays parked.ReactorSwitch: every call from another thread goes throughpost.MpscQueuehas a single consumer.Handoff. The drain runs those, and drops every other post, as ioxide drops its own.read()returns closed, andflush()drops what was staged, both as on any closed connection.read()andflush()return completed. The TLS pipe's do so without reaching the session, whose BoringSSL is the proactor's.RingTimer.delayreturnsECANCELED, and a rawsubmitTimeout's completion gets-ECANCELED.Proactor.postdrops the task, and so the fire-and-forget calls (decRefand the like) are dropped too.clear()would reopen it to a read from another thread.closeListenersmarks the listeners closed. Otherwise a handler woken by the teardown that callsstopAcceptingwould close their numbers again.run()returns.On the hot path, a post from another thread reads one volatile flag after its enqueue. The check that tells a handoff from a post runs only once the proactor has stopped, and posts from the proactor thread itself are unchanged. Everything else is in the teardown.
Tests
StopParkedTestis incommonTest, so it runs on both runtimes. It has eleven tests:RingTimerwait, or in a flush its client holds up runs itsfinally, and its connection is freed.RingTimerwait, and of one that let go, have their slab freed, and their own buffer ring in incremental mode. A connection whose handler is busy outside kronos at the stop keeps its slab.dup2) receives nothing when the ring is entered.RingTimerwait, a rawsubmitTimeoutand a post, each made from another thread after the stop, return or drop as their docs say.PoolFullTestrecycles two connections into a pool of one: the second has its slab freed right there, while the proactor runs.TlsStopParkedTest(native) makes the after-the-stop calls throughTlsDuplexPipe.IoUringLayoutTestchecks the new opcode and the sync-cancel struct against the kernel header.Taking out each part of this version fails its own tests (native):
RingTimercalls from another threadTlsStopParkedTest: BoringSSL refuses the other threadPoolFullTest: neither connection freedMarking connections closed, ending flushes and completing parked waits are unchanged from the first version, where taking out each one failed its own test.
With the change:
tlsmodule.