Repository navigation
Conversation
…w thread given that one's managed id is taken for the reactor's (#608) BindReactorThread records the reactor thread's managed id and sets t_current, and nothing took either back at the stop. Every test of whether a caller is on the reactor compares that id: OnReactorThread, ScheduleOnReactor, SubmitClientOp, SubmitTimeout, the Enqueue fast paths and UdpSendTo. So after a stop two kinds of thread took the reactor's own path against a reactor whose ring the teardown had unmapped: the thread that ran it, if it went on running after Run returned, and any new thread the runtime gave the same id once that thread's Thread object was collected. From either, a ring op read the SQ head from the unmapped ring and the process aborted with AccessViolationException, and a ReactorSwitch.Yield was queued where nothing drains it and never resumed. The thread that ran it also still had the stopped reactor as Reactor.Current. The end of Teardown, still on the reactor's thread, now sets the recorded id to -1 and clears t_current. A thread's id is its own while it runs, and Teardown runs on the reactor's thread before Run returns, so from then on no thread is taken for the reactor's. -1 is no thread's id, and it is not 0, which the submit and UDP checks read as not started, so every caller hands off and gets the stopped reactor's answer. The checks compile as before. A ring the kernel refuses still returns from Run before Teardown, and that path is unchanged. Comparing t_current with the reactor instead costs each check: a thread static outside CoreLib is reached through the thread's array of static bases, a bounds test, two loads and a null test, where the managed id is read at a fixed offset. OnReactorThread compiled to 81 bytes against 47, and ScheduleOnReactor, EnqueueReturnQ and EnqueueFlush each 21 to 28 bytes longer. AffinityTests (E2E): once Run has returned, the thread that ran a reactor is not taken for its reactor's, and neither is a new thread the runtime gives the stopped reactor thread's id; from both, a yield and a ring op throw ObjectDisposedException.
This was referenced Nov 14, 2026
Merged
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Fixes #608.
Touches core: src/ioxide/Reactor/Reactor.Runner.cs (the end of Teardown) and src/ioxide/Reactor/Reactor.Post.cs (XML docs only), because whether a caller is on the reactor is the reactor's own state, read by every satellite through
OnReactorThreadandReactor.Current: only the reactor can take it back when its thread stops being the reactor's.What was wrong.
BindReactorThreadrecords the reactor thread's managed id and setst_current, and nothing took either back at the stop. Every test of whether a caller is on the reactor compares that id:OnReactorThread(which the TLS pipes, the http2, http3, nghttp2 and nghttp3OffReactortests, httpclient, pg and Kestrel'sOnReactorbranch on),ScheduleOnReactor,SubmitClientOp,SubmitTimeout,EnqueueReturnQ,EnqueueRecycle,EnqueueFlush,EnqueueReturnQIncrementalandUdpSendTo. After a stop two kinds of thread still passed, and took the reactor's own path against a reactor whose ring the teardown had unmapped and whose buffers it had freed:Runreturns (a host's own thread, a pool thread): the id andReactor.Currentboth still matched;Threadobject has been collected: CoreCLR hands a dead thread's managed id to a new thread.Reproduced at main 19aec5c with a standalone program, on a reactor with no transport, run on a thread, stopped and joined:
Runreturned:OnReactorThreadtrue,Reactor.Currentthe stopped reactor, andReactorSwitch.Yieldnever resumed (3 of 3).OnReactorThreadwas true, andReactorSwitch.Yieldnever resumed.SubmitTimeoutfrom either: the process aborted withAccessViolationExceptioninRing.GetSqe, which read the SQ head from the ring the teardown unmapped (2 of 2 each).The other paths from reading:
EnqueueRecyclerunsRecyclethere, which submits a cancel to the same ring,EnqueueReturnQwrites through the buffer ring pointer the teardown freed and set to null, andUdpSendToallocates its send blocks again and sends on a descriptor number the stop closed.The change. The end of Teardown, still on the reactor's thread, sets the recorded id to -1 and clears
t_current. A thread's id is its own while it runs, and Teardown runs on the reactor's thread beforeRunreturns (on every path past the ring's creation, see What to weigh), so from then on no thread is taken for the reactor's: not the one that ran it, nor a new one given its id later. -1 is no thread's id, and it is not 0, whichSubmitClientOp,SubmitTimeoutandUdpSendToread as "not started", so every caller takes the path any other thread already took after a stop and gets the stopped reactor's answer. The checks themselves are unchanged.Why the id stays. The issue proposed comparing
t_currentwith the reactor. I built that first, and it works (E2E 732 passed, 0 failed), but it costs every check: a thread static outside CoreLib is reached through the thread's array of static bases (a bounds test, two loads and a null test), where the managed id is read at a fixed offset. Tier-1 code, fromDOTNET_JitDisasm:t_currentcomparedOnReactorThreadScheduleOnReactorEnqueueReturnQEnqueueFlushInvalidating the id gives the same guarantee on every path through Teardown, and with it the tier-1 code of those four methods is byte-identical to main's (addresses aside), so there was nothing for a wrk A/B to measure. A pinned microbenchmark of the check through a non-inlined call could not tell the
t_currentvariant from main on this box either (2.0 to 3.4 ns per call for both, with other suites running).What a caller notices. Only after a stop, and only on the two kinds of thread above, which now behave as every other thread already did:
OnReactorThreadis false, and on the thread that ran itReactor.Currentis null. The XML docs of both now say so.ReactorSwitch.Yield(r)throwsObjectDisposedExceptioninstead of never resuming.await new ReactorSwitch(r)already threw it on main; it now arrives through the thread pool instead of synchronously.SubmitConnect,SubmitSend,SubmitRecv,SubmitRead,SubmitWriteandSubmitTimeoutthrowObjectDisposedExceptioninstead of reaching the unmapped ring.UdpSendTothrows theInvalidOperationExceptionit throws on any other thread.ScheduleOnReactordrops the post, as for any other thread (one posted the way a switch posts runs on the thread pool), andEnqueueReturnQ,EnqueueFlushandEnqueueRecycletake their queue paths.OffReactortests, theDisposeof httpclient's pool and client, pg's queries, pool pick andDispose, and Kestrel'sOnReactor.Disposeof httpclient's pool or client, or of a pg connection, made from the thread that ran the reactor afterRunreturned is now dropped, as one from any other thread already was: it posts itself, and a stopped reactor drops a plain post. It used to run inline there.Tests. In tests/Ioxide.Tests.E2E/Core/AffinityTests.cs, both on a reactor with no transport:
Runreturns, checksOnReactorThreadandReactor.Current, then yields to the reactor and submits a timeout to it.the thread that ran it passes OnReactorThread for the stopped reactort_currentclear reverted:Reactor.Current on the thread that ran it is still the stopped reactora new thread given its id 33 passes OnReactorThread for the stopped reactort_currentclear reverted it passes, as it should: a new thread'st_currentis null.Both make the yield and the ring op only from a thread neither check takes for the reactor's, since on the reactor's path the ring op aborts the process. The revert runs were of AffinityTests alone, out of the built E2E suite (12 passed and 2 failed with the id reset reverted, or both parts; 13 and 1 with the
t_currentclear reverted).Suites. Release, pinned to the safe CPUs, one at a time:
BindReactorThread)tlsmodule)The build adds no warnings.
Overlaps.
git merge-treeagainst every open PR, nghttp2: a request says whether a body follows its HEADERS, so a host need not guess from content-length (#581) #594 to http2: an answer waiting on the client's credit has its unread upload credited, so a client that uploads everything first gets it (#588) #604 and reactor: a buffer return that finds its queue full after the stop is dropped, rather than spinning forever on a queue nothing drains (#607) #619 to reactor: a ring op submitted from a completion the stop gives fails as one handed over after the stop does, rather than never completing (#609) #621: no conflicts. Only reactor: a stop leaves the shared receive slab to the last handler still holding a connection, so a read after it gets the bytes queued before it, not freed memory (#605) #620 touches a file of this one, Reactor.Runner.cs, in the middle of Teardown, away from its end.HopDuplexPipetestsReactor.Current != _reactorto tell a pump resumed after the stop. With this, that also holds on the thread that ran the reactor.!OnReactorThread, which the two kinds of thread above now take too.EnqueueReturnQandEnqueueReturnQIncremental. With this, the two kinds of thread above reach that path too, instead of writing through the freed buffer ring pointer.EnqueueRecycle. With this, every thread takes its queue path after a stop, so a fix there covers the thread that ran the reactor too.ObjectDisposedExceptioninAllocOpSloteven without this change; their posts, theEnqueuepaths,UdpSendToand the satellites' checks would still take the reactor's path. With both, those threads hand off andHandOffthrows first.What to weigh.
Runreturns before Teardown, so there the thread that ran it still passes (checked on this branch). Nothing was mapped or freed, so its path ends in aNullReferenceExceptionfrom a ring op rather than in freed memory. That path also never closes the posts: aReactorSwitch.Yieldfrom another thread never resumes, with or without this. Taking the binding back there alone would make a switch from that thread wait forever too, so it needs its posts closed as well, which is a change of its own._reactorThreadIdnow has three values: 0 beforeRun, the thread's id while it runs, -1 after Teardown. One test already sets it to -1 to mean that every caller is another thread (AffinityTests' handoff wake count).CurrentManagedThreadId == reactorThreadId) is still accurate, so it is unchanged.