Repository navigation
Conversation
…atched ring returns, frame prefetch
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Depends on #210 (stacked: the diff of this PR is the last commit only). TUring only: it does not touch IOCP (#211) or the readiness backends.
At single N=1024 the read-ahead of #210 saved kernel cycles but lost them in user space. A
perf record -e cycles:uprofile showed cache misses, not instructions:OnBuffered(55% of its samples on the first load);WakeupReadyHandleson the indirect call).Changes:
TBufferedFdisalignas(64). The first queued buffer is kept inline as (id, size, offset), with a vector only for the rest. ATReadBufferis made only whenReadBuffer()takes it; theReadSome()adapter copies from the raw buffer and gives it straight back.Wait(), instead of oneio_uring_buf_ring_advanceper lease. A count of the buffers in the ring decides, after-ENOBUFS, whether to re-arm at once (buffers came back meanwhile) or to wait for the next flush.Tests
rpi5, Linux 7.3, Debug + ASan, head 44f8d58:
test_read_buffer_*35/35, three runs in a row.-ENOBUFS;test_read_buffer_consume(TUring)caught it.-ENOBUFS.A/B
rpi5 (Cortex-A76, Linux 7.3.0-1003-raspi, liburing 2.15, GCC 15.3, Release,
taskset -c 3), 5 alternating repetitions, median (min-max). master 85489f0, #210 fa03c20, this PR 44f8d58. Method as in #210: cycles/hop fromperf statof-w 100000minus-w 100.single N=1024:
Plain
ReadSome()on epoll (no code change; regression check):The uring
ReadSome()row of this PR (1.011) is the bench binary, not the library. #210 measured its library with master'sbench.cppat 1.00; the reader coroutine is the same.single N=16:
chain100 N=1024 (
-a 100):Result
ReadBuffer()is now 3.3% below masterReadSome()(Read ahead on TUring with multishot reads into a buffer ring for ReadBuffer() #210: 3.4% above); the adapter is 4.4% below.ReadBuffer()) and -17.1% (adapter); epoll is 4% ahead of the adapter there.IOU_PBUF_RING_INC(several small reads into one buffer) and a smallerBufferSize. The remaining misses are the state and frame lines, not the buffer lines, so neither should change much.ReadBuffer()/master is 0.866 (single N=1024), 0.874 (N=16) and 0.776 (chain100). This PR is not measured on x86 yet:run_ab.shwithHEAD=origin/read-buffer-uring-optrepeats it.ReadSome()), rpi5, 7 repetitions, msg/s: 10 nodes, master 971 737 / Read ahead on TUring with multishot reads into a buffer ring for ReadBuffer() #210 1 012 150 / this PR 1 019 310; 2 nodes, master 7 067 880 / Read ahead on TUring with multishot reads into a buffer ring for ReadBuffer() #210 7 478 340 / this PR 7 573 910. No regression.