Skip to content

Register poller-driven descriptors in a uring file table and use IOSQE_FIXED_FILE - #215

Draft
resetius wants to merge 1 commit into
masterfrom
uring-fixed-files
Draft

resetius wants to merge 1 commit into
masterfrom
uring-fixed-files

Conversation

@resetius

@resetius resetius commented Nov 16, 2026 •

Copy link
Copy Markdown
Owner

Experiment 4 of the TUring write-path series (draft, for measurement; the owner decides).

Problem

In the rpi5 profile on master 85489f0, fget + fput per SQE are the largest uring-only item: 10.4% / 6.6% / 12.9% of the cycles (546 / 219 / 553 cycles/hop) on single N=1024 / N=16 / chain100. epoll has nothing like it; fdget in a single-threaded process takes no reference.

Change

  • The ring registers a sparse file table: one slot per descriptor number, up to 64Ki slots (8 bytes each in the kernel), at most RLIMIT_NOFILE. Without it (before Linux 5.19, or a limit) the poller uses plain descriptors as before.
  • TUring::Register(fd) puts the file into its slot, io_uring_register_files_update. TPollerDrivenSocket already called Register(). TPollerDrivenFileHandle now calls RegisterFile() when the poller has it; only TUring has it, so TIOCp does not change.
  • Operations on a registered descriptor are posted with IOSQE_FIXED_FILE; the slot is the descriptor number.
  • Close: the slot holds a reference to the file, so PrepClose() frees the slot with close_direct.
    • With pending operations the chain is cancel by fd → close_direct → close, all HARDLINK. The cancel by fd also matches fixed-file requests, because it compares files.
    • Without them, close_direct and then close.
    • The minimum SQ size goes from 2 to 3 for the three-entry chain.
  • Limitation: the table keeps the file open until the poller closes the descriptor. A registered descriptor closed behind the poller's back stays open in the kernel (for example, the peer of a pipe would not see EOF). Raw TUring users that never call Register() are not affected.

Tests

  • New test_uring_registered_file_closed: closing a registered pipe end through the handle gives the reader EOF, and the next file with the same number gets its own slot. Without the close_direct it fails (the reader never sees the end).
  • The existing close and cancel tests now run on fixed files (sockets register), for example test_uring_close_cancels_only_its_operations (cancel by fd of fixed-file requests) and test_uring_closes_on_small_queues (it caught the 2-entry SQ).

A/B (rpi5)

rpi5 (Cortex-A76, Linux 7.3.0-1003-raspi, liburing 2.15, GCC 15.3), Release, examples/bench on taskset -c 3, with run_ab.sh. Median (min-max) of 7 repetitions for single N=1024 and of 5 for N=16 and chain100 (-n 1024 -a 100). Cycles/hop = cycles:u + cycles:k of -w 100000 minus -w 100.

Run 2: master 85489f0, #214 608f9e8 (deferred task work), #215 51879eb (fixed files), #214 + #215 dbf3d2b, #214 + #213 eebf0d1 (direct write). Same method; heads alternate in each repetition.

shape path µs/hop cycles/hop user kernel total / master
single N=1024 uring master 2.114 (2.088-2.166) 562 4530 5078 (5015-5163) 1.000
single N=1024 uring #214 1.972 (1.954-1.987) 555 4176 4737 (4685-4949) 0.933
single N=1024 uring #215 2.090 (2.058-2.107) 574 4447 5022 (4913-5146) 0.989
single N=1024 uring #214 + #215 1.964 (1.915-1.976) 564 4167 4732 (4595-4875) 0.932
single N=1024 uring #214 + #213 2.067 (2.029-2.100) 597 4373 4967 (4871-5111) 0.978
single N=1024 epoll master 1.661 (1.652-1.692) 408 3585 3992 (3965-4081) 1.000
single N=1024 epoll #214 / #215 / #214 + #215 / #214 + #213 1.669 / 1.677 / 1.680 / 1.661 423 / 403 / 425 / 402 3627 / 3643 / 3617 / 3609 4048 / 4046 / 4036 / 4001 1.014 / 1.013 / 1.011 / 1.002
single N=16 uring master 1.372 (1.366-1.393) 395 2907 3297 (3281-3357) 1.000
single N=16 uring #214 1.241 (1.239-1.258) 391 2589 2978 (2970-3024) 0.903
single N=16 uring #215 1.316 (1.302-1.322) 408 2755 3165 (3125-3176) 0.960
single N=16 uring #214 + #215 1.205 (1.200-1.219) 401 2494 2889 (2888-2928) 0.876
single N=16 uring #214 + #213 1.364 (1.350-1.368) 425 2844 3271 (3239-3287) 0.992
single N=16 epoll master 1.219 (1.214-1.222) 343 2583 2926 (2919-2936) 1.000
single N=16 epoll #214 / #215 / #214 + #215 / #214 + #213 1.221 / 1.217 / 1.231 / 1.218 349 / 339 / 348 / 340 2582 / 2579 / 2605 / 2585 2931 / 2921 / 2954 / 2927 1.002 / 0.998 / 1.009 / 1.000
chain100 uring master 1.719 (1.710-1.747) 331 3872 4203 (4112-4290) 1.000
chain100 uring #214 1.726 (1.703-1.763) 352 3778 4130 (4087-4312) 0.983
chain100 uring #215 1.625 (1.618-1.660) 331 3604 3933 (3866-4073) 0.936
chain100 uring #214 + #215 1.649 (1.630-1.669) 360 3577 3936 (3922-4103) 0.937
chain100 uring #214 + #213 1.824 (1.798-1.884) 340 4083 4420 (4375-4563) 1.052
chain100 epoll master 1.414 (1.407-1.435) 312 3103 3415 (3380-3520) 1.000
chain100 epoll #214 / #215 / #214 + #215 / #214 + #213 1.428 / 1.423 / 1.451 / 1.425 314 / 311 / 325 / 299 3128 / 3122 / 3169 / 3120 3433 / 3434 / 3494 / 3421 1.005 / 1.006 / 1.023 / 1.002

Local actor bench (ping_actors, 4 nodes, inflight 1024, 10M messages, 5 repetitions, timeout 60 s per run), msg/s, median (min-max), no hangs:

head uring epoll
master 3 140 060 (2 834 870-3 161 820) 2 638 130 (2 607 150-2 703 770)
#214 3 126 840 (2 912 330-3 202 830) 2 622 220 (2 555 710-2 692 790)
#215 3 052 120 (3 007 500-3 161 370) 2 609 410 (2 539 140-2 642 180)
#214 + #215 3 119 500 (2 883 150-3 191 260) 2 609 590 (2 579 490-2 681 700)
#214 + #213 3 061 730 (2 945 120-3 126 350) 2 618 180 (2 562 770-2 760 790)

The actor bench does not tell the heads apart: they are all within each other's spread.

This PR alone: 0.989 / 0.960 / 0.936 of master. Stacked on #214: 0.932 / 0.876 / 0.937 (#216).

  • The gain is smaller than the fget/fput share of the profile: part of that share is the first cache miss on struct file, which moves to the next access.
  • It is largest on chain100, with 100 operations per enter.

Checks

  • rpi5 (Linux 7.3, liburing 2.15, GCC 15.3), Debug + ASan as in CI: ctest 4/4; test_uring* 19/19.
  • macOS (Debug + ASan): ctest 4/4.

@github-actions

Copy link
Copy Markdown

Unit Test Results (Linux)

334 tests  +1   334 ✅ +1   23s ⏱️ ±0s
  4 suites ±0     0 💤 ±0 
  4 files   ±0     0 ❌ ±0 

Results for commit 51879eb. ± Comparison against base commit 85489f0.

@github-actions

Copy link
Copy Markdown

Unit Test Results (MacOS)

254 tests  ±0   254 ✅ ±0   31s ⏱️ -1s
  4 suites ±0     0 💤 ±0 
  4 files   ±0     0 ❌ ±0 

Results for commit 51879eb. ± Comparison against base commit 85489f0.

@github-actions

Copy link
Copy Markdown

Unit Test Results (Windows)

175 tests  ±0   175 ✅ ±0   44s ⏱️ ±0s
  4 suites ±0     0 💤 ±0 
  4 files   ±0     0 ❌ ±0 

Results for commit 51879eb. ± Comparison against base commit 85489f0.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant