Skip to content

Publish a reproducible, generated benchmark report - #377

Open
sgerbino wants to merge 3 commits into
developfrom
pr/benchmark-report
Open

Publish a reproducible, generated benchmark report#377
sgerbino wants to merge 3 commits into
developfrom
pr/benchmark-report

Conversation

@sgerbino

@sgerbino sgerbino commented Oct 2, 2026

Copy link
Copy Markdown
Collaborator

Closes #352.

The published benchmark page is now fully generated from fresh runs on the three dedicated machines, replacing the hand-written February report.

What's here

  • .github/workflows/benchmark-report.yml — dispatch-only workflow that runs the full comparison suite (corosio / asio coroutines / asio callbacks) on the Linux, Windows, and macOS runners and uploads raw per-iteration JSON plus an environment capture. Linux builds asio twice (epoll and io_uring reactors) so every corosio configuration is compared against a same-reactor asio.
  • .github/bench/report_page.py — stdlib-only generator: median-across-iterations aggregation with CV-based noise classification, theme-aware inline SVG charts (diverging bars vs the asio-coroutines baseline), four Antora pages (methodology landing + per-platform Summary / Test Environment / Results).
  • bench/README.md — how to run the suite, dispatch the workflow, and regenerate the pages.
  • Scenario descriptions in the harness (describe() / category_description()) rendered as category intros and chart hover text.
  • Generated pages + charts from run 36946358854 (7 iterations × 2 s, commit 90bf46a): Linux epoll median +1.3%, io_uring +2.1%, Windows IOCP +1.4%, macOS kqueue +60.3% vs Boost.Asio coroutines. The old benchmark-report.adoc URL stays alive via a page alias.

Bench fixes surfaced by production runs

  • The asio local-socket suites registered on Windows but local::connect_pair relies on socketpair(), which asio stubs out as operation_not_supported there — the thrown error aborted every full-suite run. They are now POSIX-only, matching the corosio suites.
  • Churn benchmarks RST-closed only the client side; on loopback a FIN racing the RST could still park connections in TIME_WAIT, which at production duration overflowed the macOS PCB table mid-benchmark. Both sides now RST-close, in all three implementations.

Reproducing

Dispatch benchmark-report from the Actions tab (defaults: 7 × 2 s), gh run download the artifacts, run report_page.py, review the Antora build, open a PR — see bench/README.md.

@sgerbino
sgerbino force-pushed the pr/benchmark-report branch from 5947101 to 5e36210 Compare October 2, 2026 03:31
@cppalliance-bot

cppalliance-bot commented Oct 2, 2026

Copy link
Copy Markdown

GCOVR code coverage report https://377.corosio.prtest3.cppalliance.org/gcovr/index.html
LCOV code coverage report https://377.corosio.prtest3.cppalliance.org/genhtml/index.html
Coverage Diff Report https://377.corosio.prtest3.cppalliance.org/diff-report/index.html

Build time: 2026-10-02 03:41:39 UTC

@cppalliance-bot

cppalliance-bot commented Oct 2, 2026 •
edited
Loading

Copy link
Copy Markdown

GCOVR code coverage report https://377.corosio.prtest3.cppalliance.org/gcovr/index.html
LCOV code coverage report https://377.corosio.prtest3.cppalliance.org/genhtml/index.html
Coverage Diff Report https://377.corosio.prtest3.cppalliance.org/diff-report/index.html

Build time: 2026-10-02 22:17:12 UTC

@github-actions

github-actions Bot commented Oct 2, 2026 •
edited
Loading

Copy link
Copy Markdown

Benchmark report

a80ce002fd8c (base) vs 99008d23794f (head)

  • ✅ linux — clean (epoll, uring)
  • ⚠️ windows — 2 flagged (iocp)
  • ✅ macos — clean (kqueue)

⚠️ Flagged

Platform Backend Benchmark Metric Base Head Δ Noise
windows iocp accept_churn/burst/100 ops_per_sec 152.2 ops/s 149.1 ops/s 🔴▼ -2.52% 0.60%
windows iocp accept_churn/burst_lockless/100 ops_per_sec 156.7 ops/s 151.9 ops/s 🔴▼ -3.02% 0.71%
linux — full results (226 benchmarks, 5 iterations, 1.5s each)

accept_churn

Benchmark Backend Metric Base Head Δ Noise
burst/10 epoll ops_per_sec 1.86K ops/s 1.88K ops/s 🟢▲ +0.78% 0.91%
burst/10 uring ops_per_sec 1.72K ops/s 1.70K ops/s 🟢▲ +0.08% 1.25%
burst/100 epoll ops_per_sec 187.7 ops/s 189.4 ops/s 🟢▲ +0.35% 0.99%
burst/100 uring ops_per_sec 174.5 ops/s 173.5 ops/s 🔴▼ -0.09% 0.76%
burst_lockless/10 epoll ops_per_sec 1.88K ops/s 1.89K ops/s 🟢▲ +0.25% 0.95%
burst_lockless/10 uring ops_per_sec 1.83K ops/s 1.82K ops/s 🔴▼ -0.26% 0.75%
burst_lockless/100 epoll ops_per_sec 189.1 ops/s 190.4 ops/s 🟢▲ +0.26% 0.88%
burst_lockless/100 uring ops_per_sec 187.0 ops/s 185.6 ops/s 🔴▼ -0.39% 0.77%
concurrent/1 epoll ops_per_sec 14.86K ops/s 14.89K ops/s 🟢▲ +0.21% 0.50%
concurrent/1 uring ops_per_sec 12.70K ops/s 12.68K ops/s 🟢▲ +0.13% 1.08%
concurrent/16 epoll ops_per_sec 14.85K ops/s 14.97K ops/s 🟢▲ +0.38% 0.92%
concurrent/16 uring ops_per_sec 12.54K ops/s 12.56K ops/s 🟢▲ +0.37% 1.30%
concurrent/4 epoll ops_per_sec 14.87K ops/s 14.90K ops/s 🟢▲ +0.36% 0.45%
concurrent/4 uring ops_per_sec 12.66K ops/s 12.63K ops/s 🟢▲ +0.36% 1.08%
sequential epoll ops_per_sec 14.83K ops/s 14.84K ops/s 🟢▲ +0.07% 0.53%
sequential uring ops_per_sec 12.76K ops/s 12.70K ops/s 🔴▼ -0.40% 1.03%
sequential_lockless epoll ops_per_sec 14.96K ops/s 14.96K ops/s 🔴▼ -0.14% 0.55%
sequential_lockless uring ops_per_sec 13.68K ops/s 13.62K ops/s 🔴▼ -1.24% 1.03%

fan_out

Benchmark Backend Metric Base Head Δ Noise
concurrent_parents/1 epoll ops_per_sec 3.68K ops/s 3.70K ops/s 🔴▼ -0.01% 0.99%
concurrent_parents/1 uring ops_per_sec 3.45K ops/s 3.44K ops/s 🔴▼ -0.02% 1.20%
concurrent_parents/16 epoll ops_per_sec 3.57K ops/s 3.59K ops/s 🟢▲ +0.20% 0.98%
concurrent_parents/16 uring ops_per_sec 3.32K ops/s 3.30K ops/s 🔴▼ -0.24% 0.47%
concurrent_parents/4 epoll ops_per_sec 3.59K ops/s 3.60K ops/s 🟢▲ +0.11% 0.97%
concurrent_parents/4 uring ops_per_sec 3.36K ops/s 3.33K ops/s 🔴▼ -0.03% 1.18%
concurrent_parents_lockless/1 epoll ops_per_sec 3.72K ops/s 3.73K ops/s 🟢▲ +0.28% 0.23%
concurrent_parents_lockless/1 uring ops_per_sec 3.66K ops/s 3.65K ops/s 🔴▼ -0.21% 0.34%
concurrent_parents_lockless/16 epoll ops_per_sec 3.61K ops/s 3.61K ops/s 🔴▼ -0.06% 0.14%
concurrent_parents_lockless/16 uring ops_per_sec 3.53K ops/s 3.51K ops/s 🔴▼ -0.41% 0.37%
concurrent_parents_lockless/4 epoll ops_per_sec 3.62K ops/s 3.62K ops/s 🔴▼ -0.03% 0.15%
concurrent_parents_lockless/4 uring ops_per_sec 3.55K ops/s 3.55K ops/s 🔴▼ -0.42% 0.43%
fork_join/1 epoll ops_per_sec 54.57K ops/s 54.93K ops/s 🟢▲ +0.20% 0.92%
fork_join/1 uring ops_per_sec 53.22K ops/s 53.18K ops/s 🔴▼ -0.26% 0.33%
fork_join/16 epoll ops_per_sec 3.69K ops/s 3.68K ops/s 🟢▲ +0.01% 0.29%
fork_join/16 uring ops_per_sec 3.43K ops/s 3.43K ops/s 🔴▼ -0.23% 0.27%
fork_join/4 epoll ops_per_sec 14.62K ops/s 14.66K ops/s 🟢▲ +0.14% 0.74%
fork_join/4 uring ops_per_sec 13.73K ops/s 13.72K ops/s 🔴▼ -0.17% 0.33%
fork_join/64 epoll ops_per_sec 897.2 ops/s 896.8 ops/s 🟢▲ +0.04% 0.21%
fork_join/64 uring ops_per_sec 812.8 ops/s 812.5 ops/s 🔴▼ -0.12% 0.17%
fork_join_lockless/1 epoll ops_per_sec 55.32K ops/s 55.54K ops/s 🟢▲ +0.42% 0.30%
fork_join_lockless/1 uring ops_per_sec 56.00K ops/s 55.69K ops/s 🔴▼ -0.61% 0.24%
fork_join_lockless/16 epoll ops_per_sec 3.72K ops/s 3.73K ops/s 🟢▲ +0.09% 0.18%
fork_join_lockless/16 uring ops_per_sec 3.67K ops/s 3.66K ops/s 🔴▼ -0.43% 1.04%
fork_join_lockless/4 epoll ops_per_sec 14.78K ops/s 14.81K ops/s 🟢▲ +0.19% 0.16%
fork_join_lockless/4 uring ops_per_sec 14.64K ops/s 14.52K ops/s 🔴▼ -0.53% 1.08%
fork_join_lockless/64 epoll ops_per_sec 907.5 ops/s 907.0 ops/s 🔴▼ -0.25% 0.22%
fork_join_lockless/64 uring ops_per_sec 870.3 ops/s 867.6 ops/s 🔴▼ -0.05% 1.04%
nested/16 epoll ops_per_sec 891.0 ops/s 895.0 ops/s 🟢▲ +0.17% 1.05%
nested/16 uring ops_per_sec 809.4 ops/s 805.0 ops/s 🟢▲ +0.01% 1.38%
nested/4 epoll ops_per_sec 3.66K ops/s 3.67K ops/s 🟢▲ +0.29% 0.57%
nested/4 uring ops_per_sec 3.43K ops/s 3.40K ops/s 🔴▼ -0.25% 1.47%
nested_lockless/16 epoll ops_per_sec 900.4 ops/s 904.4 ops/s 🟢▲ +0.11% 0.93%
nested_lockless/16 uring ops_per_sec 860.7 ops/s 856.4 ops/s 🔴▼ -0.24% 1.19%
nested_lockless/4 epoll ops_per_sec 3.70K ops/s 3.72K ops/s 🟢▲ +0.20% 1.07%
nested_lockless/4 uring ops_per_sec 3.65K ops/s 3.62K ops/s 🔴▼ -0.32% 1.22%

http_server

Benchmark Backend Metric Base Head Δ Noise
concurrent/1 epoll ops_per_sec 55.10K ops/s 55.11K ops/s 🔴▼ -0.05% 0.45%
concurrent/1 uring ops_per_sec 53.62K ops/s 53.73K ops/s 🟢▲ +0.29% 0.27%
concurrent/16 epoll ops_per_sec 57.75K ops/s 57.81K ops/s 🟢▲ +0.01% 0.25%
concurrent/16 uring ops_per_sec 53.22K ops/s 53.04K ops/s 🔴▼ -0.07% 0.98%
concurrent/32 epoll ops_per_sec 57.33K ops/s 57.44K ops/s 🟢▲ +0.25% 0.20%
concurrent/32 uring ops_per_sec 53.14K ops/s 52.60K ops/s 🔴▼ -0.33% 1.37%
concurrent/4 epoll ops_per_sec 57.16K ops/s 57.21K ops/s 🟢▲ +0.08% 0.16%
concurrent/4 uring ops_per_sec 53.11K ops/s 53.21K ops/s 🟢▲ +0.20% 0.24%
multithread/1 epoll ops_per_sec 57.23K ops/s 57.40K ops/s 🟢▲ +0.41% 0.26%
multithread/1 uring ops_per_sec 53.10K ops/s 52.68K ops/s 🔴▼ -0.28% 1.31%
multithread/16 epoll ops_per_sec 161.6K ops/s 162.1K ops/s 🟢▲ +0.05% 0.61%
multithread/16 uring ops_per_sec 74.91K ops/s 75.17K ops/s 🟢▲ +0.54% 19.53%
multithread/2 epoll ops_per_sec 86.61K ops/s 87.19K ops/s 🟢▲ +0.84% 0.32%
multithread/2 uring ops_per_sec 90.67K ops/s 90.44K ops/s 🔴▼ -0.12% 1.02%
multithread/4 epoll ops_per_sec 157.3K ops/s 158.7K ops/s 🔴▼ -0.47% 2.29%
multithread/4 uring ops_per_sec 79.40K ops/s 79.11K ops/s 🔴▼ -0.69% 1.14%
multithread/8 epoll ops_per_sec 161.0K ops/s 161.2K ops/s 🔴▼ -0.17% 0.54%
multithread/8 uring ops_per_sec 77.16K ops/s 77.23K ops/s 🟢▲ +0.01% 0.19%
single_conn epoll ops_per_sec 55.20K ops/s 54.99K ops/s 🔴▼ -0.30% 1.05%
single_conn uring ops_per_sec 53.56K ops/s 53.70K ops/s 🟢▲ +0.29% 0.13%
single_conn_lockless epoll ops_per_sec 55.97K ops/s 55.71K ops/s 🔴▼ -0.22% 1.03%
single_conn_lockless uring ops_per_sec 56.18K ops/s 56.35K ops/s 🟢▲ +0.32% 0.24%

local_socket_latency

Benchmark Backend Metric Base Head Δ Noise
concurrent/1 epoll latency_mean_ns 7.12 µs 7.25 µs 🟢▲ +0.17% 4.28%
concurrent/1 uring latency_mean_ns 6.86 µs 6.73 µs 🟢▲ +0.29% 5.00%
concurrent/16 epoll latency_mean_ns 110.10 µs 110.03 µs 🔴▼ -0.22% 1.02%
concurrent/16 uring latency_mean_ns 108.19 µs 108.68 µs 🔴▼ -0.01% 0.93%
concurrent/4 epoll latency_mean_ns 27.87 µs 27.62 µs 🟢▲ +0.06% 1.91%
concurrent/4 uring latency_mean_ns 27.11 µs 27.24 µs 🔴▼ -0.78% 0.89%
concurrent_lockless/1 epoll latency_mean_ns 7.14 µs 7.12 µs 🔴▼ -0.14% 4.57%
concurrent_lockless/1 uring latency_mean_ns 6.56 µs 6.83 µs 🟢▲ +0.10% 5.43%
concurrent_lockless/16 epoll latency_mean_ns 108.51 µs 108.23 µs 🔴▼ -0.06% 0.95%
concurrent_lockless/16 uring latency_mean_ns 108.30 µs 107.98 µs 🟢▲ +0.58% 0.82%
concurrent_lockless/4 epoll latency_mean_ns 27.45 µs 27.24 µs 🟢▲ +1.88% 1.28%
concurrent_lockless/4 uring latency_mean_ns 27.01 µs 26.93 µs 🟢▲ +0.56% 1.95%
pingpong/1 epoll latency_mean_ns 7.06 µs 7.18 µs 🟢▲ +0.34% 4.37%
pingpong/1 uring latency_mean_ns 6.65 µs 6.52 µs 🟢▲ +0.84% 2.62%
pingpong/1024 epoll latency_mean_ns 7.90 µs 8.00 µs 🟢▲ +0.43% 3.99%
pingpong/1024 uring latency_mean_ns 7.38 µs 7.20 µs 🟢▲ +1.01% 4.94%
pingpong/64 epoll latency_mean_ns 7.27 µs 7.37 µs 🟢▲ +0.47% 5.13%
pingpong/64 uring latency_mean_ns 6.89 µs 6.58 µs 🟢▲ +1.10% 5.15%
pingpong_lockless/1 epoll latency_mean_ns 6.96 µs 6.95 µs 🔴▼ -0.09% 0.36%
pingpong_lockless/1 uring latency_mean_ns 6.67 µs 6.52 µs 🟢▲ +0.49% 3.73%
pingpong_lockless/1024 epoll latency_mean_ns 7.40 µs 7.46 µs 🟢▲ +0.13% 4.40%
pingpong_lockless/1024 uring latency_mean_ns 7.21 µs 7.21 µs 🔴▼ -0.11% 4.90%
pingpong_lockless/64 epoll latency_mean_ns 7.01 µs 7.01 µs 🟢▲ +0.03% 0.35%
pingpong_lockless/64 uring latency_mean_ns 6.84 µs 6.70 µs 🟢▲ +0.58% 4.88%

local_socket_throughput

Benchmark Backend Metric Base Head Δ Noise
bidirectional/1024 epoll bytes_per_sec 305.4 MB/s 304.9 MB/s 🟢▲ +0.32% 1.26%
bidirectional/1024 uring bytes_per_sec 302.4 MB/s 303.2 MB/s 🟢▲ +0.14% 0.62%
bidirectional/1048576 epoll bytes_per_sec 3.17 GB/s 3.18 GB/s 🔴▼ -0.25% 2.31%
bidirectional/1048576 uring bytes_per_sec 3.28 GB/s 3.27 GB/s 🟢▲ +1.77% 3.82%
bidirectional/16384 epoll bytes_per_sec 3.08 GB/s 3.01 GB/s 🟢▲ +0.48% 4.88%
bidirectional/16384 uring bytes_per_sec 2.42 GB/s 2.41 GB/s 🔴▼ -0.22% 0.68%
bidirectional/262144 epoll bytes_per_sec 3.16 GB/s 3.13 GB/s 🟢▲ +0.26% 2.91%
bidirectional/262144 uring bytes_per_sec 3.30 GB/s 3.28 GB/s 🔴▼ -1.54% 1.98%
bidirectional/4096 epoll bytes_per_sec 995.4 MB/s 999.7 MB/s 🟢▲ +0.20% 0.61%
bidirectional/4096 uring bytes_per_sec 996.2 MB/s 986.2 MB/s 🔴▼ -0.36% 1.26%
bidirectional/65536 epoll bytes_per_sec 2.95 GB/s 2.94 GB/s 🟢▲ +0.33% 1.86%
bidirectional/65536 uring bytes_per_sec 3.02 GB/s 3.01 GB/s 🔴▼ -0.63% 1.25%
bidirectional_lockless/1024 epoll bytes_per_sec 307.3 MB/s 310.2 MB/s 🟢▲ +0.54% 1.21%
bidirectional_lockless/1024 uring bytes_per_sec 303.9 MB/s 304.4 MB/s 🔴▼ -0.44% 1.29%
bidirectional_lockless/1048576 epoll bytes_per_sec 3.17 GB/s 3.20 GB/s 🔴▼ -0.15% 2.52%
bidirectional_lockless/1048576 uring bytes_per_sec 3.33 GB/s 3.32 GB/s 🔴▼ -1.96% 4.43%
bidirectional_lockless/16384 epoll bytes_per_sec 3.12 GB/s 3.07 GB/s 🟢▲ +1.20% 5.06%
bidirectional_lockless/16384 uring bytes_per_sec 2.64 GB/s 2.64 GB/s 🟢▲ +0.91% 1.97%
bidirectional_lockless/262144 epoll bytes_per_sec 3.16 GB/s 3.14 GB/s 🟢▲ +0.93% 3.52%
bidirectional_lockless/262144 uring bytes_per_sec 3.38 GB/s 3.35 GB/s 🔴▼ -1.66% 2.33%
bidirectional_lockless/4096 epoll bytes_per_sec 1.01 GB/s 1.01 GB/s 🔴▼ -0.34% 1.42%
bidirectional_lockless/4096 uring bytes_per_sec 998.4 MB/s 993.6 MB/s 🔴▼ -1.36% 1.45%
bidirectional_lockless/65536 epoll bytes_per_sec 2.97 GB/s 2.96 GB/s 🟢▲ +0.74% 2.21%
bidirectional_lockless/65536 uring bytes_per_sec 3.19 GB/s 3.22 GB/s 🔴▼ -0.81% 2.22%
unidirectional/1024 epoll bytes_per_sec 296.3 MB/s 298.2 MB/s 🟢▲ +0.39% 1.13%
unidirectional/1024 uring bytes_per_sec 302.3 MB/s 301.8 MB/s 🟢▲ +0.09% 0.64%
unidirectional/1048576 epoll bytes_per_sec 3.27 GB/s 3.28 GB/s 🔴▼ -0.13% 3.20%
unidirectional/1048576 uring bytes_per_sec 3.40 GB/s 3.36 GB/s 🔴▼ -0.91% 4.60%
unidirectional/16384 epoll bytes_per_sec 3.08 GB/s 3.05 GB/s 🟢▲ +0.63% 3.79%
unidirectional/16384 uring bytes_per_sec 2.66 GB/s 2.65 GB/s 🔴▼ -0.21% 1.47%
unidirectional/262144 epoll bytes_per_sec 3.23 GB/s 3.28 GB/s 🟢▲ +2.46% 2.18%
unidirectional/262144 uring bytes_per_sec 3.44 GB/s 3.44 GB/s 🔴▼ -1.08% 2.44%
unidirectional/4096 epoll bytes_per_sec 976.1 MB/s 980.8 MB/s 🟢▲ +0.68% 0.53%
unidirectional/4096 uring bytes_per_sec 997.5 MB/s 994.1 MB/s 🔴▼ -0.34% 0.73%
unidirectional/65536 epoll bytes_per_sec 3.22 GB/s 3.23 GB/s 🔴▼ -0.93% 3.63%
unidirectional/65536 uring bytes_per_sec 3.29 GB/s 3.33 GB/s 🟢▲ +1.93% 2.30%
unidirectional_lockless/1024 epoll bytes_per_sec 298.3 MB/s 303.1 MB/s 🟢▲ +1.28% 1.45%
unidirectional_lockless/1024 uring bytes_per_sec 305.4 MB/s 304.7 MB/s 🔴▼ -0.32% 1.29%
unidirectional_lockless/1048576 epoll bytes_per_sec 3.33 GB/s 3.31 GB/s 🔴▼ -1.16% 3.39%
unidirectional_lockless/1048576 uring bytes_per_sec 3.35 GB/s 3.33 GB/s 🟢▲ +0.42% 4.35%
unidirectional_lockless/16384 epoll bytes_per_sec 3.09 GB/s 3.07 GB/s 🟢▲ +2.01% 3.33%
unidirectional_lockless/16384 uring bytes_per_sec 2.68 GB/s 2.64 GB/s 🔴▼ -0.21% 2.34%
unidirectional_lockless/262144 epoll bytes_per_sec 3.28 GB/s 3.30 GB/s 🟢▲ +1.15% 1.92%
unidirectional_lockless/262144 uring bytes_per_sec 3.41 GB/s 3.38 GB/s 🔴▼ -1.27% 2.76%
unidirectional_lockless/4096 epoll bytes_per_sec 989.2 MB/s 993.7 MB/s 🟢▲ +0.52% 0.77%
unidirectional_lockless/4096 uring bytes_per_sec 994.2 MB/s 996.1 MB/s 🟢▲ +0.30% 1.77%
unidirectional_lockless/65536 epoll bytes_per_sec 3.26 GB/s 3.28 GB/s 🔴▼ -1.40% 3.45%
unidirectional_lockless/65536 uring bytes_per_sec 3.28 GB/s 3.30 GB/s 🔴▼ -0.05% 2.13%

socket_latency

Benchmark Backend Metric Base Head Δ Noise
concurrent/1 epoll latency_mean_ns 14.81 µs 14.79 µs 🟢▲ +0.05% 0.22%
concurrent/1 uring latency_mean_ns 14.46 µs 14.54 µs 🔴▼ -0.07% 1.17%
concurrent/16 epoll latency_mean_ns 232.67 µs 233.55 µs 🟢▲ +0.07% 1.19%
concurrent/16 uring latency_mean_ns 234.36 µs 235.20 µs 🔴▼ -0.03% 1.08%
concurrent/4 epoll latency_mean_ns 58.24 µs 58.26 µs 🔴▼ -0.06% 0.17%
concurrent/4 uring latency_mean_ns 58.15 µs 58.35 µs 🟢▲ +0.04% 1.05%
concurrent_lockless/1 epoll latency_mean_ns 14.62 µs 14.70 µs 🔴▼ -0.38% 1.18%
concurrent_lockless/1 uring latency_mean_ns 14.41 µs 14.52 µs 🔴▼ -0.32% 1.19%
concurrent_lockless/16 epoll latency_mean_ns 230.89 µs 232.47 µs 🔴▼ -0.40% 1.23%
concurrent_lockless/16 uring latency_mean_ns 234.90 µs 234.95 µs 🟢▲ +0.05% 0.17%
concurrent_lockless/4 epoll latency_mean_ns 57.65 µs 58.02 µs 🔴▼ -0.05% 1.23%
concurrent_lockless/4 uring latency_mean_ns 58.23 µs 58.40 µs 🔴▼ -0.20% 0.36%
pingpong/1 epoll latency_mean_ns 14.65 µs 14.75 µs 🔴▼ -0.23% 1.10%
pingpong/1 uring latency_mean_ns 14.41 µs 14.32 µs 🟢▲ +0.32% 1.24%
pingpong/1024 epoll latency_mean_ns 15.06 µs 15.11 µs 🔴▼ -0.41% 0.27%
pingpong/1024 uring latency_mean_ns 14.75 µs 14.73 µs 🔴▼ -0.21% 1.03%
pingpong/64 epoll latency_mean_ns 14.77 µs 14.84 µs 🔴▼ -0.54% 0.24%
pingpong/64 uring latency_mean_ns 14.47 µs 14.42 µs ⚪ 0.00% 1.10%
pingpong_lockless/1 epoll latency_mean_ns 14.61 µs 14.62 µs 🟢▲ +0.02% 0.23%
pingpong_lockless/1 uring latency_mean_ns 14.42 µs 14.44 µs 🟢▲ +0.02% 0.44%
pingpong_lockless/1024 epoll latency_mean_ns 14.96 µs 14.97 µs 🔴▼ -0.08% 0.23%
pingpong_lockless/1024 uring latency_mean_ns 14.69 µs 14.76 µs 🔴▼ -0.05% 1.10%
pingpong_lockless/64 epoll latency_mean_ns 14.70 µs 14.70 µs 🔴▼ -0.16% 0.33%
pingpong_lockless/64 uring latency_mean_ns 14.43 µs 14.52 µs 🔴▼ -0.15% 1.28%

socket_throughput

Benchmark Backend Metric Base Head Δ Noise
bidirectional/1024 epoll bytes_per_sec 133.9 MB/s 134.1 MB/s 🟢▲ +0.15% 0.46%
bidirectional/1024 uring bytes_per_sec 141.1 MB/s 140.7 MB/s 🔴▼ -0.01% 0.91%
bidirectional/1048576 epoll bytes_per_sec 3.33 GB/s 3.24 GB/s 🔴▼ -2.70% 2.24%
bidirectional/1048576 uring bytes_per_sec 3.24 GB/s 3.20 GB/s 🔴▼ -0.78% 1.47%
bidirectional/16384 epoll bytes_per_sec 1.64 GB/s 1.64 GB/s 🟢▲ +0.16% 0.86%
bidirectional/16384 uring bytes_per_sec 1.55 GB/s 1.51 GB/s 🔴▼ -2.29% 1.17%
bidirectional/262144 epoll bytes_per_sec 3.20 GB/s 3.18 GB/s 🔴▼ -0.78% 1.38%
bidirectional/262144 uring bytes_per_sec 3.18 GB/s 3.16 GB/s 🔴▼ -0.69% 0.95%
bidirectional/4096 epoll bytes_per_sec 506.1 MB/s 506.0 MB/s 🟢▲ +0.40% 0.93%
bidirectional/4096 uring bytes_per_sec 495.7 MB/s 492.9 MB/s 🔴▼ -0.40% 0.91%
bidirectional/65536 epoll bytes_per_sec 2.50 GB/s 2.53 GB/s 🟢▲ +0.57% 2.07%
bidirectional/65536 uring bytes_per_sec 2.76 GB/s 2.74 GB/s 🔴▼ -0.75% 2.02%
bidirectional_lockless/1024 epoll bytes_per_sec 135.6 MB/s 134.5 MB/s 🔴▼ -0.12% 1.26%
bidirectional_lockless/1024 uring bytes_per_sec 141.0 MB/s 141.0 MB/s 🟢▲ +0.32% 1.01%
bidirectional_lockless/1048576 epoll bytes_per_sec 3.24 GB/s 3.23 GB/s 🟢▲ +0.71% 1.96%
bidirectional_lockless/1048576 uring bytes_per_sec 3.31 GB/s 3.32 GB/s 🟢▲ +0.41% 1.32%
bidirectional_lockless/16384 epoll bytes_per_sec 1.63 GB/s 1.64 GB/s 🟢▲ +0.99% 1.51%
bidirectional_lockless/16384 uring bytes_per_sec 1.54 GB/s 1.53 GB/s 🔴▼ -0.13% 0.94%
bidirectional_lockless/262144 epoll bytes_per_sec 3.20 GB/s 3.19 GB/s 🔴▼ -0.55% 1.03%
bidirectional_lockless/262144 uring bytes_per_sec 3.22 GB/s 3.23 GB/s 🟢▲ +0.18% 1.19%
bidirectional_lockless/4096 epoll bytes_per_sec 510.0 MB/s 509.2 MB/s 🟢▲ +0.28% 1.37%
bidirectional_lockless/4096 uring bytes_per_sec 492.9 MB/s 495.6 MB/s 🟢▲ +0.55% 0.43%
bidirectional_lockless/65536 epoll bytes_per_sec 2.53 GB/s 2.53 GB/s 🟢▲ +0.28% 2.97%
bidirectional_lockless/65536 uring bytes_per_sec 2.82 GB/s 2.81 GB/s 🟢▲ +0.47% 2.09%
multithread/2 epoll bytes_per_sec 3.41 GB/s 3.42 GB/s 🟢▲ +0.37% 0.79%
multithread/2 uring bytes_per_sec 3.67 GB/s 3.66 GB/s 🔴▼ -0.05% 0.37%
multithread/4 epoll bytes_per_sec 4.84 GB/s 4.86 GB/s 🟢▲ +0.91% 0.86%
multithread/4 uring bytes_per_sec 5.06 GB/s 5.07 GB/s 🟢▲ +0.17% 0.33%
multithread/8 epoll bytes_per_sec 4.95 GB/s 4.95 GB/s ⚪ 0.00% 0.28%
multithread/8 uring bytes_per_sec 5.12 GB/s 5.12 GB/s 🔴▼ -0.11% 0.40%
unidirectional/1024 epoll bytes_per_sec 125.3 MB/s 125.8 MB/s 🟢▲ +0.23% 0.58%
unidirectional/1024 uring bytes_per_sec 138.9 MB/s 140.1 MB/s 🟢▲ +0.11% 1.15%
unidirectional/1048576 epoll bytes_per_sec 3.33 GB/s 3.35 GB/s 🔴▼ -0.54% 1.38%
unidirectional/1048576 uring bytes_per_sec 3.40 GB/s 3.41 GB/s 🔴▼ -0.01% 1.03%
unidirectional/16384 epoll bytes_per_sec 1.55 GB/s 1.56 GB/s 🟢▲ +0.35% 0.97%
unidirectional/16384 uring bytes_per_sec 1.56 GB/s 1.54 GB/s 🔴▼ -0.39% 2.96%
unidirectional/262144 epoll bytes_per_sec 3.31 GB/s 3.32 GB/s 🟢▲ +0.35% 1.13%
unidirectional/262144 uring bytes_per_sec 3.20 GB/s 3.18 GB/s 🔴▼ -0.95% 1.66%
unidirectional/4096 epoll bytes_per_sec 474.3 MB/s 472.7 MB/s 🟢▲ +0.32% 1.24%
unidirectional/4096 uring bytes_per_sec 493.9 MB/s 496.1 MB/s 🔴▼ -0.02% 1.26%
unidirectional/65536 epoll bytes_per_sec 2.73 GB/s 2.60 GB/s 🔴▼ -5.56% 4.75%
unidirectional/65536 uring bytes_per_sec 2.70 GB/s 2.68 GB/s 🔴▼ -2.09% 4.58%
unidirectional_lockless/1024 epoll bytes_per_sec 127.1 MB/s 126.1 MB/s 🟢▲ +0.12% 1.62%
unidirectional_lockless/1024 uring bytes_per_sec 139.3 MB/s 139.8 MB/s 🟢▲ +0.26% 0.54%
unidirectional_lockless/1048576 epoll bytes_per_sec 3.35 GB/s 3.33 GB/s 🔴▼ -0.99% 0.74%
unidirectional_lockless/1048576 uring bytes_per_sec 3.32 GB/s 3.32 GB/s 🔴▼ -0.03% 0.73%
unidirectional_lockless/16384 epoll bytes_per_sec 1.56 GB/s 1.57 GB/s 🟢▲ +0.61% 1.09%
unidirectional_lockless/16384 uring bytes_per_sec 1.61 GB/s 1.60 GB/s 🟢▲ +0.58% 2.41%
unidirectional_lockless/262144 epoll bytes_per_sec 3.34 GB/s 3.32 GB/s 🔴▼ -0.56% 1.13%
unidirectional_lockless/262144 uring bytes_per_sec 3.23 GB/s 3.21 GB/s 🔴▼ -1.11% 1.03%
unidirectional_lockless/4096 epoll bytes_per_sec 476.1 MB/s 473.8 MB/s 🔴▼ -0.14% 1.33%
unidirectional_lockless/4096 uring bytes_per_sec 492.6 MB/s 489.7 MB/s 🔴▼ -0.26% 0.95%
unidirectional_lockless/65536 epoll bytes_per_sec 2.77 GB/s 2.76 GB/s 🔴▼ -0.35% 1.19%
unidirectional_lockless/65536 uring bytes_per_sec 2.65 GB/s 2.71 GB/s 🟢▲ +1.24% 2.23%
windows — full results (77 benchmarks, 5 iterations, 1.5s each)

accept_churn

Benchmark Backend Metric Base Head Δ Noise
burst/10 iocp ops_per_sec 1.56K ops/s 1.53K ops/s 🔴▼ -1.57% 0.33%
burst/100 iocp ops_per_sec 152.2 ops/s 149.1 ops/s 🔴▼ -2.52% 0.60%
burst_lockless/10 iocp ops_per_sec 1.57K ops/s 1.54K ops/s 🔴▼ -1.85% 0.48%
burst_lockless/100 iocp ops_per_sec 156.7 ops/s 151.9 ops/s 🔴▼ -3.02% 0.71%
concurrent/1 iocp ops_per_sec 12.41K ops/s 12.24K ops/s 🔴▼ -1.62% 1.51%
concurrent/16 iocp ops_per_sec 13.33K ops/s 13.25K ops/s 🔴▼ -0.43% 0.34%
concurrent/4 iocp ops_per_sec 13.10K ops/s 12.94K ops/s 🔴▼ -1.23% 0.32%
sequential iocp ops_per_sec 12.37K ops/s 12.14K ops/s 🔴▼ -1.73% 1.58%
sequential_lockless iocp ops_per_sec 12.41K ops/s 12.20K ops/s 🔴▼ -1.81% 1.50%

fan_out

Benchmark Backend Metric Base Head Δ Noise
concurrent_parents/1 iocp ops_per_sec 7.19K ops/s 7.18K ops/s 🔴▼ -0.15% 0.26%
concurrent_parents/16 iocp ops_per_sec 6.42K ops/s 6.43K ops/s 🟢▲ +0.05% 1.27%
concurrent_parents/4 iocp ops_per_sec 6.90K ops/s 6.90K ops/s 🔴▼ -0.21% 0.77%
concurrent_parents_lockless/1 iocp ops_per_sec 7.19K ops/s 7.18K ops/s 🔴▼ -0.18% 0.36%
concurrent_parents_lockless/16 iocp ops_per_sec 6.46K ops/s 6.44K ops/s 🔴▼ -0.23% 1.37%
concurrent_parents_lockless/4 iocp ops_per_sec 6.94K ops/s 6.94K ops/s 🟢▲ +0.12% 0.48%
fork_join/1 iocp ops_per_sec 113.0K ops/s 112.8K ops/s 🔴▼ -0.10% 0.43%
fork_join/16 iocp ops_per_sec 7.18K ops/s 7.14K ops/s 🔴▼ -0.83% 0.69%
fork_join/4 iocp ops_per_sec 29.24K ops/s 29.25K ops/s 🔴▼ -0.06% 0.25%
fork_join/64 iocp ops_per_sec 1.72K ops/s 1.73K ops/s 🟢▲ +0.37% 1.60%
fork_join_lockless/1 iocp ops_per_sec 112.9K ops/s 112.6K ops/s 🔴▼ -0.04% 0.42%
fork_join_lockless/16 iocp ops_per_sec 7.17K ops/s 7.20K ops/s 🟢▲ +0.35% 0.28%
fork_join_lockless/4 iocp ops_per_sec 29.22K ops/s 29.24K ops/s 🟢▲ +0.13% 0.25%
fork_join_lockless/64 iocp ops_per_sec 1.73K ops/s 1.73K ops/s 🟢▲ +0.02% 0.84%
nested/16 iocp ops_per_sec 1.70K ops/s 1.70K ops/s 🔴▼ -0.03% 0.77%
nested/4 iocp ops_per_sec 7.02K ops/s 7.04K ops/s 🟢▲ +0.50% 0.36%
nested_lockless/16 iocp ops_per_sec 1.70K ops/s 1.70K ops/s 🟢▲ +0.25% 0.81%
nested_lockless/4 iocp ops_per_sec 7.03K ops/s 7.02K ops/s 🔴▼ -0.08% 0.16%

http_server

Benchmark Backend Metric Base Head Δ Noise
concurrent/1 iocp ops_per_sec 112.0K ops/s 111.5K ops/s 🔴▼ -0.30% 0.29%
concurrent/16 iocp ops_per_sec 108.7K ops/s 108.0K ops/s 🔴▼ -0.60% 0.26%
concurrent/32 iocp ops_per_sec 108.0K ops/s 107.7K ops/s 🔴▼ -0.37% 0.27%
concurrent/4 iocp ops_per_sec 109.4K ops/s 109.2K ops/s 🔴▼ -0.52% 0.33%
multithread/1 iocp ops_per_sec 107.9K ops/s 107.7K ops/s 🔴▼ -0.19% 0.32%
multithread/16 iocp ops_per_sec 271.0K ops/s 271.3K ops/s 🟢▲ +0.09% 0.66%
multithread/2 iocp ops_per_sec 182.2K ops/s 182.2K ops/s 🟢▲ +0.10% 0.68%
multithread/4 iocp ops_per_sec 227.5K ops/s 228.9K ops/s 🟢▲ +0.66% 0.52%
multithread/8 iocp ops_per_sec 269.9K ops/s 272.5K ops/s 🟢▲ +0.90% 0.43%
single_conn iocp ops_per_sec 111.7K ops/s 111.2K ops/s 🔴▼ -0.28% 0.34%
single_conn_lockless iocp ops_per_sec 111.7K ops/s 111.4K ops/s 🔴▼ -0.21% 0.37%

socket_latency

Benchmark Backend Metric Base Head Δ Noise
concurrent/1 iocp latency_mean_ns 7.61 µs 7.60 µs 🟢▲ +0.28% 0.44%
concurrent/16 iocp latency_mean_ns 133.24 µs 126.02 µs 🟢▲ +5.51% 5.09%
concurrent/4 iocp latency_mean_ns 30.62 µs 30.54 µs 🟢▲ +0.32% 0.29%
concurrent_lockless/1 iocp latency_mean_ns 7.61 µs 7.60 µs 🟢▲ +0.11% 0.44%
concurrent_lockless/16 iocp latency_mean_ns 121.92 µs 122.20 µs 🟢▲ +0.14% 1.02%
concurrent_lockless/4 iocp latency_mean_ns 30.60 µs 30.44 µs 🟢▲ +0.33% 0.38%
pingpong/1 iocp latency_mean_ns 7.59 µs 7.60 µs ⚪ 0.00% 0.61%
pingpong/1024 iocp latency_mean_ns 7.70 µs 7.69 µs 🟢▲ +0.02% 0.40%
pingpong/64 iocp latency_mean_ns 7.62 µs 7.63 µs 🔴▼ -0.12% 0.38%
pingpong_lockless/1 iocp latency_mean_ns 7.59 µs 7.59 µs 🟢▲ +0.09% 0.58%
pingpong_lockless/1024 iocp latency_mean_ns 7.71 µs 7.69 µs 🟢▲ +0.46% 0.48%
pingpong_lockless/64 iocp latency_mean_ns 7.60 µs 7.62 µs 🔴▼ -0.22% 0.31%

socket_throughput

Benchmark Backend Metric Base Head Δ Noise
bidirectional/1024 iocp bytes_per_sec 279.3 MB/s 280.4 MB/s 🟢▲ +0.40% 0.59%
bidirectional/1048576 iocp bytes_per_sec 4.42 GB/s 4.40 GB/s 🔴▼ -0.90% 7.39%
bidirectional/16384 iocp bytes_per_sec 3.71 GB/s 3.70 GB/s 🔴▼ -0.81% 1.15%
bidirectional/262144 iocp bytes_per_sec 9.50 GB/s 9.34 GB/s 🔴▼ -1.17% 3.37%
bidirectional/4096 iocp bytes_per_sec 1.07 GB/s 1.08 GB/s 🟢▲ +0.55% 0.33%
bidirectional/65536 iocp bytes_per_sec 8.89 GB/s 8.75 GB/s 🔴▼ -1.48% 4.13%
bidirectional_lockless/1024 iocp bytes_per_sec 279.2 MB/s 280.8 MB/s 🟢▲ +0.70% 0.51%
bidirectional_lockless/1048576 iocp bytes_per_sec 4.57 GB/s 4.60 GB/s 🔴▼ -1.24% 7.18%
bidirectional_lockless/16384 iocp bytes_per_sec 3.71 GB/s 3.75 GB/s 🟢▲ +1.09% 0.76%
bidirectional_lockless/262144 iocp bytes_per_sec 9.68 GB/s 9.65 GB/s 🟢▲ +0.57% 1.10%
bidirectional_lockless/4096 iocp bytes_per_sec 1.07 GB/s 1.08 GB/s 🟢▲ +0.11% 0.49%
bidirectional_lockless/65536 iocp bytes_per_sec 8.98 GB/s 9.04 GB/s 🟢▲ +1.83% 1.98%
multithread/2 iocp bytes_per_sec 12.71 GB/s 12.77 GB/s 🟢▲ +0.42% 0.73%
multithread/4 iocp bytes_per_sec 17.03 GB/s 17.11 GB/s 🟢▲ +0.09% 1.44%
multithread/8 iocp bytes_per_sec 18.30 GB/s 18.47 GB/s 🟢▲ +0.99% 0.85%
unidirectional/1024 iocp bytes_per_sec 283.0 MB/s 284.6 MB/s 🟢▲ +0.40% 0.51%
unidirectional/1048576 iocp bytes_per_sec 3.88 GB/s 3.92 GB/s 🟢▲ +2.42% 3.75%
unidirectional/16384 iocp bytes_per_sec 3.78 GB/s 3.81 GB/s 🟢▲ +0.98% 0.46%
unidirectional/262144 iocp bytes_per_sec 8.75 GB/s 9.15 GB/s 🟢▲ +3.21% 4.74%
unidirectional/4096 iocp bytes_per_sec 1.09 GB/s 1.09 GB/s 🟢▲ +0.07% 0.39%
unidirectional/65536 iocp bytes_per_sec 9.15 GB/s 9.32 GB/s 🟢▲ +2.15% 1.42%
unidirectional_lockless/1024 iocp bytes_per_sec 283.7 MB/s 283.5 MB/s 🟢▲ +0.09% 0.41%
unidirectional_lockless/1048576 iocp bytes_per_sec 4.21 GB/s 4.21 GB/s 🟢▲ +0.89% 3.70%
unidirectional_lockless/16384 iocp bytes_per_sec 3.76 GB/s 3.81 GB/s 🟢▲ +1.53% 0.71%
unidirectional_lockless/262144 iocp bytes_per_sec 9.63 GB/s 9.68 GB/s 🟢▲ +0.93% 1.22%
unidirectional_lockless/4096 iocp bytes_per_sec 1.09 GB/s 1.09 GB/s 🟢▲ +0.23% 1.24%
unidirectional_lockless/65536 iocp bytes_per_sec 9.20 GB/s 9.40 GB/s 🟢▲ +2.61% 1.64%
macos — full results (113 benchmarks, 5 iterations, 1.5s each)

accept_churn

Benchmark Backend Metric Base Head Δ Noise
burst/10 kqueue ops_per_sec 4.25K ops/s 4.25K ops/s 🟢▲ +0.12% 0.28%
burst/100 kqueue ops_per_sec 430.7 ops/s 431.3 ops/s 🟢▲ +0.18% 0.81%
burst_lockless/10 kqueue ops_per_sec 4.26K ops/s 4.28K ops/s 🟢▲ +0.32% 0.93%
burst_lockless/100 kqueue ops_per_sec 432.2 ops/s 434.4 ops/s 🟢▲ +0.66% 0.54%
concurrent/1 kqueue ops_per_sec 19.59K ops/s 19.41K ops/s 🔴▼ -0.66% 0.76%
concurrent/16 kqueue ops_per_sec 35.24K ops/s 35.35K ops/s 🟢▲ +0.30% 0.87%
concurrent/4 kqueue ops_per_sec 29.14K ops/s 29.48K ops/s 🟢▲ +0.79% 1.27%
sequential kqueue ops_per_sec 19.91K ops/s 19.86K ops/s 🔴▼ -0.17% 0.71%
sequential_lockless kqueue ops_per_sec 19.62K ops/s 19.90K ops/s 🟢▲ +1.47% 0.67%

fan_out

Benchmark Backend Metric Base Head Δ Noise
concurrent_parents/1 kqueue ops_per_sec 7.72K ops/s 7.73K ops/s 🟢▲ +0.01% 1.25%
concurrent_parents/16 kqueue ops_per_sec 6.99K ops/s 7.10K ops/s 🟢▲ +2.42% 19.50%
concurrent_parents/4 kqueue ops_per_sec 8.18K ops/s 8.18K ops/s 🔴▼ -0.12% 2.08%
concurrent_parents_lockless/1 kqueue ops_per_sec 7.77K ops/s 7.77K ops/s 🟢▲ +0.27% 1.19%
concurrent_parents_lockless/16 kqueue ops_per_sec 7.03K ops/s 7.05K ops/s 🔴▼ -0.38% 20.69%
concurrent_parents_lockless/4 kqueue ops_per_sec 8.27K ops/s 8.25K ops/s 🔴▼ -0.19% 1.71%
fork_join/1 kqueue ops_per_sec 46.06K ops/s 45.88K ops/s 🔴▼ -0.18% 0.75%
fork_join/16 kqueue ops_per_sec 7.69K ops/s 7.71K ops/s 🟢▲ +0.19% 0.77%
fork_join/4 kqueue ops_per_sec 21.70K ops/s 21.61K ops/s 🟢▲ +0.16% 0.75%
fork_join/64 kqueue ops_per_sec 2.04K ops/s 2.06K ops/s 🟢▲ +0.39% 2.05%
fork_join_lockless/1 kqueue ops_per_sec 46.94K ops/s 47.05K ops/s 🟢▲ +0.31% 0.18%
fork_join_lockless/16 kqueue ops_per_sec 7.77K ops/s 7.80K ops/s 🟢▲ +0.41% 0.88%
fork_join_lockless/4 kqueue ops_per_sec 21.81K ops/s 21.78K ops/s 🔴▼ -0.18% 0.82%
fork_join_lockless/64 kqueue ops_per_sec 2.06K ops/s 2.07K ops/s 🟢▲ +0.17% 2.02%
nested/16 kqueue ops_per_sec 2.01K ops/s 2.03K ops/s 🟢▲ +0.37% 1.77%
nested/4 kqueue ops_per_sec 7.06K ops/s 7.08K ops/s 🟢▲ +0.27% 0.90%
nested_lockless/16 kqueue ops_per_sec 2.03K ops/s 2.05K ops/s 🟢▲ +0.47% 1.96%
nested_lockless/4 kqueue ops_per_sec 7.19K ops/s 7.20K ops/s 🟢▲ +0.32% 1.15%

http_server

Benchmark Backend Metric Base Head Δ Noise
concurrent/1 kqueue ops_per_sec 49.62K ops/s 49.67K ops/s 🟢▲ +0.08% 0.22%
concurrent/16 kqueue ops_per_sec 128.3K ops/s 127.6K ops/s 🔴▼ -0.40% 0.75%
concurrent/32 kqueue ops_per_sec 128.0K ops/s 128.3K ops/s 🟢▲ +0.91% 1.54%
concurrent/4 kqueue ops_per_sec 88.40K ops/s 87.88K ops/s 🔴▼ -0.23% 0.85%
multithread/1 kqueue ops_per_sec 127.9K ops/s 128.1K ops/s 🟢▲ +0.75% 1.34%
multithread/16 kqueue ops_per_sec 127.5K ops/s 128.1K ops/s 🟢▲ +0.54% 1.28%
multithread/2 kqueue ops_per_sec 177.6K ops/s 173.8K ops/s 🔴▼ -3.04% 1.72%
multithread/4 kqueue ops_per_sec 161.2K ops/s 161.2K ops/s 🔴▼ -0.41% 1.18%
multithread/8 kqueue ops_per_sec 132.2K ops/s 133.0K ops/s 🔴▼ -0.20% 1.28%
single_conn kqueue ops_per_sec 49.62K ops/s 49.61K ops/s 🟢▲ +0.07% 0.40%
single_conn_lockless kqueue ops_per_sec 49.78K ops/s 49.84K ops/s 🟢▲ +0.05% 0.34%

local_socket_latency

Benchmark Backend Metric Base Head Δ Noise
concurrent/1 kqueue latency_mean_ns 2.07 µs 2.06 µs 🟢▲ +0.26% 0.35%
concurrent/16 kqueue latency_mean_ns 30.68 µs 30.54 µs 🟢▲ +0.21% 0.57%
concurrent/4 kqueue latency_mean_ns 7.68 µs 7.66 µs 🟢▲ +0.09% 0.58%
concurrent_lockless/1 kqueue latency_mean_ns 2.04 µs 2.03 µs 🟢▲ +0.18% 0.67%
concurrent_lockless/16 kqueue latency_mean_ns 30.22 µs 30.15 µs 🟢▲ +0.24% 0.24%
concurrent_lockless/4 kqueue latency_mean_ns 7.56 µs 7.53 µs 🟢▲ +0.37% 0.45%
pingpong/1 kqueue latency_mean_ns 2.06 µs 2.06 µs 🟢▲ +0.35% 0.73%
pingpong/1024 kqueue latency_mean_ns 2.16 µs 2.16 µs 🟢▲ +0.02% 0.57%
pingpong/64 kqueue latency_mean_ns 2.06 µs 2.06 µs 🟢▲ +0.14% 0.77%
pingpong_lockless/1 kqueue latency_mean_ns 2.03 µs 2.02 µs 🟢▲ +0.25% 0.78%
pingpong_lockless/1024 kqueue latency_mean_ns 2.14 µs 2.13 µs 🟢▲ +0.13% 0.47%
pingpong_lockless/64 kqueue latency_mean_ns 2.03 µs 2.03 µs 🟢▲ +0.31% 0.58%

local_socket_throughput

Benchmark Backend Metric Base Head Δ Noise
bidirectional/1024 kqueue bytes_per_sec 1.22 GB/s 1.22 GB/s 🔴▼ -0.11% 0.78%
bidirectional/1048576 kqueue bytes_per_sec 3.84 GB/s 3.83 GB/s 🔴▼ -0.33% 0.55%
bidirectional/16384 kqueue bytes_per_sec 3.85 GB/s 3.83 GB/s 🔴▼ -0.26% 0.72%
bidirectional/262144 kqueue bytes_per_sec 3.84 GB/s 3.81 GB/s 🔴▼ -1.09% 0.72%
bidirectional/4096 kqueue bytes_per_sec 2.90 GB/s 2.90 GB/s 🟢▲ +0.26% 0.75%
bidirectional/65536 kqueue bytes_per_sec 3.85 GB/s 3.83 GB/s 🔴▼ -0.71% 0.67%
bidirectional_lockless/1024 kqueue bytes_per_sec 1.23 GB/s 1.23 GB/s 🟢▲ +0.10% 1.14%
bidirectional_lockless/1048576 kqueue bytes_per_sec 3.90 GB/s 3.92 GB/s 🟢▲ +0.58% 0.69%
bidirectional_lockless/16384 kqueue bytes_per_sec 3.90 GB/s 3.91 GB/s 🟢▲ +0.14% 0.85%
bidirectional_lockless/262144 kqueue bytes_per_sec 3.92 GB/s 3.91 GB/s 🔴▼ -0.25% 0.68%
bidirectional_lockless/4096 kqueue bytes_per_sec 2.95 GB/s 2.94 GB/s 🔴▼ -0.21% 0.51%
bidirectional_lockless/65536 kqueue bytes_per_sec 3.89 GB/s 3.90 GB/s 🟢▲ +0.44% 0.79%
unidirectional/1024 kqueue bytes_per_sec 1.19 GB/s 1.18 GB/s 🔴▼ -0.47% 1.20%
unidirectional/1048576 kqueue bytes_per_sec 3.47 GB/s 3.45 GB/s 🔴▼ -0.40% 0.52%
unidirectional/16384 kqueue bytes_per_sec 3.46 GB/s 3.46 GB/s 🟢▲ +0.03% 0.63%
unidirectional/262144 kqueue bytes_per_sec 3.46 GB/s 3.46 GB/s 🟢▲ +0.15% 0.47%
unidirectional/4096 kqueue bytes_per_sec 2.69 GB/s 2.67 GB/s 🔴▼ -0.67% 0.85%
unidirectional/65536 kqueue bytes_per_sec 3.46 GB/s 3.46 GB/s 🔴▼ -0.09% 0.58%
unidirectional_lockless/1024 kqueue bytes_per_sec 1.20 GB/s 1.19 GB/s 🔴▼ -0.33% 1.06%
unidirectional_lockless/1048576 kqueue bytes_per_sec 3.52 GB/s 3.52 GB/s 🟢▲ +0.04% 0.58%
unidirectional_lockless/16384 kqueue bytes_per_sec 3.51 GB/s 3.51 GB/s 🟢▲ +0.16% 0.64%
unidirectional_lockless/262144 kqueue bytes_per_sec 3.52 GB/s 3.50 GB/s 🔴▼ -0.39% 0.57%
unidirectional_lockless/4096 kqueue bytes_per_sec 2.72 GB/s 2.71 GB/s 🟢▲ +0.10% 0.87%
unidirectional_lockless/65536 kqueue bytes_per_sec 3.52 GB/s 3.51 GB/s 🔴▼ -0.25% 0.61%

socket_latency

Benchmark Backend Metric Base Head Δ Noise
concurrent/1 kqueue latency_mean_ns 18.81 µs 18.70 µs 🟢▲ +0.50% 0.72%
concurrent/16 kqueue latency_mean_ns 121.00 µs 120.41 µs 🟢▲ +0.37% 0.44%
concurrent/4 kqueue latency_mean_ns 44.09 µs 43.76 µs 🟢▲ +0.42% 0.99%
concurrent_lockless/1 kqueue latency_mean_ns 18.76 µs 18.66 µs 🟢▲ +0.58% 0.35%
concurrent_lockless/16 kqueue latency_mean_ns 120.38 µs 120.74 µs 🔴▼ -0.38% 0.16%
concurrent_lockless/4 kqueue latency_mean_ns 43.44 µs 42.97 µs 🟢▲ +0.70% 1.24%
pingpong/1 kqueue latency_mean_ns 15.45 µs 15.42 µs 🟢▲ +0.25% 0.30%
pingpong/1024 kqueue latency_mean_ns 18.90 µs 18.91 µs 🔴▼ -0.13% 0.33%
pingpong/64 kqueue latency_mean_ns 18.74 µs 18.69 µs 🟢▲ +0.27% 0.40%
pingpong_lockless/1 kqueue latency_mean_ns 15.41 µs 15.41 µs 🟢▲ +0.15% 0.54%
pingpong_lockless/1024 kqueue latency_mean_ns 18.95 µs 18.89 µs 🟢▲ +0.26% 0.19%
pingpong_lockless/64 kqueue latency_mean_ns 18.70 µs 18.65 µs 🟢▲ +0.26% 0.23%

socket_throughput

Benchmark Backend Metric Base Head Δ Noise
bidirectional/1024 kqueue bytes_per_sec 410.0 MB/s 406.1 MB/s 🔴▼ -0.55% 1.09%
bidirectional/1048576 kqueue bytes_per_sec 7.80 GB/s 7.63 GB/s 🔴▼ -1.10% 4.28%
bidirectional/16384 kqueue bytes_per_sec 4.00 GB/s 3.96 GB/s 🔴▼ -1.33% 0.80%
bidirectional/262144 kqueue bytes_per_sec 8.89 GB/s 8.79 GB/s 🔴▼ -1.77% 3.18%
bidirectional/4096 kqueue bytes_per_sec 1.50 GB/s 1.49 GB/s 🔴▼ -0.47% 0.68%
bidirectional/65536 kqueue bytes_per_sec 7.68 GB/s 7.57 GB/s 🔴▼ -2.08% 2.60%
bidirectional_lockless/1024 kqueue bytes_per_sec 411.4 MB/s 412.4 MB/s 🟢▲ +0.15% 1.42%
bidirectional_lockless/1048576 kqueue bytes_per_sec 7.76 GB/s 7.69 GB/s 🟢▲ +0.86% 5.63%
bidirectional_lockless/16384 kqueue bytes_per_sec 4.04 GB/s 3.98 GB/s 🔴▼ -1.99% 1.18%
bidirectional_lockless/262144 kqueue bytes_per_sec 9.01 GB/s 8.70 GB/s 🔴▼ -3.42% 6.32%
bidirectional_lockless/4096 kqueue bytes_per_sec 1.50 GB/s 1.49 GB/s 🔴▼ -0.39% 0.95%
bidirectional_lockless/65536 kqueue bytes_per_sec 7.61 GB/s 7.62 GB/s 🟢▲ +0.43% 1.30%
multithread/2 kqueue bytes_per_sec 7.60 GB/s 7.45 GB/s 🟢▲ +0.41% 3.68%
multithread/4 kqueue bytes_per_sec 3.66 GB/s 3.59 GB/s 🔴▼ -2.29% 3.95%
multithread/8 kqueue bytes_per_sec 3.42 GB/s 3.28 GB/s 🔴▼ -5.30% 4.56%
unidirectional/1024 kqueue bytes_per_sec 319.2 MB/s 318.4 MB/s 🔴▼ -0.64% 1.06%
unidirectional/1048576 kqueue bytes_per_sec 9.01 GB/s 8.75 GB/s 🔴▼ -0.92% 5.37%
unidirectional/16384 kqueue bytes_per_sec 3.73 GB/s 3.73 GB/s 🟢▲ +0.04% 0.94%
unidirectional/262144 kqueue bytes_per_sec 8.99 GB/s 8.93 GB/s 🟢▲ +0.18% 2.67%
unidirectional/4096 kqueue bytes_per_sec 1.23 GB/s 1.24 GB/s 🔴▼ -0.24% 2.08%
unidirectional/65536 kqueue bytes_per_sec 7.28 GB/s 7.29 GB/s 🟢▲ +0.81% 1.39%
unidirectional_lockless/1024 kqueue bytes_per_sec 328.0 MB/s 327.4 MB/s 🔴▼ -0.20% 0.36%
unidirectional_lockless/1048576 kqueue bytes_per_sec 9.04 GB/s 8.80 GB/s 🔴▼ -0.66% 4.25%
unidirectional_lockless/16384 kqueue bytes_per_sec 3.74 GB/s 3.76 GB/s 🔴▼ -0.18% 1.34%
unidirectional_lockless/262144 kqueue bytes_per_sec 8.97 GB/s 8.97 GB/s 🟢▲ +0.26% 2.46%
unidirectional_lockless/4096 kqueue bytes_per_sec 1.25 GB/s 1.25 GB/s 🟢▲ +0.09% 0.35%
unidirectional_lockless/65536 kqueue bytes_per_sec 7.34 GB/s 7.29 GB/s 🔴▼ -0.63% 1.20%

Run & raw JSON artifacts · flag rule: |median Δ| > max(2%, 3×CV of the noisier side) · advisory only

@sgerbino
sgerbino force-pushed the pr/benchmark-report branch 3 times, most recently from 1137e95 to 60bf84c Compare October 2, 2026 16:46
@sgerbino
sgerbino marked this pull request as ready for review October 2, 2026 19:20
@sgerbino
bench: skip local socket suites on Windows
a1a2ba8 
local::connect_pair relies on socketpair(), which has no Windows
implementation and throws operation_not_supported at runtime, taking
the whole comparison run down with it. The corosio local socket suites
are already POSIX-only, so Windows loses no comparison coverage.
@sgerbino
bench: prevent TIME_WAIT pileup in the churn benchmarks
f9f3f29 
Only the connecting side carried linger(0), so on loopback a server
FIN racing the client RST could still park connections in TIME_WAIT.
At production durations the leak overflowed the macOS PCB table
mid-benchmark (70k against a 12k drain threshold), socket creation
started failing, and the corosio loop ignored the open() error before
set_option threw and took down the whole suite. The listener now
carries the churn socket options -- accepted sockets inherit them on
every supported platform -- so both sides RST at close without adding
per-connection syscalls to the measured loop, and a failed open() now
ends the benchmark instead of aborting the process.
@sgerbino
bench: publish a generated, reproducible benchmark report
99008d2 
Replaces the hand-written February benchmark page with a pipeline that
regenerates it from fresh runs on the three dedicated machines
(closes #352):

- benchmark-report.yml: dispatch-only workflow running the comparison
  suite (corosio, asio coroutines, asio callbacks) on every platform,
  uploading raw per-iteration JSON and an environment capture. Linux
  builds asio against both epoll and io_uring so each corosio
  configuration is measured against a same-reactor asio.
- report_page.py: stdlib-only generator producing the four Antora
  pages and theme-aware inline SVG charts; medians across iterations,
  CV-based noise classification, diverging bars against the
  asio-coroutines baseline.
- Scenario descriptions in the bench harness (describe and
  category_description), rendered as category intros and chart hover
  text.
- bench/README.md: how to run the suite, dispatch the workflow, and
  regenerate the pages.
- Generated pages and charts from a full 7x2 production run; the old
  benchmark-report.adoc URL survives via a page alias, nav gains the
  per-platform entries, and the lint baseline follows the page swap.
@sgerbino
sgerbino force-pushed the pr/benchmark-report branch from 60bf84c to 99008d2 Compare October 2, 2026 21:55
@cppalliance-bot

cppalliance-bot commented Oct 2, 2026

Copy link
Copy Markdown

An automated preview of the documentation is available at https://377.corosio.prtest3.cppalliance.org/index.html

If more commits are pushed to the pull request, the docs will rebuild at the same URL.

2026-10-02 22:00:20 UTC

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

No reviews

Assignees

No one assigned

Labels

None yet

Projects

Beast2
Status: Backlog

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

perf: make the published benchmark report reproducible and current

2 participants

@sgerbino @cppalliance-bot