Repository navigation
connections-5m: five million held keep-alive connections, plaintext and TLS [skip-maintainer-ping] - #1815
connections-5m: five million held keep-alive connections, plaintext and TLS [skip-maintainer-ping]#1815MDA2AV wants to merge 12 commits into
Conversation
… fan-out A new opt-in, reference-only H1 profile for #1759. One million keep-alive connections are held at once and GET /baseline11 is paced at 250K req/s across them - one request per connection every four seconds - so what separates entries is what each held connection costs: memory, CPU per request while the rest wait, and the tail. One source address can hold ~64.5K connections to one destination, so scripts/zrk_fanout.py runs 32 zrk processes, one per loopback address 127.0.0.1-127.0.0.32, 31,250 connections each, started 1s apart. Only the 20s window after the whole fleet has connected (plus 10s to settle) is reported: each process streams per-second HdrHistograms, merged bucket by bucket, and the server's cgroup CPU is read at the window's edges. The established sockets on :8080 are counted after the window ("Held"). Servers must listen on 0.0.0.0 or [::]; validate.sh checks 127.0.0.2 and 127.0.0.32, and the fan-out probes before it launches. The run raises fs.nr_open, vm.max_map_count and the conntrack table (only when the profile runs), gives the container nofile 2097152 and a 96 GiB memory cap. Scored like the other fixed-rate profiles (full credit at 237,500 req/s) but reference-only. Subscribes actix, aspnet-minimal, bun, fib and node.
|
/benchmark -f actix -t connections-1m |
|
👋 Benchmark request received. A collaborator will review and approve the run. |
…iner The first million-connection run on the bench host (job 133323370552) took the host down: the runner lost communication ten minutes into "Run benchmarks" and went offline. zrk 2.3.0 keeps ~303 KiB resident per connection - measured flat at 1K, 4K and 8K connections - so a million needed ~290 GiB of a 251 GiB machine, and nothing capped the client. Two guards in the fan-out: - before launching, it counts the client (ZRK_KIB_PER_CONN, 320), the kernel's share (8 KiB a connection) and the server's memory cap against MemAvailable, and refuses with the figures when they do not fit; - each zrk container gets --memory at half again over its estimate, so a client that outgrows it is OOM-killed in its own cgroup. With these, a 1M run refuses on the bench host rather than starting.
zrk 2.3.0 keeps ~303 KiB resident per connection, so the 32-process fan-out needed ~290 GiB for a million and took the 251 GiB bench host down on its first run. gcannon 0.8.0 (MDA2AV/gcannon: --rate, --addr-span, --connect-limit, --warmup-max) holds one in ~3.3 KiB, so a single process carries the whole fleet: 250K req/s paced per connection on one shared clock, latency from each request's scheduled time, spread over 127.0.0.1-127.0.0.32, at most 64 first connects in flight per thread, and a warmup of up to 180s before the 20s window. scripts/conn_hold.py replaces scripts/zrk_fanout.py and keeps what gcannon cannot know: it requires gcannon >= 0.8.0, probes 127.0.0.2 and 127.0.0.32 for a listener on every address, refuses a run the host cannot hold in memory, checks the open-file limit, and reads the server cgroup's CPU at gcannon's window markers, so neither the connects nor the closes are in the CPU figure. "Held" is gcannon's count of connections still open at the end of the window. zrk.sh is back to main's version; the profile no longer raises vm.max_map_count, which only zrk's coroutine stacks needed. Docs follow.
…t ran gcannon 0.7.0 and later include openssl/crypto.h for the TLS handshake, and the image's build stage had no libssl-dev, so building gcannon main (or the v0.8.0 release) in docker failed with "openssl/crypto.h: No such file or directory". The runtime stage's ubuntu:24.04 already ships libssl3. Built from v0.8.0 with the fix, the image held 16,000 connections at the paced rate through the connections-1m path. connections-1m runs on whichever gcannon the run uses, which has to be >= 0.8.0 (released as v0.8.0). When it is not, the error now names the binary or image that was asked and whether it answered --version at all.
|
/benchmark -f actix -t connections-1m |
|
👋 Benchmark request received. A collaborator will review and approve the run. |
Benchmark ResultsFramework:
Composite score, main against this run, in the board's default view:
The results on main were measured in another session, so a few points up or down can be the machine and not the change. Full log |
|
/benchmark -f genhttp-ioxide -t baseline |
|
👋 Benchmark request received. A collaborator will review and approve the run. |
Benchmark ResultsFramework:
Composite score, main against this run, in the board's default view:
The results on main were measured in another session, so a few points up or down can be the machine and not the change. Full log |
It listens on 0.0.0.0 (validate's 127.0.0.2/.32 probe passes) and GenHTTP sets ioxide's read timeout to 0, so a held connection is never reaped between its requests 4 s apart.
…nHTTP#953 5967f5c6b, one commit past 28b0cd107. It adds acceptance tests only: Engine/Ioxide and the modules this entry builds are unchanged, and still on ioxide 0.26.563.
|
/benchmark -f genhttp-ioxide -t connections-1m |
|
👋 Benchmark request received. A collaborator will review and approve the run. |
Benchmark ResultsFramework:
Composite score, main against this run, in the board's default view:
The results on main were measured in another session, so a few points up or down can be the machine and not the change. Full log |
A twin of connections-1m with zrk as the load generator instead of gcannon, to compare what the two report on one load: 1M held connections, 250K req/s, the same cpusets, 64 client threads, a 20s window, the same score. zrk 3.0.2 shares its latency histograms per thread (zoxy-io/zrk#102). Measured on the dev box against actix at one request per connection every 4 s: ~41 KiB resident per connection, flat from 4K to 16K (2.3.0: ~305 KiB at 1K), so the million needs ~40 GiB. The other zrk profiles stay on the pinned 2.3.0; this one builds zrk:3.0.2 from the same Dockerfile and refuses an older zrk. zrk takes one URL, so scripts/zrk_fanout.py (restored from d9a19a7) runs one process per address 127.0.0.1-.32, starts them 1 s apart, discards 10 s, measures 20 s, merges the per-second HdrHistograms bucket by bucket, and counts the server's established sockets with ss in a 20 s tail. Its output uses conn_hold.py's keys (hold_error, connections_open, connect_errors, cpu_usec_window), so benchmark.sh treats both profiles alike (conn1m_profile). vm.max_map_count is raised again: zrk maps ~4 regions per connection. Subscribed: the six connections-1m entries.
|
/benchmark -f genhttp-ioxide -t connections-1m-zrk |
|
👋 Benchmark request received. A collaborator will review and approve the run. |
Benchmark ResultsFramework:
Composite score, main against this run, in the board's default view:
The results on main were measured in another session, so a few points up or down can be the machine and not the change. Full log |
gcannon 0.8.0's --rate scheduled each thread's connections inside one 1/T slice of the interval, so one thread at a time sent the whole fleet's rate. At a million connections, 64 threads and 250K req/s nearly every send went out 100-165 ms late, and p50 read ~150 ms on every server, where zrk read 70 us on the same entry. 0.8.1 spreads every thread over the whole interval (MDA2AV/gcannon#30). conn_hold.py now refuses an older gcannon, so a host still on 0.8.0 reports why instead of publishing its latencies.
|
/benchmark -f genhttp-ioxide -t baseline |
|
👋 Benchmark request received. A collaborator will review and approve the run. |
Benchmark ResultsFramework:
Composite score, main against this run, in the board's default view:
The results on main were measured in another session, so a few points up or down can be the machine and not the change. Full log |
|
/benchmark -f genhttp-ioxide -t connections-1m |
|
👋 Benchmark request received. A collaborator will review and approve the run. |
Benchmark ResultsFramework:
Composite score, main against this run, in the board's default view:
The results on main were measured in another session, so a few points up or down can be the machine and not the change. Full log |
connections-1m becomes connections-5m: 5,000,000 keep-alive connections, GET /baseline11 paced at 1,000,000 req/s across them (one request per connection every 5 s), on gcannon >= 0.8.1. connections-5m-tls is the same fleet over TLS 1.3 on the h1 TLS port (8081): gcannon handshakes with OpenSSL, hands the session to kernel TLS and frees its own state. connections-1m-zrk stays at a million (zrk's ~41 KiB per connection rules out five). - 160 loopback addresses, 31,250 connections each (the density the 1M runs held); validate.sh probes .2 and .160, and the TLS port over TLS 1.3. - Server caps: nofile 8,388,608, memory 128g (112g over TLS), via held_conn_limits; fs.nr_open and conntrack raised to 16M, hashsize 4M. - conn_hold.py: per-profile messages and kernel-memory estimate, logs the memory budget, and raises gcannon's open-file limit past the runner's hard limit with `sudo -n prlimit` (with the reason when it cannot). - Warmup up to 300 s (900 s over TLS: five million handshakes); 60 s between runs at 2M+ connections. - Full credit at 950,000 req/s; docs move to test-profiles/h1/connections-5m. Tested on the dev box through gcannon_build_args -> gcannon_run -> gcannon_parse with gcannon 0.8.1, 16,000 connections over all 160 addresses against actix: plaintext and TLS both held 16,000 at rate 1.0000 (4 s interval). At the profile's 5 s interval actix reconnected 51,410 times and answered 4,221 4xx: its 5 s keep-alive and first-request timeouts race the interval.
…rker At 1M req/s over five million connections each connection sends every 5 s, exactly the 5 s keep-alive default of Node, uvicorn and actix (actix's first-request timeout too). On the dev box at 16,000 connections against actix, a 5 s interval gave 51,410 reconnects and 4,221 4xx (rate 0.935); a 4 s interval none. 1.25M req/s makes it 4 s, inside those defaults, so the timeouts stay as they are. Full credit moves to 1,187,500 req/s (95%). actix-server stops accepting at 25,600 connections per worker (measured: 4 workers accepted exactly 102,400 of 120,000 offered), 1.64M on the bench host's 64. max_connections(100_000) lifts it to 6.4M; the same test then accepted all 120,000 at the full rate, no errors. The resilience suite's slow attacks hold 32,768 connections, under even the default cap of its 2-CPU box, so they never reach it.
|
/benchmark -f genhttp-ioxide -t connections-5m |
|
👋 Benchmark request received. A collaborator will review and approve the run. |
Benchmark ResultsFramework:
Composite score, main against this run, in the board's default view:
The results on main were measured in another session, so a few points up or down can be the machine and not the change. Full log |
genhttp-ioxide's first connections-5m run held all five million (79.1 GiB),
and the second was refused: MemAvailable had gone from 245 to 174 GiB. The
server container runs all three of a profile's runs, and a server that has
just held five million connections keeps what it grew to after they close (a
GC heap, allocator arenas). The check then counted that memory twice: it is
missing from MemAvailable and inside the 128 GiB cap the check adds in full.
- benchmark.sh stops the server after each held-connection run, before the
cool-down, and starts a fresh one for the next, so every run starts clean
and its Mem column measures that run alone.
- conn_hold.py and zrk_fanout.py count only what the server may still grow
by: its cap less the cgroup's memory.current now.
Tested through benchmark.sh in a scratch copy with the profile shrunk to
16,000 connections on 8 CPUs, 2 runs against actix: run 2 started on a
fresh container ("at 0.0 GiB now"), both held 16,000 at rate 1.0000, and each
run's CPU and memory came from its own container.
Adds held-connection profiles. The server holds five million keep-alive connections while
GET /baseline11is paced at 1.25M req/s across them, one request per connection every 4 s: in plaintext asconnections-5m, or over TLS 1.3 asconnections-5m-tls. They show what each held connection costs (memory, CPU per request, tail latency) and how many of the five million the server kept.connections-1m-zrkholds a million on zrk 3.0.2.Design
--rate 1250000 --addr-span 160 --connect-limit 64. 127.0.0.1–.160 carry 31,250 connections each, the density the 1M runs held. The warmup waits up to 300 s (900 s over TLS) for every connection to open, then a 20 s window is measured.https://target, gcannon handshakes TLS 1.3 with OpenSSL, hands the session to kernel TLS, and frees its own TLS state. A TLS connection costs the client about what a plaintext one does, so what the TLS variant adds is the server's TLS sessions.scripts/conn_hold.pydoes five things:sudo -n prlimit;nofile8,388,608 and a memory cap of 128 GiB (112 GiB over TLS);fs.nr_openand the conntrack table go to 16M, its hash size to 4M;test-profiles/h1/connections-5m.connections-1m-zrkis unchanged: 32 zrk 3.0.2 processes, a million connections at 250K req/s. It's the cross-check that found gcannon 0.8.0's--ratebursts (fix(rate): spread each thread's connections over the whole interval gcannon#30). It can't go to five million, because zrk needs ~41 KiB per connection.Subscribers: actix, aspnet-minimal, bun, fib, genhttp-ioxide and node, to both 5M profiles. Each serves TLS on 8081 on every address.
Why 1.25M req/s and not 1M:
actix raises its connection cap. actix-server stops accepting at 25,600 connections per worker, measured locally as 4 workers accepting exactly 102,400 of 120,000. On the bench host's 64 workers that's 1.64M.
max_connections(100_000)lifts it to 6.4M, and the same test then accepted all 120,000 at the full rate. The resilience suite's slow attacks hold 32,768 connections, under even the default cap in its 2-CPU box, so actix's resilience results don't change.genhttp-ioxide needs nothing raised:
Its memory, ~80–110 GiB projected at 5M against the 128 GiB cap, is the thing to watch.
Tested on the dev box with gcannon 0.8.1, through the PR's own
gcannon_build_args→gcannon_run→gcannon_parse, at 16,000 connections over all 160 addresses against actix:sudo -n prlimitfails;check_badge_parity.jspass.History:
--rate(Add --rate: fixed request rate paced per connection, plus --addr-span and --connect-limit (0.8.0) gcannon#28).Also:
docker/gcannon.Dockerfilegetslibssl-dev, so Docker mode builds gcannon ≥ 0.7.0.5967f5c). That commit adds only acceptance tests, so the server it builds is unchanged.Overlap: conflicts with #1714 (my draft) on benchmark.sh's cool-down line, nothing else.
Part of #1759. The other connection levels, WebSocket, and how memory should be scored are still open there.
🤖 Generated with Claude Code