Skip to content

connections-5m: five million held keep-alive connections, plaintext and TLS [skip-maintainer-ping] - #1815

Open
MDA2AV wants to merge 12 commits into
mainfrom
connections-1m
Open

MDA2AV wants to merge 12 commits into
mainfrom
connections-1m

Conversation

@MDA2AV

@MDA2AV MDA2AV commented Nov 16, 2026 •

Copy link
Copy Markdown
Owner

Status: connections-5m ran once for genhttp-ioxide: all 5,000,000 held, 1.28M req/s, 79.1 GiB. Its runs 2 and 3 were refused by the memory check, which is fixed (fresh server per run). connections-5m-tls hasn't run yet.

Adds held-connection profiles. The server holds five million keep-alive connections while GET /baseline11 is paced at 1.25M req/s across them, one request per connection every 4 s: in plaintext as connections-5m, or over TLS 1.3 as connections-5m-tls. They show what each held connection costs (memory, CPU per request, tail latency) and how many of the five million the server kept. connections-1m-zrk holds a million on zrk 3.0.2.

Design

  • One gcannon: --rate 1250000 --addr-span 160 --connect-limit 64. 127.0.0.1–.160 carry 31,250 connections each, the density the 1M runs held. The warmup waits up to 300 s (900 s over TLS) for every connection to open, then a 20 s window is measured.
  • TLS: with an https:// target, gcannon handshakes TLS 1.3 with OpenSSL, hands the session to kernel TLS, and frees its own TLS state. A TLS connection costs the client about what a plaintext one does, so what the TLS variant adds is the server's TLS sessions.
  • scripts/conn_hold.py does five things:
    • it requires gcannon ≥ 0.8.1;
    • it probes 127.0.0.2 and .160 on the profile's port;
    • it refuses a run the host can't hold in memory, and logs the budget;
    • it raises gcannon's open-file limit past the runner's hard limit with sudo -n prlimit;
    • it reads the server's CPU at gcannon's window markers.
  • Limits, raised only for these profiles:
    • the server container gets nofile 8,388,608 and a memory cap of 128 GiB (112 GiB over TLS);
    • fs.nr_open and the conntrack table go to 16M, its hash size to 4M;
    • runs are 60 s apart, and each starts on a fresh server container. A server that has just held five million connections keeps what it grew to after they close: genhttp-ioxide still held ~71 GiB a minute after its first 5M run, and the memory check refused run 2.
  • The memory check counts only what the server can still grow by (its cap less its cgroup's current memory), not the whole cap on top of what it already holds.
  • Scoring and board: "Held" is a board column. Scored like the other fixed-rate profiles (full credit at 1,187,500 req/s), reference-only. The docs live at test-profiles/h1/connections-5m.
  • connections-1m-zrk is unchanged: 32 zrk 3.0.2 processes, a million connections at 250K req/s. It's the cross-check that found gcannon 0.8.0's --rate bursts (fix(rate): spread each thread's connections over the whole interval gcannon#30). It can't go to five million, because zrk needs ~41 KiB per connection.

Subscribers: actix, aspnet-minimal, bun, fib, genhttp-ioxide and node, to both 5M profiles. Each serves TLS on 8081 on every address.

Why 1.25M req/s and not 1M:

  • The timeout race: 1M req/s over five million connections is a 5 s interval, exactly the 5 s keep-alive default of node, uvicorn and actix (actix's first-request timeout too). Locally, at 16,000 connections against actix, a 5 s interval gave 51,410 reconnects and 4,221 4xx responses (rate 0.935). A 4 s interval was clean.
  • The fix: 1.25M req/s is a 4 s interval, so every subscriber's timeouts stay as they are.

actix raises its connection cap. actix-server stops accepting at 25,600 connections per worker, measured locally as 4 workers accepting exactly 102,400 of 120,000. On the bench host's 64 workers that's 1.64M. max_connections(100_000) lifts it to 6.4M, and the same test then accepted all 120,000 at the full rate. The resilience suite's slow attacks hold 32,768 connections, under even the default cap in its 2-CPU box, so actix's resilience results don't change.

genhttp-ioxide needs nothing raised:

  • it has no idle timeout;
  • ioxide's 4,096-connection limit only applies to a receive mode GenHTTP leaves off;
  • it uses ~0.014 memory mappings per connection.

Its memory, ~80–110 GiB projected at 5M against the 128 GiB cap, is the thing to watch.

Tested on the dev box with gcannon 0.8.1, through the PR's own gcannon_build_args → gcannon_run → gcannon_parse, at 16,000 connections over all 160 addresses against actix:

  • plaintext and TLS both held all 16,000 at rate 1.0000 (4 s interval), with p99 69 µs and 91 µs;
  • the window CPU was read;
  • a 1024-file hard limit makes the run stop with the reason, after sudo -n prlimit fails;
  • the site generator and check_badge_parity.js pass.

History:

Also:

Overlap: conflicts with #1714 (my draft) on benchmark.sh's cool-down line, nothing else.

Part of #1759. The other connection levels, WebSocket, and how memory should be scored are still open there.

🤖 Generated with Claude Code

… fan-out

A new opt-in, reference-only H1 profile for #1759. One million keep-alive
connections are held at once and GET /baseline11 is paced at 250K req/s
across them - one request per connection every four seconds - so what
separates entries is what each held connection costs: memory, CPU per
request while the rest wait, and the tail.

One source address can hold ~64.5K connections to one destination, so
scripts/zrk_fanout.py runs 32 zrk processes, one per loopback address
127.0.0.1-127.0.0.32, 31,250 connections each, started 1s apart. Only the
20s window after the whole fleet has connected (plus 10s to settle) is
reported: each process streams per-second HdrHistograms, merged bucket by
bucket, and the server's cgroup CPU is read at the window's edges. The
established sockets on :8080 are counted after the window ("Held").

Servers must listen on 0.0.0.0 or [::]; validate.sh checks 127.0.0.2 and
127.0.0.32, and the fan-out probes before it launches. The run raises
fs.nr_open, vm.max_map_count and the conntrack table (only when the profile
runs), gives the container nofile 2097152 and a 96 GiB memory cap.

Scored like the other fixed-rate profiles (full credit at 237,500 req/s) but
reference-only. Subscribes actix, aspnet-minimal, bun, fib and node.
@MDA2AV

MDA2AV commented Nov 16, 2026

Copy link
Copy Markdown
Owner Author

/benchmark -f actix -t connections-1m

@github-actions

Copy link
Copy Markdown
Contributor

👋 Benchmark request received. A collaborator will review and approve the run.

…iner

The first million-connection run on the bench host (job 133323370552) took
the host down: the runner lost communication ten minutes into "Run
benchmarks" and went offline. zrk 2.3.0 keeps ~303 KiB resident per
connection - measured flat at 1K, 4K and 8K connections - so a million
needed ~290 GiB of a 251 GiB machine, and nothing capped the client.

Two guards in the fan-out:
- before launching, it counts the client (ZRK_KIB_PER_CONN, 320), the
  kernel's share (8 KiB a connection) and the server's memory cap against
  MemAvailable, and refuses with the figures when they do not fit;
- each zrk container gets --memory at half again over its estimate, so a
  client that outgrows it is OOM-killed in its own cgroup.

With these, a 1M run refuses on the bench host rather than starting.
zrk 2.3.0 keeps ~303 KiB resident per connection, so the 32-process fan-out
needed ~290 GiB for a million and took the 251 GiB bench host down on its
first run. gcannon 0.8.0 (MDA2AV/gcannon: --rate, --addr-span,
--connect-limit, --warmup-max) holds one in ~3.3 KiB, so a single process
carries the whole fleet: 250K req/s paced per connection on one shared
clock, latency from each request's scheduled time, spread over
127.0.0.1-127.0.0.32, at most 64 first connects in flight per thread, and
a warmup of up to 180s before the 20s window.

scripts/conn_hold.py replaces scripts/zrk_fanout.py and keeps what gcannon
cannot know: it requires gcannon >= 0.8.0, probes 127.0.0.2 and
127.0.0.32 for a listener on every address, refuses a run the host cannot
hold in memory, checks the open-file limit, and reads the server cgroup's
CPU at gcannon's window markers, so neither the connects nor the closes are
in the CPU figure. "Held" is gcannon's count of connections still open at
the end of the window.

zrk.sh is back to main's version; the profile no longer raises
vm.max_map_count, which only zrk's coroutine stacks needed. Docs follow.
…t ran

gcannon 0.7.0 and later include openssl/crypto.h for the TLS handshake, and
the image's build stage had no libssl-dev, so building gcannon main (or the
v0.8.0 release) in docker failed with "openssl/crypto.h: No such file or
directory". The runtime stage's ubuntu:24.04 already ships libssl3. Built
from v0.8.0 with the fix, the image held 16,000 connections at the paced
rate through the connections-1m path.

connections-1m runs on whichever gcannon the run uses, which has to be
>= 0.8.0 (released as v0.8.0). When it is not, the error now names the
binary or image that was asked and whether it answered --version at all.
@MDA2AV

MDA2AV commented Nov 16, 2026

Copy link
Copy Markdown
Owner Author

/benchmark -f actix -t connections-1m

@github-actions

Copy link
Copy Markdown
Contributor

👋 Benchmark request received. A collaborator will review and approve the run.

@github-actions

Copy link
Copy Markdown
Contributor

Benchmark Results

Framework: actix | Test: connections-1m

Test Conn RPS Rate CPU/req p99 p99.9 CPU Mem Δ RPS Δ Mem
connections-1m 1000000 251,211 1.0048 32.54us 173900.0us 197800.0us 638.9% 22.7GiB NEW NEW

Composite score, main against this run, in the board's default view:

Family main this run Δ
HTTP/1.1 6322.4 (10 of 109) 6322.4 (10 of 109) +0.0
HTTP/2 2187.9 (8 of 41) 2187.9 (8 of 41) +0.0
WebSocket 2234.0 (11 of 39) 2234.0 (11 of 39) +0.0

The results on main were measured in another session, so a few points up or down can be the machine and not the change.

Full log
   "4xx": 0,
   "5xx": 0,
   "other": 0
  },
  "errors": {
   "connect": 0,
   "read": 0,
   "timeout": 0
  },
  "reconnects": 0,
  "rate": {
   "target": 250000,
   "achieved": 251216.95,
   "ratio": 1.0049,
   "paced_sends": 5025698,
   "late": 4760967,
   "max_lag_us": 124879
  },
  "connections_active": 1000000
 }
}
[info] CPU 638.9% | Mem 22.7GiB

[run 2/3]
gcannon: warmup done, measuring for 20s
gcannon: measurement done
[conn-hold] 1000000 of 1000000 connections open at the end of the window
{
 "connections": 1000000,
 "connections_open": 1000000,
 "target_rate": 250000,
 "rps": 250249.36,
 "rate_ratio": 1.001,
 "duration_s": 20.051,
 "bytes_per_sec": 25525261.62,
 "max_schedule_lag_us": 130747,
 "latency_us": {
  "mean": 149492,
  "p50": 149000,
  "p90": 164100,
  "p99": 183100,
  "p99_9": 201500
 },
 "status_codes": {
  "2xx": 5017750,
  "3xx": 0,
  "4xx": 0,
  "5xx": 0
 },
 "errors": {
  "connect": 0,
  "read": 0,
  "timeout": 0
 },
 "reconnects": 0,
 "server_cpu_usec": 164651083,
 "gcannon": {
  "version": "0.8.0",
  "target": "127.0.0.1:8080/baseline11?a=1&b=2",
  "mode": "http",
  "connections": 1000000,
  "threads": 64,
  "pipeline_depth": 1,
  "duration_sec": 20,
  "elapsed_sec": 20.051,
  "requests": 5015538,
  "responses": 5017750,
  "rps": 250247.66,
  "bandwidth_bps": 25525261.62,
  "bytes_read": 511810500,
  "latency_us": {
   "avg": 149492,
   "p50": 149000,
   "p90": 164100,
   "p99": 183100,
   "p999": 201500,
   "samples": 5017750
  },
  "status": {
   "2xx": 5017750,
   "3xx": 0,
   "4xx": 0,
   "5xx": 0,
   "other": 0
  },
  "errors": {
   "connect": 0,
   "read": 0,
   "timeout": 0
  },
  "reconnects": 0,
  "rate": {
   "target": 250000,
   "achieved": 250247.66,
   "ratio": 1.001,
   "paced_sends": 5015538,
   "late": 4748356,
   "max_lag_us": 130747
  },
  "connections_active": 1000000
 }
}
[info] CPU 738.0% | Mem 17.4GiB

[run 3/3]
gcannon: warmup done, measuring for 20s
gcannon: measurement done
[conn-hold] 1000000 of 1000000 connections open at the end of the window
{
 "connections": 1000000,
 "connections_open": 1000000,
 "target_rate": 250000,
 "rps": 250151.84,
 "rate_ratio": 1.0006,
 "duration_s": 20.054,
 "bytes_per_sec": 25515994.17,
 "max_schedule_lag_us": 132167,
 "latency_us": {
  "mean": 150230,
  "p50": 149500,
  "p90": 165300,
  "p99": 190400,
  "p99_9": 201800
 },
 "status_codes": {
  "2xx": 5016545,
  "3xx": 0,
  "4xx": 0,
  "5xx": 0
 },
 "errors": {
  "connect": 0,
  "read": 0,
  "timeout": 0
 },
 "reconnects": 0,
 "server_cpu_usec": 163857462,
 "gcannon": {
  "version": "0.8.0",
  "target": "127.0.0.1:8080/baseline11?a=1&b=2",
  "mode": "http",
  "connections": 1000000,
  "threads": 64,
  "pipeline_depth": 1,
  "duration_sec": 20,
  "elapsed_sec": 20.054,
  "requests": 5010677,
  "responses": 5016545,
  "rps": 250156.81,
  "bandwidth_bps": 25515994.17,
  "bytes_read": 511687590,
  "latency_us": {
   "avg": 150230,
   "p50": 149500,
   "p90": 165300,
   "p99": 190400,
   "p999": 201800,
   "samples": 5016545
  },
  "status": {
   "2xx": 5016545,
   "3xx": 0,
   "4xx": 0,
   "5xx": 0,
   "other": 0
  },
  "errors": {
   "connect": 0,
   "read": 0,
   "timeout": 0
  },
  "reconnects": 0,
  "rate": {
   "target": 250000,
   "achieved": 250156.81,
   "ratio": 1.0006,
   "paced_sends": 5010677,
   "late": 4742938,
   "max_lag_us": 132167
  },
  "connections_active": 1000000
 }
}
[info] CPU 706.5% | Mem 17.0GiB
[info] run 1 wins on score

=== Best: 251211 req/s (CPU: 638.9%, Mem: 22.7GiB) ===
[info] exact CPU: 163.70 core-seconds | 32.536 us/req
=== Fixed-rate: cpu/req 32.54us | p99 173900.0us | p99.9 197800.0us | rate_ratio 1.0048 ===
=== Connections: 1000000 open of 1000000 | connect errors in the window 0 ===
[info] saved results/connections-1m/1000000/actix.json
httparena-bench-actix
httparena-bench-actix
[info] rebuilding site/data/*.json
[updated] /home/diogo/actions-runner/_work/HttpArena/HttpArena/site/data/frameworks.json
[updated] /home/diogo/actions-runner/_work/HttpArena/HttpArena/site/data/results/actix.json - 1 new, 33 total
[updated] /home/diogo/actions-runner/_work/HttpArena/HttpArena/site/data/current.json
[info] done
httparena-postgres
[info] restoring loopback MTU to 65536

@MDA2AV

MDA2AV commented Nov 16, 2026

Copy link
Copy Markdown
Owner Author

/benchmark -f genhttp-ioxide -t baseline

@github-actions

Copy link
Copy Markdown
Contributor

👋 Benchmark request received. A collaborator will review and approve the run.

@github-actions

Copy link
Copy Markdown
Contributor

Benchmark Results

Framework: genhttp-ioxide | Test: baseline

Test Conn RPS CPU Mem Δ RPS Δ Mem
baseline 4096 3,502,279 6160.4% 2.2GiB -1.4% -4.3%

Composite score, main against this run, in the board's default view:

Family main this run Δ
HTTP/1.1 7074.6 (3 of 109) 7023.1 (3 of 109) -51.5
HTTP/2 3321.9 (1 of 41) 3210.5 (2 of 41) -111.4
HTTP/3 1593.6 (2 of 20) 1208.1 (5 of 20) -385.6
WebSocket 2532.9 (5 of 39) 2633.7 (5 of 39) +100.7

The results on main were measured in another session, so a few points up or down can be the machine and not the change.

Full log
[info] available CPUs: 128
[info] framework: genhttp-ioxide (genhttp-ioxide, C#)
[info] subscribed tests: baseline,async,latency-1m,latency-10k,latency-500k-8cpu,pipelined,limited-conn,json-comp,json-tls,static-tls,baseline-h2c,json-h2c,baseline-h2,static-h2,static-h3,baseline-h3,8gbit,async-db,echo-ws,echo-ws-pipeline,echo-ws-limited
[info] building image: httparena-genhttp-ioxide
#0 building with "default" instance using docker driver

#1 [internal] load build definition from Dockerfile
#1 transferring dockerfile: 460B done
#1 DONE 0.0s

#2 [internal] load metadata for mcr.microsoft.com/dotnet/runtime:11.0
#2 ...

#3 [internal] load metadata for mcr.microsoft.com/dotnet/sdk:11.0
#3 DONE 0.4s

#2 [internal] load metadata for mcr.microsoft.com/dotnet/runtime:11.0
#2 DONE 0.4s

#4 [internal] load .dockerignore
#4 transferring context: 69B done
#4 DONE 0.0s

#5 [internal] load build context
#5 DONE 0.0s

#6 [stage-1 1/3] FROM mcr.microsoft.com/dotnet/runtime:11.0@sha256:3fe5f5495141c97986a49d3a4e76061f6225c3f93f94e8a7e522781376193863
#6 resolve mcr.microsoft.com/dotnet/runtime:11.0@sha256:3fe5f5495141c97986a49d3a4e76061f6225c3f93f94e8a7e522781376193863 0.1s done
#6 DONE 0.1s

#7 [build 1/6] FROM mcr.microsoft.com/dotnet/sdk:11.0@sha256:65f3c7c6a25a239ab3e6719f2896147e7b68242ff2d8818dd70498bde6ef8bea
#7 resolve mcr.microsoft.com/dotnet/sdk:11.0@sha256:65f3c7c6a25a239ab3e6719f2896147e7b68242ff2d8818dd70498bde6ef8bea 0.1s done
#7 DONE 0.1s

#5 [internal] load build context
#5 transferring context: 37.15kB done
#5 DONE 0.0s

#8 [build 5/6] COPY . .
#8 CACHED

#9 [stage-1 2/3] WORKDIR /app
#9 CACHED

#10 [build 6/6] RUN dotnet publish genhttp.csproj -c Release --no-self-contained -o /app
#10 CACHED

#11 [build 2/6] WORKDIR /src/app
#11 CACHED

#12 [build 3/6] COPY genhttp.csproj ./
#12 CACHED

#13 [build 4/6] RUN dotnet restore
#13 CACHED

#14 [stage-1 3/3] COPY --from=build /app .
#14 CACHED

#15 exporting to image
#15 exporting layers done
#15 exporting manifest sha256:46af16ff6c58c972564dcdd9643a4daaf45edaf552fc7e4854bb7be8f01e421b 0.0s done
#15 exporting config sha256:03bde063e18c4579399f2a317e3005255ff4a6ea5d3b5879954c9457e7247fb9 0.0s done
#15 exporting attestation manifest sha256:4e67e43c59ef4052d1d3ff647ceb8831feb17c7f48c478c85d741c815f944d84
#15 exporting attestation manifest sha256:4e67e43c59ef4052d1d3ff647ceb8831feb17c7f48c478c85d741c815f944d84 0.0s done
#15 exporting manifest list sha256:f8239902e1106c2cdfd5ddf1dd764a3a38dba8988209c0e54695700022193c3c 0.0s done
#15 naming to docker.io/library/httparena-genhttp-ioxide:latest done
#15 unpacking to docker.io/library/httparena-genhttp-ioxide:latest
#15 unpacking to docker.io/library/httparena-genhttp-ioxide:latest 0.2s done
#15 DONE 0.4s
[info] zrk: docker mode (zrk:local)
[info] tuning host for benchmark runs
[info] CPU governor → performance
[info] setting kernel socket limits
[info] setting UDP buffer sizes for QUIC
[info] setting loopback MTU to 1500 (realistic Ethernet)
[info] restarting docker daemon
[info] dropping kernel caches
[info] starting postgres sidecar
[info] postgres ready (seeded)

==============================================
=== genhttp-ioxide / baseline / 4096c (tool=gcannon) ===
==============================================
[info] waiting for server...
[info] server ready

[run 1/3]
gcannon v0.8.0
  Target:    localhost:8080/
  Threads:   64
  Conns:     4096 (64/thread)
  Pipeline:  1
  Req/conn:  unlimited (keep-alive)
  Templates: 3
  Expected:  200
  Duration:  5s


  Thread Stats   Avg      p50      p90      p99    p99.9
    Latency   1.38ms   1.19ms   1.56ms   6.39ms   16.60ms

  14889897 requests in 5.00s, 14889897 responses
  Throughput: 2.98M req/s
  Bandwidth:  360.57MB/s
  Status codes: 2xx=14889897, 3xx=0, 4xx=0, 5xx=0
  Latency samples: 14889897 / 14889897 responses (100.0%)
  Per-template: 4963007,4940787,4986103
  Per-template-ok: 4963007,4940787,4986103
[info] CPU 5979.4% | Mem 2.7GiB

[run 2/3]
gcannon v0.8.0
  Target:    localhost:8080/
  Threads:   64
  Conns:     4096 (64/thread)
  Pipeline:  1
  Req/conn:  unlimited (keep-alive)
  Templates: 3
  Expected:  200
  Duration:  5s


  Thread Stats   Avg      p50      p90      p99    p99.9
    Latency   1.18ms   1.16ms   1.43ms   1.95ms   6.19ms

  17340744 requests in 5.00s, 17340744 responses
  Throughput: 3.47M req/s
  Bandwidth:  419.93MB/s
  Status codes: 2xx=17340744, 3xx=0, 4xx=0, 5xx=0
  Latency samples: 17340744 / 17340744 responses (100.0%)
  Per-template: 5795557,5769113,5776074
  Per-template-ok: 5795557,5769113,5776074
[info] CPU 6460.7% | Mem 2.1GiB

[run 3/3]
gcannon v0.8.0
  Target:    localhost:8080/
  Threads:   64
  Conns:     4096 (64/thread)
  Pipeline:  1
  Req/conn:  unlimited (keep-alive)
  Templates: 3
  Expected:  200
  Duration:  5s


  Thread Stats   Avg      p50      p90      p99    p99.9
    Latency   1.17ms   1.16ms   1.42ms   1.90ms   4.71ms

  17511399 requests in 5.00s, 17511399 responses
  Throughput: 3.50M req/s
  Bandwidth:  424.07MB/s
  Status codes: 2xx=17511399, 3xx=0, 4xx=0, 5xx=0
  Latency samples: 17511399 / 17511399 responses (100.0%)
  Per-template: 5866008,5837636,5807755
  Per-template-ok: 5866008,5837636,5807755
[info] CPU 6160.4% | Mem 2.2GiB

=== Best: 3502279 req/s (CPU: 6160.4%, Mem: 2.2GiB) ===
[info] input BW: 270.54MB/s (avg template: 81 bytes)
[info] saved results/baseline/4096/genhttp-ioxide.json
httparena-bench-genhttp-ioxide
httparena-bench-genhttp-ioxide
[info] rebuilding site/data/*.json
[updated] /home/diogo/actions-runner/_work/HttpArena/HttpArena/site/data/frameworks.json
[updated] /home/diogo/actions-runner/_work/HttpArena/HttpArena/site/data/results/genhttp-ioxide.json - 1 new, 33 total
[updated] /home/diogo/actions-runner/_work/HttpArena/HttpArena/site/data/current.json
[info] done
httparena-postgres
[info] restoring loopback MTU to 65536

It listens on 0.0.0.0 (validate's 127.0.0.2/.32 probe passes) and GenHTTP
sets ioxide's read timeout to 0, so a held connection is never reaped
between its requests 4 s apart.
…nHTTP#953

5967f5c6b, one commit past 28b0cd107. It adds acceptance tests only:
Engine/Ioxide and the modules this entry builds are unchanged, and still
on ioxide 0.26.563.
@MDA2AV

MDA2AV commented Nov 16, 2026

Copy link
Copy Markdown
Owner Author

/benchmark -f genhttp-ioxide -t connections-1m

@github-actions

Copy link
Copy Markdown
Contributor

👋 Benchmark request received. A collaborator will review and approve the run.

@github-actions

Copy link
Copy Markdown
Contributor

Benchmark Results

Framework: genhttp-ioxide | Test: connections-1m

Test Conn RPS Rate CPU/req p99 p99.9 CPU Mem Δ RPS Δ Mem
connections-1m 1000000 251,952 1.0078 32.15us 218400.0us 223900.0us 979.6% 16.3GiB NEW NEW

Composite score, main against this run, in the board's default view:

Family main this run Δ
HTTP/1.1 7074.6 (3 of 109) 7074.6 (3 of 109) +0.0
HTTP/2 3321.9 (1 of 41) 3321.9 (1 of 41) +0.0
HTTP/3 1593.6 (2 of 20) 1593.6 (2 of 20) +0.0
WebSocket 2532.9 (5 of 39) 2532.9 (5 of 39) +0.0

The results on main were measured in another session, so a few points up or down can be the machine and not the change.

Full log
   "4xx": 0,
   "5xx": 0,
   "other": 0
  },
  "errors": {
   "connect": 0,
   "read": 0,
   "timeout": 0
  },
  "reconnects": 0,
  "rate": {
   "target": 250000,
   "achieved": 251957.54,
   "ratio": 1.0078,
   "paced_sends": 5026479,
   "late": 4766026,
   "max_lag_us": 160617
  },
  "connections_active": 1000000
 }
}
[info] CPU 979.6% | Mem 16.3GiB

[run 2/3]
gcannon: warmup done, measuring for 20s
gcannon: measurement done
[conn-hold] 1000000 of 1000000 connections open at the end of the window
{
 "connections": 1000000,
 "connections_open": 1000000,
 "target_rate": 250000,
 "rps": 251323.68,
 "rate_ratio": 1.0053,
 "duration_s": 20.023,
 "bytes_per_sec": 31666843.03,
 "max_schedule_lag_us": 164497,
 "latency_us": {
  "mean": 157668,
  "p50": 150500,
  "p90": 205000,
  "p99": 226500,
  "p99_9": 275100
 },
 "status_codes": {
  "2xx": 5032254,
  "3xx": 0,
  "4xx": 0,
  "5xx": 0
 },
 "errors": {
  "connect": 0,
  "read": 0,
  "timeout": 0
 },
 "reconnects": 0,
 "server_cpu_usec": 165700861,
 "gcannon": {
  "version": "0.8.0",
  "target": "127.0.0.1:8080/baseline11?a=1&b=2",
  "mode": "http",
  "connections": 1000000,
  "threads": 64,
  "pipeline_depth": 1,
  "duration_sec": 20,
  "elapsed_sec": 20.023,
  "requests": 5018753,
  "responses": 5032254,
  "rps": 251324.15,
  "bandwidth_bps": 31666843.03,
  "bytes_read": 634064004,
  "latency_us": {
   "avg": 157668,
   "p50": 150500,
   "p90": 205000,
   "p99": 226500,
   "p999": 275100,
   "samples": 5032254
  },
  "status": {
   "2xx": 5032254,
   "3xx": 0,
   "4xx": 0,
   "5xx": 0,
   "other": 0
  },
  "errors": {
   "connect": 0,
   "read": 0,
   "timeout": 0
  },
  "reconnects": 0,
  "rate": {
   "target": 250000,
   "achieved": 251324.15,
   "ratio": 1.0053,
   "paced_sends": 5018753,
   "late": 4764505,
   "max_lag_us": 164497
  },
  "connections_active": 1000000
 }
}
[info] CPU 907.0% | Mem 22.1GiB

[run 3/3]
gcannon: warmup done, measuring for 20s
gcannon: measurement done
[conn-hold] 1000000 of 1000000 connections open at the end of the window
{
 "connections": 1000000,
 "connections_open": 1000000,
 "target_rate": 250000,
 "rps": 251553.4,
 "rate_ratio": 1.0062,
 "duration_s": 20.056,
 "bytes_per_sec": 31696380.04,
 "max_schedule_lag_us": 165096,
 "latency_us": {
  "mean": 157058,
  "p50": 149700,
  "p90": 205000,
  "p99": 226300,
  "p99_9": 230000
 },
 "status_codes": {
  "2xx": 5045155,
  "3xx": 0,
  "4xx": 0,
  "5xx": 0
 },
 "errors": {
  "connect": 0,
  "read": 0,
  "timeout": 0
 },
 "reconnects": 0,
 "server_cpu_usec": 167390397,
 "gcannon": {
  "version": "0.8.0",
  "target": "127.0.0.1:8080/baseline11?a=1&b=2",
  "mode": "http",
  "connections": 1000000,
  "threads": 64,
  "pipeline_depth": 1,
  "duration_sec": 20,
  "elapsed_sec": 20.056,
  "requests": 5031468,
  "responses": 5045155,
  "rps": 251558.57,
  "bandwidth_bps": 31696380.04,
  "bytes_read": 635689530,
  "latency_us": {
   "avg": 157058,
   "p50": 149700,
   "p90": 205000,
   "p99": 226300,
   "p999": 230000,
   "samples": 5045155
  },
  "status": {
   "2xx": 5045155,
   "3xx": 0,
   "4xx": 0,
   "5xx": 0,
   "other": 0
  },
  "errors": {
   "connect": 0,
   "read": 0,
   "timeout": 0
  },
  "reconnects": 0,
  "rate": {
   "target": 250000,
   "achieved": 251558.57,
   "ratio": 1.0062,
   "paced_sends": 5031468,
   "late": 4774129,
   "max_lag_us": 165096
  },
  "connections_active": 1000000
 }
}
[info] CPU 1014.0% | Mem 23.6GiB
[info] run 1 wins on score

=== Best: 251952 req/s (CPU: 979.6%, Mem: 16.3GiB) ===
[info] exact CPU: 162.08 core-seconds | 32.150 us/req
=== Fixed-rate: cpu/req 32.15us | p99 218400.0us | p99.9 223900.0us | rate_ratio 1.0078 ===
=== Connections: 1000000 open of 1000000 | connect errors in the window 0 ===
[info] saved results/connections-1m/1000000/genhttp-ioxide.json
httparena-bench-genhttp-ioxide
httparena-bench-genhttp-ioxide
[info] rebuilding site/data/*.json
[updated] /home/diogo/actions-runner/_work/HttpArena/HttpArena/site/data/frameworks.json
[updated] /home/diogo/actions-runner/_work/HttpArena/HttpArena/site/data/results/genhttp-ioxide.json - 1 new, 34 total
[updated] /home/diogo/actions-runner/_work/HttpArena/HttpArena/site/data/current.json
[info] done
httparena-postgres
[info] restoring loopback MTU to 65536

A twin of connections-1m with zrk as the load generator instead of gcannon,
to compare what the two report on one load: 1M held connections, 250K req/s,
the same cpusets, 64 client threads, a 20s window, the same score.

zrk 3.0.2 shares its latency histograms per thread (zoxy-io/zrk#102). Measured
on the dev box against actix at one request per connection every 4 s: ~41 KiB
resident per connection, flat from 4K to 16K (2.3.0: ~305 KiB at 1K), so the
million needs ~40 GiB. The other zrk profiles stay on the pinned 2.3.0; this
one builds zrk:3.0.2 from the same Dockerfile and refuses an older zrk.

zrk takes one URL, so scripts/zrk_fanout.py (restored from d9a19a7) runs one
process per address 127.0.0.1-.32, starts them 1 s apart, discards 10 s,
measures 20 s, merges the per-second HdrHistograms bucket by bucket, and
counts the server's established sockets with ss in a 20 s tail. Its output
uses conn_hold.py's keys (hold_error, connections_open, connect_errors,
cpu_usec_window), so benchmark.sh treats both profiles alike (conn1m_profile).
vm.max_map_count is raised again: zrk maps ~4 regions per connection.

Subscribed: the six connections-1m entries.
@MDA2AV

MDA2AV commented Nov 17, 2026

Copy link
Copy Markdown
Owner Author

/benchmark -f genhttp-ioxide -t connections-1m-zrk

@github-actions

Copy link
Copy Markdown
Contributor

👋 Benchmark request received. A collaborator will review and approve the run.

@github-actions

Copy link
Copy Markdown
Contributor

Benchmark Results

Framework: genhttp-ioxide | Test: connections-1m-zrk

Test Conn RPS Rate CPU/req p99 p99.9 CPU Mem Δ RPS Δ Mem
connections-1m-zrk 1000000 250,000 1.0000 35.24us 730.0us 2843.0us 885.1% 16.9GiB NEW NEW

Composite score, main against this run, in the board's default view:

Family main this run Δ
HTTP/1.1 7074.6 (3 of 109) 7074.6 (3 of 109) +0.0
HTTP/2 3321.9 (1 of 41) 3321.9 (1 of 41) +0.0
HTTP/3 1593.6 (2 of 20) 1593.6 (2 of 20) +0.0
WebSocket 2532.9 (5 of 39) 2532.9 (5 of 39) +0.0

The results on main were measured in another session, so a few points up or down can be the machine and not the change.

Full log
    "rate": 7813,
    "launched": 31250,
    "window_requests": 156254,
    "window_s": 20.0
   },
   {
    "addr": "127.0.0.16",
    "connections": 31250,
    "rate": 7813,
    "launched": 31250,
    "window_requests": 156255,
    "window_s": 20.0
   },
   {
    "addr": "127.0.0.17",
    "connections": 31250,
    "rate": 7812,
    "launched": 31250,
    "window_requests": 156236,
    "window_s": 20.0
   },
   {
    "addr": "127.0.0.18",
    "connections": 31250,
    "rate": 7812,
    "launched": 31250,
    "window_requests": 156240,
    "window_s": 20.0
   },
   {
    "addr": "127.0.0.19",
    "connections": 31250,
    "rate": 7812,
    "launched": 31250,
    "window_requests": 156237,
    "window_s": 20.0
   },
   {
    "addr": "127.0.0.20",
    "connections": 31250,
    "rate": 7812,
    "launched": 31250,
    "window_requests": 156241,
    "window_s": 20.0
   },
   {
    "addr": "127.0.0.21",
    "connections": 31250,
    "rate": 7812,
    "launched": 31250,
    "window_requests": 156242,
    "window_s": 20.0
   },
   {
    "addr": "127.0.0.22",
    "connections": 31250,
    "rate": 7812,
    "launched": 31250,
    "window_requests": 156238,
    "window_s": 20.0
   },
   {
    "addr": "127.0.0.23",
    "connections": 31250,
    "rate": 7812,
    "launched": 31250,
    "window_requests": 156231,
    "window_s": 20.0
   },
   {
    "addr": "127.0.0.24",
    "connections": 31250,
    "rate": 7812,
    "launched": 31250,
    "window_requests": 156241,
    "window_s": 20.0
   },
   {
    "addr": "127.0.0.25",
    "connections": 31250,
    "rate": 7812,
    "launched": 31250,
    "window_requests": 156246,
    "window_s": 20.0
   },
   {
    "addr": "127.0.0.26",
    "connections": 31250,
    "rate": 7812,
    "launched": 31250,
    "window_requests": 156232,
    "window_s": 20.0
   },
   {
    "addr": "127.0.0.27",
    "connections": 31250,
    "rate": 7812,
    "launched": 31250,
    "window_requests": 156235,
    "window_s": 20.0
   },
   {
    "addr": "127.0.0.28",
    "connections": 31250,
    "rate": 7812,
    "launched": 31250,
    "window_requests": 156242,
    "window_s": 20.0
   },
   {
    "addr": "127.0.0.29",
    "connections": 31250,
    "rate": 7812,
    "launched": 31250,
    "window_requests": 156247,
    "window_s": 20.0
   },
   {
    "addr": "127.0.0.30",
    "connections": 31250,
    "rate": 7812,
    "launched": 31250,
    "window_requests": 156240,
    "window_s": 20.0
   },
   {
    "addr": "127.0.0.31",
    "connections": 31250,
    "rate": 7812,
    "launched": 31250,
    "window_requests": 156238,
    "window_s": 20.0
   },
   {
    "addr": "127.0.0.32",
    "connections": 31250,
    "rate": 7812,
    "launched": 31250,
    "window_requests": 156238,
    "window_s": 20.0
   }
  ]
 },
 "connections": 1000000,
 "launched": 1000000,
 "connections_open": 1000000,
 "duration_s": 20.0,
 "requests": 4999947,
 "bytes": 629993322,
 "achieved_rate": 249997.35,
 "target_rate": 250000,
 "rate_ratio": 1.0,
 "bytes_per_sec": 31499666.1,
 "max_schedule_lag_us": 0,
 "server_cpu_usec": 186069857,
 "latency_us": {
  "min": 31,
  "mean": 115.2,
  "max": 7779,
  "p50": 70,
  "p75": 82,
  "p90": 99,
  "p99": 2117,
  "p99_9": 4406,
  "p99_99": 5294
 },
 "status_codes": {
  "1xx": 0,
  "2xx": 4999947,
  "3xx": 0,
  "4xx": 0,
  "5xx": 0
 },
 "errors": {
  "connect": 0,
  "read": 0,
  "write": 0,
  "timeout": 0,
  "deadline": 0,
  "non_2xx_3xx": 0
 },
 "ramp_connect_errors": 0
}
[info] CPU 959.2% | Mem 21.3GiB
[info] run 1 wins on score

=== Best: 250000 req/s (CPU: 885.1%, Mem: 16.9GiB) ===
[info] exact CPU: 176.21 core-seconds | 35.241 us/req
=== Fixed-rate: cpu/req 35.24us | p99 730.0us | p99.9 2843.0us | rate_ratio 1.0000 ===
=== Connections: 1000000 open of 1000000 | connect errors in the window 0 ===
[info] saved results/connections-1m-zrk/1000000/genhttp-ioxide.json
httparena-bench-genhttp-ioxide
httparena-bench-genhttp-ioxide
[info] rebuilding site/data/*.json
[updated] /home/diogo/actions-runner/_work/HttpArena/HttpArena/site/data/frameworks.json
[updated] /home/diogo/actions-runner/_work/HttpArena/HttpArena/site/data/results/genhttp-ioxide.json - 1 new, 34 total
[updated] /home/diogo/actions-runner/_work/HttpArena/HttpArena/site/data/current.json
[info] done
httparena-postgres
[info] restoring loopback MTU to 65536

gcannon 0.8.0's --rate scheduled each thread's connections inside one 1/T
slice of the interval, so one thread at a time sent the whole fleet's rate.
At a million connections, 64 threads and 250K req/s nearly every send went
out 100-165 ms late, and p50 read ~150 ms on every server, where zrk read
70 us on the same entry. 0.8.1 spreads every thread over the whole interval
(MDA2AV/gcannon#30). conn_hold.py now refuses an older gcannon, so a host
still on 0.8.0 reports why instead of publishing its latencies.
@MDA2AV

MDA2AV commented Nov 18, 2026

Copy link
Copy Markdown
Owner Author

/benchmark -f genhttp-ioxide -t baseline

@github-actions

Copy link
Copy Markdown
Contributor

👋 Benchmark request received. A collaborator will review and approve the run.

@github-actions

Copy link
Copy Markdown
Contributor

Benchmark Results

Framework: genhttp-ioxide | Test: baseline

Test Conn RPS CPU Mem Δ RPS Δ Mem
baseline 4096 3,577,455 6267.6% 2.3GiB +0.8% ~0%

Composite score, main against this run, in the board's default view:

Family main this run Δ
HTTP/1.1 7074.6 (3 of 109) 7074.6 (3 of 109) +0.0
HTTP/2 3321.9 (1 of 41) 3321.9 (1 of 41) +0.0
HTTP/3 1593.6 (2 of 20) 1593.6 (2 of 20) +0.0
WebSocket 2532.9 (5 of 39) 2532.9 (5 of 39) +0.0

The results on main were measured in another session, so a few points up or down can be the machine and not the change.

Full log
[info] available CPUs: 128
[info] framework: genhttp-ioxide (genhttp-ioxide, C#)
[info] subscribed tests: baseline,async,latency-1m,connections-1m,connections-1m-zrk,latency-10k,latency-500k-8cpu,pipelined,limited-conn,json-comp,json-tls,static-tls,baseline-h2c,json-h2c,baseline-h2,static-h2,static-h3,baseline-h3,8gbit,async-db,echo-ws,echo-ws-pipeline,echo-ws-limited
[info] building image: httparena-genhttp-ioxide
#0 building with "default" instance using docker driver

#1 [internal] load build definition from Dockerfile
#1 transferring dockerfile: 945B done
#1 DONE 0.0s

#2 [internal] load metadata for mcr.microsoft.com/dotnet/sdk:11.0
#2 DONE 0.3s

#3 [internal] load metadata for mcr.microsoft.com/dotnet/runtime:11.0
#3 DONE 0.3s

#4 [internal] load .dockerignore
#4 transferring context: 69B done
#4 DONE 0.0s

#5 [stage-1 1/3] FROM mcr.microsoft.com/dotnet/runtime:11.0@sha256:3fe5f5495141c97986a49d3a4e76061f6225c3f93f94e8a7e522781376193863
#5 resolve mcr.microsoft.com/dotnet/runtime:11.0@sha256:3fe5f5495141c97986a49d3a4e76061f6225c3f93f94e8a7e522781376193863 0.1s done
#5 DONE 0.1s

#6 [build 1/7] FROM mcr.microsoft.com/dotnet/sdk:11.0@sha256:65f3c7c6a25a239ab3e6719f2896147e7b68242ff2d8818dd70498bde6ef8bea
#6 resolve mcr.microsoft.com/dotnet/sdk:11.0@sha256:65f3c7c6a25a239ab3e6719f2896147e7b68242ff2d8818dd70498bde6ef8bea 0.1s done
#6 DONE 0.1s

#7 [internal] load build context
#7 transferring context: 38.05kB done
#7 DONE 0.0s

#8 [build 3/7] WORKDIR /src/app
#8 CACHED

#9 [build 7/7] RUN dotnet publish genhttp.csproj -c Release --no-self-contained -o /app
#9 CACHED

#10 [stage-1 2/3] WORKDIR /app
#10 CACHED

#11 [build 2/7] RUN git init -q /genhttp  && git -C /genhttp fetch -q --depth 1 https://github.com/Kaliumhexacyanoferrat/GenHTTP.git "5967f5c6bf64d3d67a0eed7a669fa2f8c846dd02"  && git -C /genhttp checkout -q FETCH_HEAD
#11 CACHED

#12 [build 4/7] COPY genhttp.csproj ./
#12 CACHED

#13 [build 6/7] COPY . .
#13 CACHED

#14 [build 5/7] RUN dotnet restore
#14 CACHED

#15 [stage-1 3/3] COPY --from=build /app .
#15 CACHED

#16 exporting to image
#16 exporting layers done
#16 exporting manifest sha256:52b647029798fc2ae0a1b59cd81368af2fb6f5c803eee64593a06c41d99c9b8f done
#16 exporting config sha256:7ddae00b438e4f8eb2a61ebc51e19ff288b2278636891c9623016cfd6f0a0d1b done
#16 exporting attestation manifest sha256:f5e4342cf892c1d5b3fe74cce443fe5eabd223a881742cee8ef5455c921b6fd5 0.0s done
#16 exporting manifest list sha256:8401674462a0b5a2c3727f59c369369d5b0ffac9122fc0d0247c8c6ae29717e9
#16 exporting manifest list sha256:8401674462a0b5a2c3727f59c369369d5b0ffac9122fc0d0247c8c6ae29717e9 0.0s done
#16 naming to docker.io/library/httparena-genhttp-ioxide:latest done
#16 unpacking to docker.io/library/httparena-genhttp-ioxide:latest 0.0s done
#16 DONE 0.2s
[info] zrk: docker mode (zrk:local)
[info] tuning host for benchmark runs
[info] CPU governor → performance
[info] setting kernel socket limits
[info] setting UDP buffer sizes for QUIC
[info] setting loopback MTU to 1500 (realistic Ethernet)
[info] restarting docker daemon
[info] dropping kernel caches
[info] starting postgres sidecar
[info] postgres ready (seeded)

==============================================
=== genhttp-ioxide / baseline / 4096c (tool=gcannon) ===
==============================================
[info] waiting for server...
[info] server ready

[run 1/3]
gcannon v0.8.1
  Target:    localhost:8080/
  Threads:   64
  Conns:     4096 (64/thread)
  Pipeline:  1
  Req/conn:  unlimited (keep-alive)
  Templates: 3
  Expected:  200
  Duration:  5s


  Thread Stats   Avg      p50      p90      p99    p99.9
    Latency   1.36ms   1.17ms   1.51ms   6.10ms   20.10ms

  15081602 requests in 5.00s, 15081602 responses
  Throughput: 3.02M req/s
  Bandwidth:  365.21MB/s
  Status codes: 2xx=15081602, 3xx=0, 4xx=0, 5xx=0
  Latency samples: 15081602 / 15081602 responses (100.0%)
  Per-template: 5025377,5034750,5021475
  Per-template-ok: 5025377,5034750,5021475
[info] CPU 5867.8% | Mem 2.7GiB

[run 2/3]
gcannon v0.8.1
  Target:    localhost:8080/
  Threads:   64
  Conns:     4096 (64/thread)
  Pipeline:  1
  Req/conn:  unlimited (keep-alive)
  Templates: 3
  Expected:  200
  Duration:  5s


  Thread Stats   Avg      p50      p90      p99    p99.9
    Latency   1.17ms   1.13ms   1.42ms   3.90ms   5.98ms

  17477046 requests in 5.00s, 17477046 responses
  Throughput: 3.49M req/s
  Bandwidth:  423.24MB/s
  Status codes: 2xx=17477046, 3xx=0, 4xx=0, 5xx=0
  Latency samples: 17477046 / 17477046 responses (100.0%)
  Per-template: 5867449,5807530,5802067
  Per-template-ok: 5867449,5807530,5802067
[info] CPU 6449.2% | Mem 2.2GiB

[run 3/3]
gcannon v0.8.1
  Target:    localhost:8080/
  Threads:   64
  Conns:     4096 (64/thread)
  Pipeline:  1
  Req/conn:  unlimited (keep-alive)
  Templates: 3
  Expected:  200
  Duration:  5s


  Thread Stats   Avg      p50      p90      p99    p99.9
    Latency   1.14ms   1.14ms   1.40ms   1.79ms   4.50ms

  17887279 requests in 5.00s, 17887279 responses
  Throughput: 3.58M req/s
  Bandwidth:  433.13MB/s
  Status codes: 2xx=17887279, 3xx=0, 4xx=0, 5xx=0
  Latency samples: 17887279 / 17887279 responses (100.0%)
  Per-template: 5966798,5962213,5958268
  Per-template-ok: 5966798,5962213,5958268
[info] CPU 6267.6% | Mem 2.3GiB

=== Best: 3577455 req/s (CPU: 6267.6%, Mem: 2.3GiB) ===
[info] input BW: 276.35MB/s (avg template: 81 bytes)
[info] saved results/baseline/4096/genhttp-ioxide.json
httparena-bench-genhttp-ioxide
httparena-bench-genhttp-ioxide
[info] rebuilding site/data/*.json
[updated] /home/diogo/actions-runner/_work/HttpArena/HttpArena/site/data/frameworks.json
[updated] /home/diogo/actions-runner/_work/HttpArena/HttpArena/site/data/results/genhttp-ioxide.json - 1 new, 33 total
[updated] /home/diogo/actions-runner/_work/HttpArena/HttpArena/site/data/current.json
[info] done
httparena-postgres
[info] restoring loopback MTU to 65536

@MDA2AV

MDA2AV commented Nov 18, 2026

Copy link
Copy Markdown
Owner Author

/benchmark -f genhttp-ioxide -t connections-1m

@github-actions

Copy link
Copy Markdown
Contributor

👋 Benchmark request received. A collaborator will review and approve the run.

@github-actions

Copy link
Copy Markdown
Contributor

Benchmark Results

Framework: genhttp-ioxide | Test: connections-1m

Test Conn RPS Rate CPU/req p99 p99.9 CPU Mem Δ RPS Δ Mem
connections-1m 1000000 249,981 0.9999 32.43us 93.0us 1457.0us 955.5% 22.1GiB NEW NEW

Composite score, main against this run, in the board's default view:

Family main this run Δ
HTTP/1.1 7074.6 (3 of 109) 7074.6 (3 of 109) +0.0
HTTP/2 3321.9 (1 of 41) 3321.9 (1 of 41) +0.0
HTTP/3 1593.6 (2 of 20) 1593.6 (2 of 20) +0.0
WebSocket 2532.9 (5 of 39) 2532.9 (5 of 39) +0.0

The results on main were measured in another session, so a few points up or down can be the machine and not the change.

Full log
   "4xx": 0,
   "5xx": 0,
   "other": 0
  },
  "errors": {
   "connect": 0,
   "read": 0,
   "timeout": 0
  },
  "reconnects": 0,
  "rate": {
   "target": 250000,
   "achieved": 250095.45,
   "ratio": 1.0004,
   "paced_sends": 5000348,
   "late": 0,
   "max_lag_us": 172
  },
  "connections_active": 1000000
 }
}
[info] CPU 977.8% | Mem 16.3GiB

[run 2/3]
gcannon: warmup done, measuring for 20s
gcannon: measurement done
[conn-hold] 1000000 of 1000000 connections open at the end of the window
{
 "connections": 1000000,
 "connections_open": 1000000,
 "target_rate": 250000,
 "rps": 249980.7,
 "rate_ratio": 0.9999,
 "duration_s": 20.003,
 "bytes_per_sec": 31498033.14,
 "max_schedule_lag_us": 193,
 "latency_us": {
  "mean": 58,
  "p50": 51,
  "p90": 62,
  "p99": 93,
  "p99_9": 1457
 },
 "status_codes": {
  "2xx": 5000364,
  "3xx": 0,
  "4xx": 0,
  "5xx": 0
 },
 "errors": {
  "connect": 0,
  "read": 0,
  "timeout": 0
 },
 "reconnects": 0,
 "server_cpu_usec": 162177990,
 "gcannon": {
  "version": "0.8.1",
  "target": "127.0.0.1:8080/baseline11?a=1&b=2",
  "mode": "http",
  "connections": 1000000,
  "threads": 64,
  "pipeline_depth": 1,
  "duration_sec": 20,
  "elapsed_sec": 20.003,
  "requests": 5000365,
  "responses": 5000364,
  "rps": 249984.39,
  "bandwidth_bps": 31498033.14,
  "bytes_read": 630045864,
  "latency_us": {
   "avg": 58,
   "p50": 51,
   "p90": 62,
   "p99": 93,
   "p999": 1457,
   "samples": 5000364
  },
  "status": {
   "2xx": 5000364,
   "3xx": 0,
   "4xx": 0,
   "5xx": 0,
   "other": 0
  },
  "errors": {
   "connect": 0,
   "read": 0,
   "timeout": 0
  },
  "reconnects": 0,
  "rate": {
   "target": 250000,
   "achieved": 249984.39,
   "ratio": 0.9999,
   "paced_sends": 5000365,
   "late": 0,
   "max_lag_us": 193
  },
  "connections_active": 1000000
 }
}
[info] CPU 955.5% | Mem 22.1GiB

[run 3/3]
gcannon: warmup done, measuring for 20s
gcannon: measurement done
[conn-hold] 1000000 of 1000000 connections open at the end of the window
{
 "connections": 1000000,
 "connections_open": 1000000,
 "target_rate": 250000,
 "rps": 249979.25,
 "rate_ratio": 0.9999,
 "duration_s": 20.003,
 "bytes_per_sec": 31498062.51,
 "max_schedule_lag_us": 601,
 "latency_us": {
  "mean": 65,
  "p50": 51,
  "p90": 63,
  "p99": 222,
  "p99_9": 2256
 },
 "status_codes": {
  "2xx": 5000335,
  "3xx": 0,
  "4xx": 0,
  "5xx": 0
 },
 "errors": {
  "connect": 0,
  "read": 0,
  "timeout": 0
 },
 "reconnects": 0,
 "server_cpu_usec": 166298487,
 "gcannon": {
  "version": "0.8.1",
  "target": "127.0.0.1:8080/baseline11?a=1&b=2",
  "mode": "http",
  "connections": 1000000,
  "threads": 64,
  "pipeline_depth": 1,
  "duration_sec": 20,
  "elapsed_sec": 20.003,
  "requests": 5000331,
  "responses": 5000335,
  "rps": 249984.62,
  "bandwidth_bps": 31498062.51,
  "bytes_read": 630042210,
  "latency_us": {
   "avg": 65,
   "p50": 51,
   "p90": 63,
   "p99": 222,
   "p999": 2256,
   "samples": 5000335
  },
  "status": {
   "2xx": 5000335,
   "3xx": 0,
   "4xx": 0,
   "5xx": 0,
   "other": 0
  },
  "errors": {
   "connect": 0,
   "read": 0,
   "timeout": 0
  },
  "reconnects": 0,
  "rate": {
   "target": 250000,
   "achieved": 249984.62,
   "ratio": 0.9999,
   "paced_sends": 5000331,
   "late": 0,
   "max_lag_us": 601
  },
  "connections_active": 1000000
 }
}
[info] CPU 1027.0% | Mem 23.8GiB
[info] run 2 wins on score

=== Best: 249981 req/s (CPU: 955.5%, Mem: 22.1GiB) ===
[info] exact CPU: 162.18 core-seconds | 32.433 us/req
=== Fixed-rate: cpu/req 32.43us | p99 93.0us | p99.9 1457.0us | rate_ratio 0.9999 ===
=== Connections: 1000000 open of 1000000 | connect errors in the window 0 ===
[info] saved results/connections-1m/1000000/genhttp-ioxide.json
httparena-bench-genhttp-ioxide
httparena-bench-genhttp-ioxide
[info] rebuilding site/data/*.json
[updated] /home/diogo/actions-runner/_work/HttpArena/HttpArena/site/data/frameworks.json
[updated] /home/diogo/actions-runner/_work/HttpArena/HttpArena/site/data/results/genhttp-ioxide.json - 1 new, 34 total
[updated] /home/diogo/actions-runner/_work/HttpArena/HttpArena/site/data/current.json
[info] done
httparena-postgres
[info] restoring loopback MTU to 65536

connections-1m becomes connections-5m: 5,000,000 keep-alive connections,
GET /baseline11 paced at 1,000,000 req/s across them (one request per
connection every 5 s), on gcannon >= 0.8.1. connections-5m-tls is the same
fleet over TLS 1.3 on the h1 TLS port (8081): gcannon handshakes with OpenSSL,
hands the session to kernel TLS and frees its own state. connections-1m-zrk
stays at a million (zrk's ~41 KiB per connection rules out five).

- 160 loopback addresses, 31,250 connections each (the density the 1M runs
  held); validate.sh probes .2 and .160, and the TLS port over TLS 1.3.
- Server caps: nofile 8,388,608, memory 128g (112g over TLS), via
  held_conn_limits; fs.nr_open and conntrack raised to 16M, hashsize 4M.
- conn_hold.py: per-profile messages and kernel-memory estimate, logs the
  memory budget, and raises gcannon's open-file limit past the runner's hard
  limit with `sudo -n prlimit` (with the reason when it cannot).
- Warmup up to 300 s (900 s over TLS: five million handshakes); 60 s between
  runs at 2M+ connections.
- Full credit at 950,000 req/s; docs move to test-profiles/h1/connections-5m.

Tested on the dev box through gcannon_build_args -> gcannon_run ->
gcannon_parse with gcannon 0.8.1, 16,000 connections over all 160 addresses
against actix: plaintext and TLS both held 16,000 at rate 1.0000 (4 s
interval). At the profile's 5 s interval actix reconnected 51,410 times and
answered 4,221 4xx: its 5 s keep-alive and first-request timeouts race the
interval.
@MDA2AV MDA2AV changed the title connections-1m: one million held keep-alive connections [skip-maintainer-ping] connections-5m: five million held keep-alive connections, plaintext and TLS [skip-maintainer-ping] Nov 18, 2026
…rker

At 1M req/s over five million connections each connection sends every 5 s,
exactly the 5 s keep-alive default of Node, uvicorn and actix (actix's
first-request timeout too). On the dev box at 16,000 connections against
actix, a 5 s interval gave 51,410 reconnects and 4,221 4xx (rate 0.935); a
4 s interval none. 1.25M req/s makes it 4 s, inside those defaults, so the
timeouts stay as they are. Full credit moves to 1,187,500 req/s (95%).

actix-server stops accepting at 25,600 connections per worker (measured: 4
workers accepted exactly 102,400 of 120,000 offered), 1.64M on the bench
host's 64. max_connections(100_000) lifts it to 6.4M; the same test then
accepted all 120,000 at the full rate, no errors. The resilience suite's slow
attacks hold 32,768 connections, under even the default cap of its 2-CPU box,
so they never reach it.
@MDA2AV

MDA2AV commented Nov 18, 2026

Copy link
Copy Markdown
Owner Author

/benchmark -f genhttp-ioxide -t connections-5m

@github-actions

Copy link
Copy Markdown
Contributor

👋 Benchmark request received. A collaborator will review and approve the run.

@github-actions

Copy link
Copy Markdown
Contributor

Benchmark Results

Framework: genhttp-ioxide | Test: connections-5m

Test Conn RPS Rate CPU/req p99 p99.9 CPU Mem Δ RPS Δ Mem
connections-5m 5000000 1,282,613 1.0261 43.67us 728400.0us 938300.0us 4204.9% 79.1GiB NEW NEW

Composite score, main against this run, in the board's default view:

Family main this run Δ
HTTP/1.1 7074.6 (3 of 109) 7074.6 (3 of 109) +0.0
HTTP/2 3321.9 (1 of 41) 3321.9 (1 of 41) +0.0
HTTP/3 1593.6 (2 of 20) 1593.6 (2 of 20) +0.0
WebSocket 2532.9 (5 of 39) 2532.9 (5 of 39) +0.0

The results on main were measured in another session, so a few points up or down can be the machine and not the change.

Full log
#13 6.339 /root/.nuget/packages/microsoft.sourcelink.common/10.0.403/build/Microsoft.SourceLink.Common.targets(56,5): warning : Source control information is not available - the generated source link is empty. [/genhttp/Modules/Redirects/GenHTTP.Modules.Redirects.csproj::TargetFramework=net11.0]
#13 6.339 /root/.nuget/packages/microsoft.build.tasks.git/10.0.403/build/Microsoft.Build.Tasks.Git.targets(25,5): warning : Repository '/genhttp' has no remote. [/genhttp/Modules/Compression/GenHTTP.Modules.Compression.csproj::TargetFramework=net11.0]
#13 6.339   GenHTTP.Modules.Pages -> /genhttp/Modules/Pages/bin/Release/net11.0/GenHTTP.Modules.Pages.dll
#13 6.339 /root/.nuget/packages/microsoft.sourcelink.common/10.0.403/build/Microsoft.SourceLink.Common.targets(56,5): warning : Source control information is not available - the generated source link is empty. [/genhttp/Modules/Compression/GenHTTP.Modules.Compression.csproj::TargetFramework=net11.0]
#13 6.379   GenHTTP.Modules.Conversion -> /genhttp/Modules/Conversion/bin/Release/net11.0/GenHTTP.Modules.Conversion.dll
#13 6.459   GenHTTP.Modules.Redirects -> /genhttp/Modules/Redirects/bin/Release/net11.0/GenHTTP.Modules.Redirects.dll
#13 6.499   GenHTTP.Modules.Compression -> /genhttp/Modules/Compression/bin/Release/net11.0/GenHTTP.Modules.Compression.dll
#13 6.499 /root/.nuget/packages/microsoft.build.tasks.git/10.0.403/build/Microsoft.Build.Tasks.Git.targets(25,5): warning : Repository '/genhttp' has no remote. [/genhttp/Modules/Websockets/GenHTTP.Modules.Websockets.csproj::TargetFramework=net11.0]
#13 6.539 /root/.nuget/packages/microsoft.build.tasks.git/10.0.403/build/Microsoft.Build.Tasks.Git.targets(25,5): warning : Repository '/genhttp' has no remote. [/genhttp/Modules/Reflection/GenHTTP.Modules.Reflection.csproj::TargetFramework=net11.0]
#13 6.539 /root/.nuget/packages/microsoft.sourcelink.common/10.0.403/build/Microsoft.SourceLink.Common.targets(56,5): warning : Source control information is not available - the generated source link is empty. [/genhttp/Modules/Websockets/GenHTTP.Modules.Websockets.csproj::TargetFramework=net11.0]
#13 6.539 /root/.nuget/packages/microsoft.build.tasks.git/10.0.403/build/Microsoft.Build.Tasks.Git.targets(25,5): warning : Repository '/genhttp' has no remote. [/genhttp/Modules/ErrorHandling/GenHTTP.Modules.ErrorHandling.csproj::TargetFramework=net11.0]
#13 6.539 /root/.nuget/packages/microsoft.sourcelink.common/10.0.403/build/Microsoft.SourceLink.Common.targets(56,5): warning : Source control information is not available - the generated source link is empty. [/genhttp/Modules/Reflection/GenHTTP.Modules.Reflection.csproj::TargetFramework=net11.0]
#13 6.539 /root/.nuget/packages/microsoft.sourcelink.common/10.0.403/build/Microsoft.SourceLink.Common.targets(56,5): warning : Source control information is not available - the generated source link is empty. [/genhttp/Modules/ErrorHandling/GenHTTP.Modules.ErrorHandling.csproj::TargetFramework=net11.0]
#13 6.740   GenHTTP.Modules.ErrorHandling -> /genhttp/Modules/ErrorHandling/bin/Release/net11.0/GenHTTP.Modules.ErrorHandling.dll
#13 6.740   GenHTTP.Modules.Websockets -> /genhttp/Modules/Websockets/bin/Release/net11.0/GenHTTP.Modules.Websockets.dll
#13 6.779 /root/.nuget/packages/microsoft.build.tasks.git/10.0.403/build/Microsoft.Build.Tasks.Git.targets(25,5): warning : Repository '/genhttp' has no remote. [/genhttp/Engine/Shared/GenHTTP.Engine.Shared.csproj::TargetFramework=net11.0]
#13 6.819 /root/.nuget/packages/microsoft.sourcelink.common/10.0.403/build/Microsoft.SourceLink.Common.targets(56,5): warning : Source control information is not available - the generated source link is empty. [/genhttp/Engine/Shared/GenHTTP.Engine.Shared.csproj::TargetFramework=net11.0]
#13 6.819 /root/.nuget/packages/microsoft.build.tasks.git/10.0.403/build/Microsoft.Build.Tasks.Git.targets(25,5): warning : Repository '/genhttp' has no remote. [/genhttp/Modules/Files/GenHTTP.Modules.Files.csproj::TargetFramework=net11.0]
#13 6.859 /root/.nuget/packages/microsoft.sourcelink.common/10.0.403/build/Microsoft.SourceLink.Common.targets(56,5): warning : Source control information is not available - the generated source link is empty. [/genhttp/Modules/Files/GenHTTP.Modules.Files.csproj::TargetFramework=net11.0]
#13 6.859   GenHTTP.Modules.Reflection -> /genhttp/Modules/Reflection/bin/Release/net11.0/GenHTTP.Modules.Reflection.dll
#13 6.940 /root/.nuget/packages/microsoft.build.tasks.git/10.0.403/build/Microsoft.Build.Tasks.Git.targets(25,5): warning : Repository '/genhttp' has no remote. [/genhttp/Modules/Layouting/GenHTTP.Modules.Layouting.csproj::TargetFramework=net11.0]
#13 6.940 /root/.nuget/packages/microsoft.sourcelink.common/10.0.403/build/Microsoft.SourceLink.Common.targets(56,5): warning : Source control information is not available - the generated source link is empty. [/genhttp/Modules/Layouting/GenHTTP.Modules.Layouting.csproj::TargetFramework=net11.0]
#13 6.978   GenHTTP.Engine.Shared -> /genhttp/Engine/Shared/bin/Release/net11.0/GenHTTP.Engine.Shared.dll
#13 7.019   GenHTTP.Modules.Layouting -> /genhttp/Modules/Layouting/bin/Release/net11.0/GenHTTP.Modules.Layouting.dll
#13 7.019   GenHTTP.Modules.Files -> /genhttp/Modules/Files/bin/Release/net11.0/GenHTTP.Modules.Files.dll
#13 7.099 /root/.nuget/packages/microsoft.build.tasks.git/10.0.403/build/Microsoft.Build.Tasks.Git.targets(25,5): warning : Repository '/genhttp' has no remote. [/genhttp/Engine/Ioxide/GenHTTP.Engine.Ioxide.csproj::TargetFramework=net11.0]
#13 7.099 /root/.nuget/packages/microsoft.sourcelink.common/10.0.403/build/Microsoft.SourceLink.Common.targets(56,5): warning : Source control information is not available - the generated source link is empty. [/genhttp/Engine/Ioxide/GenHTTP.Engine.Ioxide.csproj::TargetFramework=net11.0]
#13 7.139 /root/.nuget/packages/microsoft.build.tasks.git/10.0.403/build/Microsoft.Build.Tasks.Git.targets(25,5): warning : Repository '/genhttp' has no remote. [/genhttp/Modules/Webservices/GenHTTP.Modules.Webservices.csproj::TargetFramework=net11.0]
#13 7.139 /root/.nuget/packages/microsoft.sourcelink.common/10.0.403/build/Microsoft.SourceLink.Common.targets(56,5): warning : Source control information is not available - the generated source link is empty. [/genhttp/Modules/Webservices/GenHTTP.Modules.Webservices.csproj::TargetFramework=net11.0]
#13 7.299   GenHTTP.Modules.Webservices -> /genhttp/Modules/Webservices/bin/Release/net11.0/GenHTTP.Modules.Webservices.dll
#13 7.539   GenHTTP.Engine.Ioxide -> /genhttp/Engine/Ioxide/bin/Release/net11.0/GenHTTP.Engine.Ioxide.dll
#13 7.899   genhttp -> /src/app/bin/Release/net11.0/genhttp.dll
#13 7.979   genhttp -> /app/
#13 DONE 8.2s

#14 [stage-1 2/3] WORKDIR /app
#14 CACHED

#15 [stage-1 3/3] COPY --from=build /app .
#15 CACHED

#16 exporting to image
#16 exporting layers done
#16 exporting manifest sha256:52b647029798fc2ae0a1b59cd81368af2fb6f5c803eee64593a06c41d99c9b8f done
#16 exporting config sha256:7ddae00b438e4f8eb2a61ebc51e19ff288b2278636891c9623016cfd6f0a0d1b done
#16 exporting attestation manifest sha256:4e492988b23f2941771d6faff8b2775dcf442d1c95876da3d91bf7d7df7b0394 0.1s done
#16 exporting manifest list sha256:b44f974c7cafa2586474148ee4cdee653545237cff33a599bd9c7e907f54e834 0.0s done
#16 naming to docker.io/library/httparena-genhttp-ioxide:latest
#16 naming to docker.io/library/httparena-genhttp-ioxide:latest done
#16 unpacking to docker.io/library/httparena-genhttp-ioxide:latest 0.0s done
#16 DONE 0.2s
[info] zrk: docker mode (zrk:local)
[info] tuning host for benchmark runs
[info] CPU governor → performance
[info] setting kernel socket limits
[info] setting UDP buffer sizes for QUIC
[info] setting loopback MTU to 1500 (realistic Ethernet)
[info] restarting docker daemon
[info] dropping kernel caches
[info] raising host limits for the held-connection profiles
[info] starting postgres sidecar
[info] postgres ready (seeded)

==============================================
=== genhttp-ioxide / connections-5m / 5000000c (tool=gcannon) ===
==============================================
[info] waiting for server...
[info] server ready

[run 1/3]
[conn-hold] memory: the client ~19.1 GiB, the kernel ~38.1 GiB, and the server may grow to 128 GiB; 245 GiB available
gcannon: warmup done, measuring for 20s
gcannon: measurement done
[conn-hold] 5000000 of 5000000 connections open at the end of the window
{
 "connections": 5000000,
 "connections_open": 5000000,
 "target_rate": 1250000,
 "rps": 1282612.51,
 "rate_ratio": 1.0261,
 "duration_s": 20.016,
 "bytes_per_sec": 161612616.75,
 "max_schedule_lag_us": 31824,
 "latency_us": {
  "mean": 88523,
  "p50": 10100,
  "p90": 389700,
  "p99": 728400,
  "p99_9": 938300
 },
 "status_codes": {
  "2xx": 25672772,
  "3xx": 0,
  "4xx": 0,
  "5xx": 0
 },
 "errors": {
  "connect": 0,
  "read": 0,
  "timeout": 0
 },
 "reconnects": 0,
 "server_cpu_usec": 1121253347,
 "gcannon": {
  "version": "0.8.1",
  "target": "127.0.0.1:8080/baseline11?a=1&b=2",
  "mode": "http",
  "connections": 5000000,
  "threads": 64,
  "pipeline_depth": 1,
  "duration_sec": 20,
  "elapsed_sec": 20.016,
  "requests": 25003543,
  "responses": 25672772,
  "rps": 1282639.82,
  "bandwidth_bps": 161612616.75,
  "bytes_read": 3234769272,
  "latency_us": {
   "avg": 88523,
   "p50": 10100,
   "p90": 389700,
   "p99": 728400,
   "p999": 938300,
   "samples": 25672772
  },
  "status": {
   "2xx": 25672772,
   "3xx": 0,
   "4xx": 0,
   "5xx": 0,
   "other": 0
  },
  "errors": {
   "connect": 0,
   "read": 0,
   "timeout": 0
  },
  "reconnects": 0,
  "rate": {
   "target": 1250000,
   "achieved": 1282639.82,
   "ratio": 1.0261,
   "paced_sends": 25003543,
   "late": 69337,
   "max_lag_us": 31824
  },
  "connections_active": 5000000
 }
}
[info] CPU 4204.9% | Mem 79.1GiB

[run 2/3]
[conn-hold] not enough memory to hold 5000000 connections: the client ~19.1 GiB, the kernel ~38.1 GiB, and the server may grow to 128 GiB, against 174 GiB available
{
 "hold_error": "not enough memory to hold 5000000 connections: the client ~19.1 GiB, the kernel ~38.1 GiB, and the server may grow to 128 GiB, against 174 GiB available",
 "connections": 5000000,
 "target_rate": 1250000,
 "rps": 0,
 "rate_ratio": 0,
 "duration_s": 0,
 "latency_us": {},
 "status_codes": {},
 "errors": {},
 "connections_open": null,
 "server_cpu_usec": null
}
[info] CPU 0% | Mem 0MiB

[run 3/3]
[conn-hold] not enough memory to hold 5000000 connections: the client ~19.1 GiB, the kernel ~38.1 GiB, and the server may grow to 128 GiB, against 185 GiB available
{
 "hold_error": "not enough memory to hold 5000000 connections: the client ~19.1 GiB, the kernel ~38.1 GiB, and the server may grow to 128 GiB, against 185 GiB available",
 "connections": 5000000,
 "target_rate": 1250000,
 "rps": 0,
 "rate_ratio": 0,
 "duration_s": 0,
 "latency_us": {},
 "status_codes": {},
 "errors": {},
 "connections_open": null,
 "server_cpu_usec": null
}
[info] CPU 0% | Mem 0MiB
[info] run 1 wins on score

=== Best: 1282613 req/s (CPU: 4204.9%, Mem: 79.1GiB) ===
[info] exact CPU: 1121.25 core-seconds | 43.675 us/req
=== Fixed-rate: cpu/req 43.67us | p99 728400.0us | p99.9 938300.0us | rate_ratio 1.0261 ===
=== Connections: 5000000 open of 5000000 | connect errors in the window 0 ===
[info] saved results/connections-5m/5000000/genhttp-ioxide.json
httparena-bench-genhttp-ioxide
httparena-bench-genhttp-ioxide
[info] rebuilding site/data/*.json
[updated] /home/diogo/actions-runner/_work/HttpArena/HttpArena/site/data/frameworks.json
[updated] /home/diogo/actions-runner/_work/HttpArena/HttpArena/site/data/results/genhttp-ioxide.json - 1 new, 34 total
[updated] /home/diogo/actions-runner/_work/HttpArena/HttpArena/site/data/current.json
[info] done
httparena-postgres
[info] restoring loopback MTU to 65536

genhttp-ioxide's first connections-5m run held all five million (79.1 GiB),
and the second was refused: MemAvailable had gone from 245 to 174 GiB. The
server container runs all three of a profile's runs, and a server that has
just held five million connections keeps what it grew to after they close (a
GC heap, allocator arenas). The check then counted that memory twice: it is
missing from MemAvailable and inside the 128 GiB cap the check adds in full.

- benchmark.sh stops the server after each held-connection run, before the
  cool-down, and starts a fresh one for the next, so every run starts clean
  and its Mem column measures that run alone.
- conn_hold.py and zrk_fanout.py count only what the server may still grow
  by: its cap less the cgroup's memory.current now.

Tested through benchmark.sh in a scratch copy with the profile shrunk to
16,000 connections on 8 CPUs, 2 runs against actix: run 2 started on a
fresh container ("at 0.0 GiB now"), both held 16,000 at rate 1.0000, and each
run's CPU and memory came from its own container.

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Proposal: add high-concurrency connection tests (e.g., 100k / 500k / 1M connections)

1 participant