Repository navigation
roadrunner: update to v0.16.2 - #1757
Conversation
|
/benchmark-multiple -f roadrunner,roadrunner-gateway |
|
👋 Benchmark request received. A collaborator will review and approve the run. |
Benchmark ResultsFrameworks: 2 | Test: ✅
|
| Test | Conn | RPS | Rate | CPU/req | p99 | p99.9 | CPU | Mem | Δ RPS | Δ Mem |
|---|---|---|---|---|---|---|---|---|---|---|
| baseline | 4096 | 1,192,703 | — | — | — | — | 6702.2% | 400MiB | +0.3% | +11.1% |
| pipelined | 4096 | 6,297,139 | — | — | — | — | 6279.6% | 310MiB | +8.2% | +28.1% |
| limited-conn | 4096 | 524,946 | — | — | — | — | 3644.5% | 267MiB | +1.2% | +15.1% |
| json-comp | 4096 | 147,566 | — | — | — | — | 6298.1% | 934MiB | +32.8% | +45.7% |
| json-comp | 16384 | 148,188 | — | — | — | — | 6183.4% | 958MiB | +41.5% | +38.2% |
| json-tls | 4096 | 224,141 | — | — | — | — | 6201.6% | 1.9GiB | +35.3% | +125.4% |
| 8gbit | 512 | 48,373 | 0.9675 |
288.76us | 998.0us | 7258.0us | 1337.7% | 486MiB | -0.2% | +2.3% |
| static-tls | 1024 | 86,659 | — | — | — | — | 6032.5% | 720MiB | +3.9% | +1.3% |
| async-db | 1024 | 160,227 | — | — | — | — | 6414.7% | 1.1GiB | +30.6% | +55.6% |
| fortunes | 1024 | 70,329 | — | — | — | — | 6453.0% | 1.1GiB | -0.3% | +23.1% |
| baseline-h2 | 256 | 607,689 | — | — | — | — | 6557.6% | 354MiB | +0.3% | -1.7% |
| baseline-h2 | 1024 | 609,592 | — | — | — | — | 6513.2% | 465MiB | +0.3% | -2.9% |
| static-h2 | 256 | 74,610 | — | — | — | — | 5857.4% | 437MiB | -6.0% | -17.2% |
| static-h2 | 1024 | 72,820 | — | — | — | — | 6197.0% | 1.0GiB | -3.5% | ~0% |
| baseline-h2c | 256 | 953,992 | — | — | — | — | 6600.9% | 251MiB | -0.9% | -4.6% |
| baseline-h2c | 1024 | 933,964 | — | — | — | — | 6558.0% | 387MiB | -1.6% | +4.3% |
| baseline-h2c | 4096 | 847,926 | — | — | — | — | 6322.3% | 605MiB | -2.4% | -4.1% |
| json-h2c | 1024 | 136,323 | — | — | — | — | 3303.9% | 1.0GiB | +23.7% | +14.4% |
| json-h2c | 4096 | 139,769 | — | — | — | — | 3986.4% | 2.2GiB | +35.6% | +69.2% |
| baseline-h3 | 64 | 186,051 | — | — | — | — | 3917.8% | 354MiB | +1.0% | +9.9% |
| static-h3 | 64 | 41,022 | — | — | — | — | 2897.1% | 1.4GiB | +9.3% | +7.7% |
| echo-ws | 512 | 1,949,099 | — | — | — | — | 6794.0% | 194MiB | -1.0% | -0.5% |
| echo-ws | 4096 | 1,914,161 | — | — | — | — | 6436.2% | 281MiB | -2.4% | +5.6% |
| echo-ws | 16384 | 1,745,998 | — | — | — | — | 6253.2% | 417MiB | -6.3% | +21.9% |
| echo-ws-pipeline | 512 | 17,670,908 | — | — | — | — | 6654.9% | 199MiB | -0.7% | +4.7% |
| echo-ws-pipeline | 4096 | 16,935,305 | — | — | — | — | 6397.1% | 300MiB | +1.0% | -1.3% |
| echo-ws-pipeline | 16384 | 16,024,870 | — | — | — | — | 6585.2% | 396MiB | -2.3% | +20.4% |
| echo-ws-limited | 512 | 540,383 | — | — | — | — | 3304.3% | 208MiB | +0.6% | -12.6% |
| echo-ws-limited | 4096 | 528,179 | — | — | — | — | 3343.0% | 212MiB | -1.8% | -21.5% |
| latency-1m | 1024 | 997,425 | 0.9974 | 62.42us | 21032.0us | 36304.0us | 6465.6% | 457MiB | ~0% | -5.2% |
| latency-10k | 1024 | 9,981 | 0.9981 | 119.32us | 142.0us | 781.0us | 115.6% | 197MiB | ~0% | +2.1% |
| latency-500k-8cpu | 1024 | 178,004 | 0.3560 |
44.82us | 12849152.0us | 13103104.0us | 774.2% | 146MiB | +1.6% | ~0% |
| async | 32000 | 1,141,605 | — | — | — | — | 6341.3% | 1.0GiB | -0.1% | +13.9% |
Composite score, main against this run, in the board's default view:
| Family | main | this run | Δ |
|---|---|---|---|
| HTTP/1.1 | 2566.7 (58 of 107) | 2666.5 (57 of 107) | +99.8 |
| HTTP/2 | 273.1 (34 of 40) | 282.0 (34 of 40) | +8.9 |
| HTTP/3 | 166.1 (17 of 19) | 178.6 (17 of 19) | +12.4 |
| Gateway | 1834.7 (5 of 9) | 1804.7 (5 of 9) | -30.0 |
| WebSocket | 1018.0 (23 of 38) | 999.2 (23 of 38) | -18.8 |
The results on main were measured in another session, so a few points up or down can be the machine and not the change.
✅ roadrunner-gateway
| Test | Conn | RPS | CPU | Mem | Δ RPS | Δ Mem |
|---|---|---|---|---|---|---|
| gateway-64 | 512 | 157,440 | 5988.4% | 4.5GiB | +2.1% | +55.2% |
| gateway-64 | 1024 | 151,470 | 5574.9% | 6.4GiB | +0.6% | +39.1% |
| gateway-h3 | 64 | 79,789 | 5248.0% | 904MiB | -1.8% | +15.5% |
| gateway-h3 | 256 | 58,242 | 4069.8% | 2.1GiB | +0.2% | +23.5% |
| production-stack | 256 | 222,057 | 5287.3% | 1.5GiB | -5.9% | ~0% |
| production-stack | 1024 | 193,520 | 5459.2% | 2.1GiB | -0.9% | ~0% |
|
/benchmark-multiple -f roadrunner,roadrunner-gateway --save |
|
👋 Benchmark request received. A collaborator will review and approve the run. |
Benchmark ResultsFrameworks: 2 | Test: ✅
|
| Test | Conn | RPS | Rate | CPU/req | p99 | p99.9 | CPU | Mem | Δ RPS | Δ Mem |
|---|---|---|---|---|---|---|---|---|---|---|
| baseline | 4096 | 1,138,679 | — | — | — | — | 6711.2% | 415MiB | -4.3% | +15.3% |
| pipelined | 4096 | 5,672,619 | — | — | — | — | 6261.9% | 312MiB | -2.6% | +28.9% |
| limited-conn | 4096 | 510,823 | — | — | — | — | 3772.7% | 269MiB | -1.5% | +15.9% |
| json-comp | 4096 | 145,867 | — | — | — | — | 6481.2% | 954MiB | +31.3% | +48.8% |
| json-comp | 16384 | 142,049 | — | — | — | — | 6169.6% | 949MiB | +35.6% | +36.9% |
| json-tls | 4096 | 226,716 | — | — | — | — | 6196.2% | 1.7GiB | +36.8% | +101.7% |
| 8gbit | 512 | 48,483 | 0.9697 |
289.87us | 613.0us | 7554.0us | 1321.9% | 474MiB | ~0% | -0.2% |
| static-tls | 1024 | 87,268 | — | — | — | — | 6082.7% | 709MiB | +4.6% | -0.3% |
| async-db | 1024 | 163,859 | — | — | — | — | 6374.1% | 1.0GiB | +33.5% | +41.4% |
| fortunes | 1024 | 69,879 | — | — | — | — | 6290.6% | 1.2GiB | -0.9% | +34.3% |
| baseline-h2 | 256 | 597,551 | — | — | — | — | 6570.2% | 347MiB | -1.4% | -3.6% |
| baseline-h2 | 1024 | 598,371 | — | — | — | — | 6600.9% | 471MiB | -1.6% | -1.7% |
| static-h2 | 256 | 80,212 | — | — | — | — | 6140.7% | 508MiB | +1.0% | -3.8% |
| static-h2 | 1024 | 72,732 | — | — | — | — | 6160.3% | 1.0GiB | -3.6% | ~0% |
| baseline-h2c | 256 | 936,244 | — | — | — | — | 6673.0% | 255MiB | -2.7% | -3.0% |
| baseline-h2c | 1024 | 917,156 | — | — | — | — | 6498.2% | 386MiB | -3.4% | +4.0% |
| baseline-h2c | 4096 | 835,437 | — | — | — | — | 6333.2% | 636MiB | -3.9% | +0.8% |
| json-h2c | 1024 | 137,515 | — | — | — | — | 3044.5% | 1.1GiB | +24.8% | +25.9% |
| json-h2c | 4096 | 139,716 | — | — | — | — | 4039.8% | 2.1GiB | +35.5% | +61.5% |
| baseline-h3 | 64 | 182,387 | — | — | — | — | 3759.5% | 334MiB | -1.0% | +3.7% |
| static-h3 | 64 | 42,585 | — | — | — | — | 3795.6% | 1.4GiB | +13.5% | +7.7% |
| echo-ws | 512 | 1,872,425 | — | — | — | — | 6707.4% | 194MiB | -4.9% | -0.5% |
| echo-ws | 4096 | 1,878,319 | — | — | — | — | 6633.2% | 290MiB | -4.2% | +9.0% |
| echo-ws | 16384 | 1,720,550 | — | — | — | — | 6147.6% | 426MiB | -7.7% | +24.6% |
| echo-ws-pipeline | 512 | 17,272,726 | — | — | — | — | 6594.8% | 195MiB | -2.9% | +2.6% |
| echo-ws-pipeline | 4096 | 17,085,494 | — | — | — | — | 6438.0% | 295MiB | +1.9% | -3.0% |
| echo-ws-pipeline | 16384 | 16,173,676 | — | — | — | — | 6569.6% | 384MiB | -1.4% | +16.7% |
| echo-ws-limited | 512 | 529,719 | — | — | — | — | 3308.3% | 198MiB | -1.4% | -16.8% |
| echo-ws-limited | 4096 | 533,540 | — | — | — | — | 3418.5% | 210MiB | -0.8% | -22.2% |
| latency-1m | 1024 | 998,206 | 0.9982 | 62.89us | 23320.0us | 41296.0us | 6557.8% | 419MiB | ~0% | -13.1% |
| latency-10k | 1024 | 9,980 | 0.9980 | 108.35us | 157.0us | 7698.0us | 99.3% | 197MiB | ~0% | +2.1% |
| latency-500k-8cpu | 1024 | 175,818 | 0.3516 |
45.39us | 13021184.0us | 13283328.0us | 772.8% | 145MiB | +0.3% | -0.7% |
| async | 32000 | 1,124,751 | — | — | — | — | 6332.6% | 1.0GiB | -1.6% | +13.9% |
Composite score, main against this run, in the board's default view:
| Family | main | this run | Δ |
|---|---|---|---|
| HTTP/1.1 | 2566.7 (58 of 107) | 2644.7 (57 of 107) | +78.0 |
| HTTP/2 | 273.1 (34 of 40) | 282.8 (34 of 40) | +9.6 |
| HTTP/3 | 166.1 (17 of 19) | 183.2 (17 of 19) | +17.1 |
| Gateway | 1834.7 (5 of 9) | 1807.7 (5 of 9) | -27.0 |
| WebSocket | 1018.0 (23 of 38) | 986.0 (23 of 38) | -32.0 |
The results on main were measured in another session, so a few points up or down can be the machine and not the change.
✅ roadrunner-gateway
| Test | Conn | RPS | CPU | Mem | Δ RPS | Δ Mem |
|---|---|---|---|---|---|---|
| gateway-64 | 512 | 155,804 | 6013.5% | 4.6GiB | +1.0% | +58.6% |
| gateway-64 | 1024 | 151,935 | 5640.8% | 6.8GiB | +0.9% | +47.8% |
| gateway-h3 | 64 | 79,716 | 5221.6% | 861MiB | -1.9% | +10.0% |
| gateway-h3 | 256 | 57,715 | 4414.2% | 2.2GiB | -0.7% | +29.4% |
| production-stack | 256 | 226,919 | 5595.3% | 1.5GiB | -3.9% | ~0% |
| production-stack | 1024 | 191,735 | 5519.9% | 2.2GiB | -1.8% | +4.8% |
|
/benchmark-multiple -f roadrunner,roadrunner-gateway |
|
👋 Benchmark request received. A collaborator will review and approve the run. |
Benchmark ResultsFrameworks: 2 | Test: ✅
|
| Test | Conn | RPS | Rate | CPU/req | p99 | p99.9 | CPU | Mem | Δ RPS | Δ Mem |
|---|---|---|---|---|---|---|---|---|---|---|
| baseline | 4096 | 1,188,348 | — | — | — | — | 6753.0% | 392MiB | ~0% | +8.9% |
| pipelined | 4096 | 6,533,524 | — | — | — | — | 6287.1% | 312MiB | +12.2% | +28.9% |
| limited-conn | 4096 | 522,337 | — | — | — | — | 3694.4% | 286MiB | +0.7% | +23.3% |
| json-comp | 4096 | 148,913 | — | — | — | — | 6282.1% | 880MiB | +34.0% | +37.3% |
| json-comp | 16384 | 152,497 | — | — | — | — | 6134.6% | 879MiB | +45.6% | +26.8% |
| json-tls | 4096 | 227,447 | — | — | — | — | 6205.4% | 1.8GiB | +37.3% | +113.6% |
| 8gbit | 512 | 48,487 | 0.9697 |
277.89us | 778.0us | 6198.0us | 1289.6% | 476MiB | ~0% | +0.2% |
| static-tls | 1024 | 88,454 | — | — | — | — | 5816.5% | 714MiB | +6.0% | +0.4% |
| async-db | 1024 | 166,908 | — | — | — | — | 6459.4% | 983MiB | +36.0% | +35.8% |
| fortunes | 1024 | 70,328 | — | — | — | — | 6431.2% | 1.1GiB | -0.3% | +23.1% |
| baseline-h2 | 256 | 610,301 | — | — | — | — | 6664.6% | 352MiB | +0.7% | -2.2% |
| baseline-h2 | 1024 | 607,057 | — | — | — | — | 6520.8% | 456MiB | -0.1% | -4.8% |
| static-h2 | 256 | 86,913 | — | — | — | — | 5726.0% | 506MiB | +9.5% | -4.2% |
| static-h2 | 1024 | 74,361 | — | — | — | — | 6135.5% | 980MiB | -1.4% | -4.3% |
| baseline-h2c | 256 | 945,129 | — | — | — | — | 6607.8% | 258MiB | -1.8% | -1.9% |
| baseline-h2c | 1024 | 931,591 | — | — | — | — | 6554.9% | 329MiB | -1.8% | -11.3% |
| baseline-h2c | 4096 | 854,567 | — | — | — | — | 6482.1% | 611MiB | -1.7% | -3.2% |
| json-h2c | 1024 | 137,933 | — | — | — | — | 3219.4% | 1001MiB | +25.2% | +11.8% |
| json-h2c | 4096 | 140,715 | — | — | — | — | 3969.9% | 2.1GiB | +36.5% | +61.5% |
| baseline-h3 | 64 | 186,874 | — | — | — | — | 3810.2% | 350MiB | +1.5% | +8.7% |
| static-h3 | 64 | 40,499 | — | — | — | — | 3041.7% | 1.2GiB | +7.9% | -7.7% |
| echo-ws | 512 | 1,966,849 | — | — | — | — | 6788.8% | 192MiB | -0.1% | -1.5% |
| echo-ws | 4096 | 1,905,289 | — | — | — | — | 6459.2% | 276MiB | -2.8% | +3.8% |
| echo-ws | 16384 | 1,759,692 | — | — | — | — | 6341.0% | 422MiB | -5.6% | +23.4% |
| echo-ws-pipeline | 512 | 17,752,876 | — | — | — | — | 6595.2% | 234MiB | -0.2% | +23.2% |
| echo-ws-pipeline | 4096 | 17,244,377 | — | — | — | — | 6641.3% | 284MiB | +2.8% | -6.6% |
| echo-ws-pipeline | 16384 | 16,146,211 | — | — | — | — | 6153.7% | 429MiB | -1.6% | +30.4% |
| echo-ws-limited | 512 | 542,279 | — | — | — | — | 3292.4% | 194MiB | +1.0% | -18.5% |
| echo-ws-limited | 4096 | 538,079 | — | — | — | — | 3192.7% | 198MiB | ~0% | -26.7% |
| latency-1m | 1024 | 997,623 | 0.9976 | 62.51us | 20280.0us | 37264.0us | 6485.8% | 378MiB | ~0% | -21.6% |
| latency-10k | 1024 | 9,983 | 0.9983 | 127.84us | 140.0us | 466.0us | 124.3% | 195MiB | ~0% | +1.0% |
| latency-500k-8cpu | 1024 | 176,240 | 0.3525 |
45.29us | 12914688.0us | 13152256.0us | 773.0% | 140MiB | +0.6% | -4.1% |
| async | 32000 | 1,128,800 | — | — | — | — | 6362.6% | 1019MiB | -1.3% | +13.3% |
Composite score, main against this run, in the board's default view:
| Family | main | this run | Δ |
|---|---|---|---|
| HTTP/1.1 | 2566.7 (58 of 107) | 2685.0 (57 of 107) | +118.3 |
| HTTP/2 | 273.1 (34 of 40) | 289.8 (34 of 40) | +16.7 |
| HTTP/3 | 166.1 (17 of 19) | 176.9 (17 of 19) | +10.8 |
| Gateway | 1834.7 (5 of 9) | 1850.7 (5 of 9) | +16.0 |
| WebSocket | 1018.0 (23 of 38) | 1006.7 (23 of 38) | -11.3 |
The results on main were measured in another session, so a few points up or down can be the machine and not the change.
✅ roadrunner-gateway
| Test | Conn | RPS | CPU | Mem | Δ RPS | Δ Mem |
|---|---|---|---|---|---|---|
| gateway-64 | 512 | 155,948 | 5762.4% | 4.5GiB | +1.1% | +55.2% |
| gateway-64 | 1024 | 150,842 | 5919.2% | 6.4GiB | +0.2% | +39.1% |
| gateway-h3 | 64 | 80,681 | 5158.7% | 873MiB | -0.7% | +11.5% |
| gateway-h3 | 256 | 64,988 | 4645.3% | 1.4GiB | +11.8% | -17.6% |
| production-stack | 256 | 237,009 | 5687.8% | 1.5GiB | +0.4% | ~0% |
| production-stack | 1024 | 192,406 | 5533.2% | 2.1GiB | -1.4% | ~0% |
|
/benchmark-multiple -f roadrunner,roadrunner-gateway --save |
|
👋 Benchmark request received. A collaborator will review and approve the run. |
Benchmark ResultsFrameworks: 2 | Test: ✅
|
| Test | Conn | RPS | Rate | CPU/req | p99 | p99.9 | CPU | Mem | Δ RPS | Δ Mem |
|---|---|---|---|---|---|---|---|---|---|---|
| baseline | 4096 | 1,169,909 | — | — | — | — | 6699.8% | 408MiB | -1.6% | +13.3% |
| pipelined | 4096 | 5,543,115 | — | — | — | — | 6269.3% | 254MiB | -4.8% | +5.0% |
| limited-conn | 4096 | 518,569 | — | — | — | — | 3996.9% | 301MiB | ~0% | +29.7% |
| json-comp | 4096 | 152,700 | — | — | — | — | 6438.1% | 903MiB | +37.4% | +40.9% |
| json-comp | 16384 | 141,531 | — | — | — | — | 6429.9% | 951MiB | +35.1% | +37.2% |
| json-tls | 4096 | 228,614 | — | — | — | — | 6186.8% | 1.9GiB | +38.0% | +125.4% |
| 8gbit | 512 | 48,415 | 0.9683 |
282.26us | 659.0us | 5002.0us | 1261.2% | 466MiB | -0.1% | -1.9% |
| static-tls | 1024 | 88,349 | — | — | — | — | 5683.1% | 708MiB | +5.9% | -0.4% |
| async-db | 1024 | 165,371 | — | — | — | — | 6346.9% | 1.1GiB | +34.7% | +55.6% |
| fortunes | 1024 | 69,824 | — | — | — | — | 6429.8% | 1014MiB | -1.0% | +10.8% |
| baseline-h2 | 256 | 598,542 | — | — | — | — | 6570.7% | 379MiB | -1.2% | +5.3% |
| baseline-h2 | 1024 | 601,869 | — | — | — | — | 6636.9% | 463MiB | -1.0% | -3.3% |
| static-h2 | 256 | 87,862 | — | — | — | — | 5620.2% | 509MiB | +10.7% | -3.6% |
| static-h2 | 1024 | 74,046 | — | — | — | — | 6122.9% | 1.0GiB | -1.9% | ~0% |
| baseline-h2c | 256 | 933,337 | — | — | — | — | 6676.1% | 258MiB | -3.1% | -1.9% |
| baseline-h2c | 1024 | 912,638 | — | — | — | — | 6538.8% | 346MiB | -3.8% | -6.7% |
| baseline-h2c | 4096 | 840,930 | — | — | — | — | 6296.8% | 590MiB | -3.2% | -6.5% |
| json-h2c | 1024 | 138,330 | — | — | — | — | 3236.5% | 985MiB | +25.5% | +10.1% |
| json-h2c | 4096 | 140,668 | — | — | — | — | 3961.0% | 2.2GiB | +36.4% | +69.2% |
| baseline-h3 | 64 | 184,409 | — | — | — | — | 3857.1% | 335MiB | +0.1% | +4.0% |
| static-h3 | 64 | 54,088 | — | — | — | — | 4512.7% | 1.3GiB | +44.1% | ~0% |
| echo-ws | 512 | 1,890,209 | — | — | — | — | 6757.8% | 195MiB | -4.0% | ~0% |
| echo-ws | 4096 | 1,919,907 | — | — | — | — | 6749.9% | 250MiB | -2.1% | -6.0% |
| echo-ws | 16384 | 1,715,824 | — | — | — | — | 6640.2% | 410MiB | -7.9% | +19.9% |
| echo-ws-pipeline | 512 | 17,308,262 | — | — | — | — | 6745.9% | 188MiB | -2.7% | -1.1% |
| echo-ws-pipeline | 4096 | 16,730,243 | — | — | — | — | 6690.1% | 295MiB | -0.3% | -3.0% |
| echo-ws-pipeline | 16384 | 15,660,924 | — | — | — | — | 6325.1% | 430MiB | -4.6% | +30.7% |
| echo-ws-limited | 512 | 530,607 | — | — | — | — | 3256.8% | 195MiB | -1.2% | -18.1% |
| echo-ws-limited | 4096 | 531,405 | — | — | — | — | 3345.3% | 202MiB | -1.2% | -25.2% |
| latency-1m | 1024 | 997,158 | 0.9972 | 62.78us | 25976.0us | 55408.0us | 6503.9% | 353MiB | ~0% | -26.8% |
| latency-10k | 1024 | 9,981 | 0.9981 | 128.58us | 135.0us | 1384.0us | 124.7% | 207MiB | ~0% | +7.3% |
| latency-500k-8cpu | 1024 | 173,131 | 0.3463 |
46.08us | 13012992.0us | 13242368.0us | 771.3% | 137MiB | -1.2% | -6.2% |
| async | 32000 | 1,115,540 | — | — | — | — | 6354.7% | 1.0GiB | -2.4% | +13.9% |
Composite score, main against this run, in the board's default view:
| Family | main | this run | Δ |
|---|---|---|---|
| HTTP/1.1 | 2566.7 (58 of 107) | 2660.4 (57 of 107) | +93.7 |
| HTTP/2 | 273.1 (34 of 40) | 288.1 (34 of 40) | +15.0 |
| HTTP/3 | 166.1 (17 of 19) | 223.4 (17 of 19) | +57.2 |
| Gateway | 1834.7 (5 of 9) | 1807.7 (5 of 9) | -27.0 |
| WebSocket | 1018.0 (23 of 38) | 985.2 (23 of 38) | -32.7 |
The results on main were measured in another session, so a few points up or down can be the machine and not the change.
❌ roadrunner-gateway
| Test | Conn | RPS | CPU | Mem | Δ RPS | Δ Mem |
|---|---|---|---|---|---|---|
| gateway-64 | 512 | 154,940 | 5953.9% | 4.7GiB | +0.5% | +62.1% |
| gateway-64 | 1024 | 151,301 | 5651.9% | 6.6GiB | +0.5% | +43.5% |
| gateway-h3 | 64 | 80,503 | 5131.4% | 913MiB | -0.9% | +16.6% |
| gateway-h3 | 256 | 57,106 | 4048.9% | 2.1GiB | -1.7% | +23.5% |
Log tail
#41 DONE 0.2s
#43 [authsvc] resolving provenance for metadata file
#43 DONE 0.0s
Image httparena-roadrunner-gateway-production-stack-authsvc Built
Image httparena-roadrunner-gateway-production-stack-server Built
Image httparena-roadrunner-gateway-production-stack-edge Built
Container httparena-roadrunner-gateway-production-stack-cache-1 Creating
Container httparena-roadrunner-gateway-production-stack-authsvc-1 Creating
Container httparena-roadrunner-gateway-production-stack-authsvc-1 Created
Container httparena-roadrunner-gateway-production-stack-cache-1 Created
Container httparena-roadrunner-gateway-production-stack-server-1 Creating
Container httparena-roadrunner-gateway-production-stack-server-1 Created
Container httparena-roadrunner-gateway-production-stack-edge-1 Creating
Container httparena-roadrunner-gateway-production-stack-edge-1 Created
Container httparena-roadrunner-gateway-production-stack-cache-1 Starting
Container httparena-roadrunner-gateway-production-stack-authsvc-1 Starting
Container httparena-roadrunner-gateway-production-stack-authsvc-1 Started
Container httparena-roadrunner-gateway-production-stack-cache-1 Started
Container httparena-roadrunner-gateway-production-stack-cache-1 Waiting
Container httparena-roadrunner-gateway-production-stack-cache-1 Healthy
Container httparena-roadrunner-gateway-production-stack-server-1 Starting
Container httparena-roadrunner-gateway-production-stack-server-1 Started
Container httparena-roadrunner-gateway-production-stack-authsvc-1 Waiting
Container httparena-roadrunner-gateway-production-stack-server-1 Waiting
Container httparena-roadrunner-gateway-production-stack-authsvc-1 Error dependency authsvc failed to start
dependency failed to start: container httparena-roadrunner-gateway-production-stack-authsvc-1 exited (0)
─── httparena-roadrunner-gateway-production-stack-edge-1 — status=created exit=0 oom=false error=
─── httparena-roadrunner-gateway-production-stack-edge-1 — the container produced no output at all
─── httparena-roadrunner-gateway-production-stack-server-1 — status=running exit=0 oom=false error=
─── httparena-roadrunner-gateway-production-stack-server-1 — last 120 log lines ───
| Exec: /app/erts-17.0.2/bin/erlexec -noinput +Bd -boot /app/releases/0.1.0/start -mode embedded -boot_var SYSTEM_LIB_DIR /app/lib -config /app/releases/0.1.0/sys.config -args_file /app/releases/0.1.0/vm.args -- foreground
| Root: /app
| /app
| =ESOCK WARNING MSG==== 13-Nov-2026::08:09:06.279486 ===
| [UNIX-ESSIO] Failed open sctp dynamic library: libsctp.so.1
─── httparena-roadrunner-gateway-production-stack-server-1 — end of logs ───
─── httparena-roadrunner-gateway-production-stack-cache-1 — status=running exit=0 oom=false error=
─── httparena-roadrunner-gateway-production-stack-cache-1 — last 120 log lines ───
| 7:C 13 Nov 2026 08:09:04.420 # WARNING Memory overcommit must be enabled! Without it, a background save or replication may fail under low memory condition. Being disabled, it can also cause failures without low memory condition, see https://github.com/jemalloc/jemalloc/issues/1328. To fix this issue add 'vm.overcommit_memory = 1' to /etc/sysctl.conf and then reboot or run the command 'sysctl vm.overcommit_memory=1' for this to take effect.
| 7:C 13 Nov 2026 08:09:04.420 * oO0OoO0OoO0Oo Redis is starting oO0OoO0OoO0Oo
| 7:C 13 Nov 2026 08:09:04.420 * Redis version=7.4.8, bits=64, commit=00000000, modified=0, pid=7, just started
| 7:C 13 Nov 2026 08:09:04.420 * Configuration loaded
| 7:M 13 Nov 2026 08:09:04.420 * monotonic clock: POSIX clock_gettime
| 7:M 13 Nov 2026 08:09:04.424 * Running mode=standalone, port=6379.
| 7:M 13 Nov 2026 08:09:04.424 * Server initialized
| 7:M 13 Nov 2026 08:09:04.425 * Ready to accept connections tcp
| [redis] seeded from /seed.txt
─── httparena-roadrunner-gateway-production-stack-cache-1 — end of logs ───
─── httparena-roadrunner-gateway-production-stack-authsvc-1 — status=exited exit=0 oom=false error=
─── httparena-roadrunner-gateway-production-stack-authsvc-1 — the container produced no output at all
[FAIL] gateway compose up failed
httparena-postgres
[info] restoring loopback MTU to 65536
|
/benchmark-multiple -f roadrunner,roadrunner-gateway --save |
|
👋 Benchmark request received. A collaborator will review and approve the run. |
Benchmark ResultsFrameworks: 2 | Test: ✅
|
| Test | Conn | RPS | Rate | CPU/req | p99 | p99.9 | CPU | Mem | Δ RPS | Δ Mem |
|---|---|---|---|---|---|---|---|---|---|---|
| baseline | 4096 | 1,190,380 | — | — | — | — | 6409.9% | 382MiB | ~0% | +6.1% |
| pipelined | 4096 | 6,106,728 | — | — | — | — | 6289.3% | 329MiB | +4.9% | +36.0% |
| limited-conn | 4096 | 530,359 | — | — | — | — | 3621.4% | 281MiB | +2.2% | +21.1% |
| json-comp | 4096 | 150,319 | — | — | — | — | 6340.3% | 891MiB | +35.3% | +39.0% |
| json-comp | 16384 | 148,211 | — | — | — | — | 6492.6% | 923MiB | +41.5% | +33.2% |
| json-tls | 4096 | 223,898 | — | — | — | — | 6158.5% | 1.8GiB | +35.1% | +113.6% |
| 8gbit | 512 | 48,426 | 0.9685 |
276.68us | 847.0us | 5666.0us | 1240.5% | 462MiB | -0.1% | -2.7% |
| static-tls | 1024 | 89,655 | — | — | — | — | 5663.6% | 721MiB | +7.4% | +1.4% |
| async-db | 1024 | 165,313 | — | — | — | — | 6379.1% | 1.0GiB | +34.7% | +41.4% |
| fortunes | 1024 | 69,357 | — | — | — | — | 6424.1% | 1.4GiB | -1.7% | +56.7% |
| baseline-h2 | 256 | 608,773 | — | — | — | — | 6529.8% | 423MiB | +0.5% | +17.5% |
| baseline-h2 | 1024 | 601,261 | — | — | — | — | 6513.9% | 478MiB | -1.1% | -0.2% |
| static-h2 | 256 | 73,012 | — | — | — | — | 6278.1% | 478MiB | -8.0% | -9.5% |
| static-h2 | 1024 | 74,934 | — | — | — | — | 5910.4% | 998MiB | -0.7% | -2.5% |
| baseline-h2c | 256 | 942,518 | — | — | — | — | 6654.7% | 257MiB | -2.1% | -2.3% |
| baseline-h2c | 1024 | 925,316 | — | — | — | — | 6506.3% | 346MiB | -2.5% | -6.7% |
| baseline-h2c | 4096 | 853,129 | — | — | — | — | 6271.0% | 578MiB | -1.8% | -8.4% |
| json-h2c | 1024 | 135,561 | — | — | — | — | 3230.1% | 1.1GiB | +23.0% | +25.9% |
| json-h2c | 4096 | 140,766 | — | — | — | — | 3867.1% | 2.1GiB | +36.5% | +61.5% |
| baseline-h3 | 64 | 180,097 | — | — | — | — | 3903.1% | 352MiB | -2.2% | +9.3% |
| static-h3 | 64 | 43,952 | — | — | — | — | 3879.1% | 1.4GiB | +17.1% | +7.7% |
| echo-ws | 512 | 1,944,473 | — | — | — | — | 6621.1% | 189MiB | -1.3% | -3.1% |
| echo-ws | 4096 | 1,889,351 | — | — | — | — | 6498.0% | 246MiB | -3.6% | -7.5% |
| echo-ws | 16384 | 1,755,757 | — | — | — | — | 6256.9% | 404MiB | -5.8% | +18.1% |
| echo-ws-pipeline | 512 | 17,175,317 | — | — | — | — | 6563.7% | 237MiB | -3.5% | +24.7% |
| echo-ws-pipeline | 4096 | 17,633,724 | — | — | — | — | 6726.6% | 306MiB | +5.1% | +0.7% |
| echo-ws-pipeline | 16384 | 16,224,509 | — | — | — | — | 6252.8% | 422MiB | -1.1% | +28.3% |
| echo-ws-limited | 512 | 539,392 | — | — | — | — | 3314.5% | 191MiB | +0.4% | -19.7% |
| echo-ws-limited | 4096 | 533,500 | — | — | — | — | 3303.3% | 195MiB | -0.8% | -27.8% |
| latency-1m | 1024 | 997,716 | 0.9977 | 62.46us | 19496.0us | 35312.0us | 6479.1% | 376MiB | ~0% | -22.0% |
| latency-10k | 1024 | 9,981 | 0.9981 | 132.93us | 140.0us | 16968.0us | 128.1% | 187MiB | ~0% | -3.1% |
| latency-500k-8cpu | 1024 | 171,678 | 0.3434 |
46.49us | 13111296.0us | 13398016.0us | 772.0% | 135MiB | -2.0% | -7.5% |
| async | 32000 | 1,133,425 | — | — | — | — | 6393.6% | 1002MiB | -0.9% | +11.5% |
Composite score, main against this run, in the board's default view:
| Family | main | this run | Δ |
|---|---|---|---|
| HTTP/1.1 | 2566.7 (58 of 107) | 2658.4 (57 of 107) | +91.7 |
| HTTP/2 | 273.1 (34 of 40) | 281.5 (34 of 40) | +8.4 |
| HTTP/3 | 166.1 (17 of 19) | 187.5 (17 of 19) | +21.4 |
| Gateway | 1834.7 (5 of 9) | 1807.7 (5 of 9) | -27.0 |
| WebSocket | 1018.0 (23 of 38) | 1000.9 (23 of 38) | -17.1 |
The results on main were measured in another session, so a few points up or down can be the machine and not the change.
❌ roadrunner-gateway
| Test | Conn | RPS | CPU | Mem | Δ RPS | Δ Mem |
|---|---|---|---|---|---|---|
| gateway-64 | 512 | 156,035 | 5985.3% | 4.5GiB | +1.2% | +55.2% |
| gateway-64 | 1024 | 151,330 | 5802.2% | 6.4GiB | +0.5% | +39.1% |
| gateway-h3 | 64 | 79,929 | 5243.2% | 869MiB | -1.6% | +11.0% |
| gateway-h3 | 256 | 58,808 | 4093.8% | 2.1GiB | +1.2% | +23.5% |
Log tail
#42 DONE 0.1s
#43 [authsvc] resolving provenance for metadata file
#43 DONE 0.0s
Image httparena-roadrunner-gateway-production-stack-server Built
Image httparena-roadrunner-gateway-production-stack-authsvc Built
Image httparena-roadrunner-gateway-production-stack-edge Built
Container httparena-roadrunner-gateway-production-stack-authsvc-1 Creating
Container httparena-roadrunner-gateway-production-stack-cache-1 Creating
Container httparena-roadrunner-gateway-production-stack-authsvc-1 Created
Container httparena-roadrunner-gateway-production-stack-cache-1 Created
Container httparena-roadrunner-gateway-production-stack-server-1 Creating
Container httparena-roadrunner-gateway-production-stack-server-1 Created
Container httparena-roadrunner-gateway-production-stack-edge-1 Creating
Container httparena-roadrunner-gateway-production-stack-edge-1 Created
Container httparena-roadrunner-gateway-production-stack-authsvc-1 Starting
Container httparena-roadrunner-gateway-production-stack-cache-1 Starting
Container httparena-roadrunner-gateway-production-stack-authsvc-1 Started
Container httparena-roadrunner-gateway-production-stack-cache-1 Started
Container httparena-roadrunner-gateway-production-stack-cache-1 Waiting
Container httparena-roadrunner-gateway-production-stack-cache-1 Healthy
Container httparena-roadrunner-gateway-production-stack-server-1 Starting
Container httparena-roadrunner-gateway-production-stack-server-1 Started
Container httparena-roadrunner-gateway-production-stack-authsvc-1 Waiting
Container httparena-roadrunner-gateway-production-stack-server-1 Waiting
Container httparena-roadrunner-gateway-production-stack-authsvc-1 Error dependency authsvc failed to start
dependency failed to start: container httparena-roadrunner-gateway-production-stack-authsvc-1 exited (0)
─── httparena-roadrunner-gateway-production-stack-edge-1 — status=created exit=0 oom=false error=
─── httparena-roadrunner-gateway-production-stack-edge-1 — the container produced no output at all
─── httparena-roadrunner-gateway-production-stack-server-1 — status=running exit=0 oom=false error=
─── httparena-roadrunner-gateway-production-stack-server-1 — last 120 log lines ───
| Exec: /app/erts-17.0.2/bin/erlexec -noinput +Bd -boot /app/releases/0.1.0/start -mode embedded -boot_var SYSTEM_LIB_DIR /app/lib -config /app/releases/0.1.0/sys.config -args_file /app/releases/0.1.0/vm.args -- foreground
| Root: /app
| /app
| =ESOCK WARNING MSG==== 13-Nov-2026::14:07:53.138385 ===
| [UNIX-ESSIO] Failed open sctp dynamic library: libsctp.so.1
─── httparena-roadrunner-gateway-production-stack-server-1 — end of logs ───
─── httparena-roadrunner-gateway-production-stack-cache-1 — status=running exit=0 oom=false error=
─── httparena-roadrunner-gateway-production-stack-cache-1 — last 120 log lines ───
| 7:C 13 Nov 2026 14:07:51.282 # WARNING Memory overcommit must be enabled! Without it, a background save or replication may fail under low memory condition. Being disabled, it can also cause failures without low memory condition, see https://github.com/jemalloc/jemalloc/issues/1328. To fix this issue add 'vm.overcommit_memory = 1' to /etc/sysctl.conf and then reboot or run the command 'sysctl vm.overcommit_memory=1' for this to take effect.
| 7:C 13 Nov 2026 14:07:51.282 * oO0OoO0OoO0Oo Redis is starting oO0OoO0OoO0Oo
| 7:C 13 Nov 2026 14:07:51.282 * Redis version=7.4.8, bits=64, commit=00000000, modified=0, pid=7, just started
| 7:C 13 Nov 2026 14:07:51.282 * Configuration loaded
| 7:M 13 Nov 2026 14:07:51.282 * monotonic clock: POSIX clock_gettime
| 7:M 13 Nov 2026 14:07:51.285 * Running mode=standalone, port=6379.
| 7:M 13 Nov 2026 14:07:51.286 * Server initialized
| 7:M 13 Nov 2026 14:07:51.286 * Ready to accept connections tcp
| [redis] seeded from /seed.txt
─── httparena-roadrunner-gateway-production-stack-cache-1 — end of logs ───
─── httparena-roadrunner-gateway-production-stack-authsvc-1 — status=exited exit=0 oom=false error=
─── httparena-roadrunner-gateway-production-stack-authsvc-1 — the container produced no output at all
[FAIL] gateway compose up failed
httparena-postgres
[info] restoring loopback MTU to 65536
|
@MDA2AV the |
…uthsvc on Rust 1.100, unbreak the aspnet gateway entries (#1767) * validate.sh: stop the standalone container before the compose profiles The standalone container runs on --network host whenever an entry subscribes to a compose profile, and it was left running while the gateway and production-stack stacks came up. An entry that also serves h2/h3 keeps 8443 (and 8080), so the stack's edge failed to bind, shared the port, or exited on the QUIC port, and the checks reached the standalone server instead of the stack. * authsvc: ship the real binary under cargo 1.100's build-dir layout production-stack has failed on the bench host since the morning of 2026-11-13 on every entry (roadrunner-gateway #1757, morojs #1761, tako-gateway #1762): the shared authsvc container exits 0 with no output, the edge never sees it healthy, and the run ends with no results. rust:1-alpine moved to Rust 1.100 on 2026-11-12. Its cargo keeps each package's artifacts under target/release/build/<package>/<hash>/ rather than target/release/deps/, so the cache-priming step's `rm -rf target/release/deps/authsvc*` stopped matching anything. The placeholder built from `fn main() {}` stayed up to date, the second `cargo build` finished in 0.04s without compiling src/, and the image shipped the placeholder: a 418 KB binary that exits 0 at once. `cargo clean --release -p authsvc` removes the placeholder's artifacts wherever cargo keeps them, and a grep for the server's startup line fails the build if the placeholder is ever shipped again. Built against the current rust:1-alpine (1.100.0): a 1.2 MB binary that logs "authsvc listening on ...", answers /_health with 200 and an /_auth without a token with 401. * aspnet-minimal_nginx, aspnet-minimal_caddy: build against aspnet-minimal's Services and Types Both Dockerfiles still COPY aspnet-minimal's AppData.cs and Models.cs, which #1002 (2026-07-18) split into Services/ and Types/, so neither image has built since. They were the only failures in #1615's validation sweep, and their published results have not been refreshed since July. Copy Handlers.cs with Services/ and Types/, register DatabaseService, DatasetService, ItemService and the AppJsonContext resolver the way aspnet-minimal's Program.cs does, and give the nginx entry's /api routes DatabaseService's Postgres pool in place of AppData.PgDataSource. The caddy entry gains the StackExchange.Redis reference DatabaseService needs; it connects to nothing unless REDIS_URL is set, which gateway-h3 never does. --------- Co-authored-by: Jacob Rothstein <hi@jbr.me>
|
Thanks for flagging it. The problem was the shared authsvc, not roadrunner. Nothing on this branch needs to change. A production-stack re-run with |
|
Thanks for the quick fix and the explanation! If everything looks good on your side, this one is ready to merge. |
Description
Updates roadrunner from v0.13.1 to v0.16.2, replacing #1751 (v0.15.0). From 0.14, roadrunner accepts new connections without waiting for each connection process to start, so many clients connecting at once to a busy server are accepted sooner; WebSocket compression is off by default; and WebSocket sessions read with a buffer that grows only for large frames, so the entry drops its fixed 4 KB
ws.recv_bufferand uses the default. From 0.15, plain-TCP connections wait with a 16 KB read buffer instead of 64 KB and grow it only when a large body arrives, and QUIC sockets no longer setreuseaddr, so an HTTP/3 port is never shared with another socket. From 0.16, a connection keeps the heap its last request needed as the size the next one starts at (HTTP/2 and HTTP/3 stream workers start at the size their connection learned, and WebSocket sessions learn it fromhandle_info/2), so a handler building a sizeable response, such as the JSON ones, garbage-collects once or twice per request instead of several times; it is on by default and the entry sets nothing for it. In 0.16.1, plain-TCP connections and WebSocket sessions start with a 4 KB read buffer instead of 16 KB and grow it for large heads, bodies and frames, so idle connections hold less memory; the entry sets nothing for this either. In 0.16.2, WebSocket sessions measure each outgoing payload once instead of twice. No option the entry sets changed meaning. Theroadrunner-gatewayentry builds fromframeworks/roadrunner, so it picks up the new version too.PR Commands — comment on this PR to trigger (requires collaborator approval):
/benchmark -f <framework>/benchmark -f <framework> -t <test>/benchmark -f <framework> --save/benchmark -f <framework> -t <test> --save/benchmark -f <framework> --compare <other>/benchmark-multiple -f <fw1>,<fw2>,...-tand--savetoo; saved results land in a single commit/benchmark-multiple --save-fneeded: benchmark and save every framework the PR touches/benchmark-test -t <test><test>and save the resultsFor
/benchmark, always specify-f <framework>; the flags combine in any order. Results come back as a comment with a per-profile table of RPS, p99, CPU and memory — one table per framework on multi runs. A new benchmark comment while a run is in flight queues behind it (one deep) instead of cancelling it. For multi-framework PRs (dependency bumps, same-language refactors) prefer/benchmark-multiple, which runs everything in a single job and commits all saved results together, so no run overwrites another.--compareworks on single-framework runs only.What the deltas are measured against. By default, this framework's own results published on
main- answering "did this change help?". When you are tuning a variant or a successor entry,--comparere-bases them on another entry instead:The reply states which baseline it used, and profiles the other framework does not run show
n/arather than a delta.Run benchmarks locally
You can validate and benchmark your framework locally with the lite script — no CPU pinning, fixed connection counts, all load generators run in Docker.
Requirements: Docker Engine on Linux. Load generators (gcannon, h2load, h2load-h3, wrk) are built as self-contained Docker images on first run.