You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Interesting but also not relevant (to uWS). All of these benchmark projects just collect various hacks that have zero standards compliance. Also, we are not limited by not-winning-benchmarks, we are limited by Node.js itself. Also, this project already established itself as "very fast" and there is no more market to be had from participating in these lists. These kinds of lists were very relevant 10 years ago. Today, not so much.
TLDR: the main problem with these benchmark websites is that they never equalize standards compliance (there is no common standards compliance check-list all participants must pass to be part of the test), and so they all end up just collecting hacks that would be instant security holes if used in production.
To shortly chime in: HTTP Arena has a equalizing standard (validation phase) to ensure we are talking to web servers and not as with TFB Framework Benchmarks where servers did not even pass a simple TCP fragmentation check. The author did also write https://www.http-probe.com/ - the findings did influence the validation in arena a lot.
Additionally, arena has a classification system to filter out those overfitting entries (also config changes are only supported in the tuned section). I would consider uWebSockets.js in the flagship tier, so it bypasses most of the problems you mentioned and framework benchmarks clearly had.
h1probe results are not an absolute metric, pass/warn/fail sometimes doesn't mean much as it always depend on the context, what matters is the behavior difference/divergence between entries. You can look at a failed test and say you are fine with it because you want the framework to behave like that, which is fine.
This is very good for tracking spec. compliance bugs. But it would be a lot
better if the C++ version was added, not JS and if it was pointed to master
git so I can push fixes and see them update the results in Http11Probe,
also kind of pointless to benchmark a server that is 50% restrained by
Node.js
Den lör 15 aug. 2026 19:01Diogo Martins ***@***.***> skrev:
yes, once merged full probe is triggered that takes around 1hour ish and everything is updated on the site
typically once a week, since not much contributions/changes happening. It can run on demand by one action trigger anytime someone requests.
There are 21 failures only due to uWS giving 505 instead of 400. By swapping "default error" code to 400, we drastically would increase the score to top 20.
I'd say score itself isn't very important, finding tests where behavior diverges from the majority expected is, actually there are 2-3 tests where passing is the anomaly. A higher score is a plus for sure but the highest scoring ones there are actually parsers that people overfitted to the test suite using AI agents, so unlikely to be a good solution either.
There are roughly 50 tests not yet in the repo, numbers will be increasing over time
I know from testing other servers that many of them lack fragmentation support (entirely). I was thinking, the simplest and most through test for fragmentation would probably be to just have 1 maybe 2 maybe 3 valid HTTP requests and then just run them through 1-byte-per-100ms sends and see if all of them return 200. That should cover all possible failures with very little code. Maybe you don't even need to wait at all, if TCP_NAGLE is off for the sender?
Ah wait I'm so stupid. What you do is - you take the valid HTTP request, you chop it up in 2 parts at all the byte offsets possible, then you send them all at the same time on its own connection, then wait 1 second, send the other chunk. That way the test finishes in 1 second and covers all breaking spots.
I know from testing other servers that many of them lack fragmentation support (entirely). I was thinking, the simplest and most through test for fragmentation would probably be to just have 1 maybe 2 maybe 3 valid HTTP requests and then just run them through 1-byte-per-100ms sends and see if all of them return 200. That should cover all possible failures with very little code. Maybe you don't even need to wait at all, if TCP_NAGLE is off for the sender?
If you want to test TCP frag properly, enable TLS and send a fragment with less than 5 byte, that is the minimum required for TLS 1.3, this will break the ankles of some frameworks. The 100ms probably help or the Kernel might rearrange the fragments into a single read. Chunked encoding is usually as good candidate for it too, hit the fragmentation in the chunk length.
But if we really want to check an edge case, try QUIC connection migration with certificate rotation that invalidates a certificate the moment the connection is migrating.
That doesn't actually test the server's parser for all the (multiple various kinds of) failures from not handling fragmentation. A server can handle fragmentation just fine at some place but fail completely at another place. So you need to test all byte offsets like the https://github.com/uNetworking/uWebSockets/blob/master/tests/fragment_test.ts does (I added it to my CI testing).
Activity
uNetworkingAB commentedon Aug 10, 2026
Interesting but also not relevant (to uWS). All of these benchmark projects just collect various hacks that have zero standards compliance. Also, we are not limited by not-winning-benchmarks, we are limited by Node.js itself. Also, this project already established itself as "very fast" and there is no more market to be had from participating in these lists. These kinds of lists were very relevant 10 years ago. Today, not so much.
uNetworkingAB commentedon Aug 10, 2026
TLDR: the main problem with these benchmark websites is that they never equalize standards compliance (there is no common standards compliance check-list all participants must pass to be part of the test), and so they all end up just collecting hacks that would be instant security holes if used in production.
Kaliumhexacyanoferrat commentedon Aug 14, 2026
To shortly chime in: HTTP Arena has a equalizing standard (validation phase) to ensure we are talking to web servers and not as with TFB Framework Benchmarks where servers did not even pass a simple TCP fragmentation check. The author did also write https://www.http-probe.com/ - the findings did influence the validation in arena a lot.
Additionally, arena has a classification system to filter out those overfitting entries (also config changes are only supported in the
tunedsection). I would consider uWebSockets.js in theflagshiptier, so it bypasses most of the problems you mentioned and framework benchmarks clearly had.uNetworkingAB commentedon Aug 14, 2026
I would rather add uWS to Http11Probe and fix standards behavior than chase some benchmark list,
Bun has 113 pass 20 warn 26 fail 54 unscored · 213 total, so we should get similar. Meaning there are things to fix first
[-]TechEmpower is died[/-][+]Add to Http11Probe[/+]MDA2AV commentedon Aug 15, 2026
here MDA2AV/Http11Probe#151
h1probe results are not an absolute metric, pass/warn/fail sometimes doesn't mean much as it always depend on the context, what matters is the behavior difference/divergence between entries. You can look at a failed test and say you are fine with it because you want the framework to behave like that, which is fine.
uNetworkingAB commentedon Aug 15, 2026
Add uWebSockets (C++)
uNetworkingAB commentedon Aug 15, 2026
Wow that's very fast, so if this merges, I will see results at https://www.http-probe.com ? And with what interval are they run? Daily?
MDA2AV commentedon Aug 15, 2026
yes, once merged full probe is triggered that takes around 1hour ish and everything is updated on the site
typically once a week, since not much contributions/changes happening. It can run on demand by one action trigger anytime someone requests.
Add uWebSockets (C++) (#152)
uNetworkingAB commentedon Aug 15, 2026
There are 21 failures only due to uWS giving 505 instead of 400. By swapping "default error" code to 400, we drastically would increase the score to top 20.
uNetworkingAB commentedon Aug 15, 2026
I get 137/159 now
MDA2AV commentedon Aug 15, 2026
I'd say score itself isn't very important, finding tests where behavior diverges from the majority expected is, actually there are 2-3 tests where passing is the anomaly. A higher score is a plus for sure but the highest scoring ones there are actually parsers that people overfitted to the test suite using AI agents, so unlikely to be a good solution either.
There are roughly 50 tests not yet in the repo, numbers will be increasing over time
uNetworkingAB commentedon Aug 16, 2026
145/159 👌 📈
Have you seen these tests https://github.com/uNetworking/h1spec
uNetworkingAB commentedon Aug 16, 2026
You need this commit to pass the no-body test uNetworkingAB/Http11Probe@54b9fad
MDA2AV commentedon Aug 16, 2026
I'll take a look if probe is missing any of those and add thanks.
It doesn't currently test for tcp fragmentation as it isn't http protocol related, those type of tests are on httparena's validator.
uNetworkingAB commentedon Aug 18, 2026
I know from testing other servers that many of them lack fragmentation support (entirely). I was thinking, the simplest and most through test for fragmentation would probably be to just have 1 maybe 2 maybe 3 valid HTTP requests and then just run them through 1-byte-per-100ms sends and see if all of them return 200. That should cover all possible failures with very little code. Maybe you don't even need to wait at all, if TCP_NAGLE is off for the sender?
uNetworkingAB commentedon Aug 18, 2026
Ah wait I'm so stupid. What you do is - you take the valid HTTP request, you chop it up in 2 parts at all the byte offsets possible, then you send them all at the same time on its own connection, then wait 1 second, send the other chunk. That way the test finishes in 1 second and covers all breaking spots.
uNetworkingAB commentedon Aug 18, 2026
https://github.com/uNetworking/uWebSockets/blob/master/tests/fragment_test.ts does that
nigrosimone commentedon Aug 19, 2026
Just a quick note: I’ve added many of the most famous HTTP servers, including yours, see:
MDA2AV commentedon Aug 19, 2026
If you want to test TCP frag properly, enable TLS and send a fragment with less than 5 byte, that is the minimum required for TLS 1.3, this will break the ankles of some frameworks. The 100ms probably help or the Kernel might rearrange the fragments into a single read. Chunked encoding is usually as good candidate for it too, hit the fragmentation in the chunk length.
But if we really want to check an edge case, try QUIC connection migration with certificate rotation that invalidates a certificate the moment the connection is migrating.
uNetworkingAB commentedon Aug 19, 2026
That doesn't actually test the server's parser for all the (multiple various kinds of) failures from not handling fragmentation. A server can handle fragmentation just fine at some place but fail completely at another place. So you need to test all byte offsets like the https://github.com/uNetworking/uWebSockets/blob/master/tests/fragment_test.ts does (I added it to my CI testing).
MDA2AV commentedon Aug 19, 2026
That is a nice test, added an equivalent script to httparena validation suite, I added query params to it. Does roughly 1050 frag combinations total.
https://www.http-arena.com/docs/test-profiles/h1/isolated/baseline/validation/