Skip to content

Add to Http11Probe #1298

Closed
Closed

Description

@cesco69

From readme:

fastest standards compliant web server in the TechEmpower

See TechEmpower/FrameworkBenchmarks#10932

there is a new rising http benchemark https://http-arena.com

Activity

  1. uNetworkingAB commented on Aug 10, 2026

    @uNetworkingAB
    Contributor

    Interesting but also not relevant (to uWS). All of these benchmark projects just collect various hacks that have zero standards compliance. Also, we are not limited by not-winning-benchmarks, we are limited by Node.js itself. Also, this project already established itself as "very fast" and there is no more market to be had from participating in these lists. These kinds of lists were very relevant 10 years ago. Today, not so much.

  2. uNetworkingAB commented on Aug 10, 2026

    @uNetworkingAB
    Contributor

    TLDR: the main problem with these benchmark websites is that they never equalize standards compliance (there is no common standards compliance check-list all participants must pass to be part of the test), and so they all end up just collecting hacks that would be instant security holes if used in production.

  3. Kaliumhexacyanoferrat commented on Aug 14, 2026

    @Kaliumhexacyanoferrat

    To shortly chime in: HTTP Arena has a equalizing standard (validation phase) to ensure we are talking to web servers and not as with TFB Framework Benchmarks where servers did not even pass a simple TCP fragmentation check. The author did also write https://www.http-probe.com/ - the findings did influence the validation in arena a lot.

    Additionally, arena has a classification system to filter out those overfitting entries (also config changes are only supported in the tuned section). I would consider uWebSockets.js in the flagship tier, so it bypasses most of the problems you mentioned and framework benchmarks clearly had.

  4. uNetworkingAB commented on Aug 14, 2026

    @uNetworkingAB
    Contributor

    I would rather add uWS to Http11Probe and fix standards behavior than chase some benchmark list,

    Bun has 113 pass 20 warn 26 fail 54 unscored · 213 total, so we should get similar. Meaning there are things to fix first

  5. changed the title [-]TechEmpower is died[/-] [+]Add to Http11Probe[/+] on Aug 15, 2026
  6. MDA2AV commented on Aug 15, 2026

    @MDA2AV

    here MDA2AV/Http11Probe#151

    h1probe results are not an absolute metric, pass/warn/fail sometimes doesn't mean much as it always depend on the context, what matters is the behavior difference/divergence between entries. You can look at a failed test and say you are fine with it because you want the framework to behave like that, which is fine.

  7. uNetworkingAB commented on Aug 15, 2026

    @uNetworkingAB
    Contributor
  8. added a commit that references this issue on Aug 15, 2026
  9. uNetworkingAB commented on Aug 15, 2026

    @uNetworkingAB
    Contributor

    Wow that's very fast, so if this merges, I will see results at https://www.http-probe.com ? And with what interval are they run? Daily?

  10. MDA2AV commented on Aug 15, 2026

    @MDA2AV

    yes, once merged full probe is triggered that takes around 1hour ish and everything is updated on the site
    typically once a week, since not much contributions/changes happening. It can run on demand by one action trigger anytime someone requests.

  11. added a commit that references this issue on Aug 15, 2026
  12. uNetworkingAB commented on Aug 15, 2026

    @uNetworkingAB
    Contributor

    There are 21 failures only due to uWS giving 505 instead of 400. By swapping "default error" code to 400, we drastically would increase the score to top 20.

  13. uNetworkingAB commented on Aug 15, 2026

    @uNetworkingAB
    Contributor

    I get 137/159 now

  14. MDA2AV commented on Aug 15, 2026

    @MDA2AV

    I'd say score itself isn't very important, finding tests where behavior diverges from the majority expected is, actually there are 2-3 tests where passing is the anomaly. A higher score is a plus for sure but the highest scoring ones there are actually parsers that people overfitted to the test suite using AI agents, so unlikely to be a good solution either.

    There are roughly 50 tests not yet in the repo, numbers will be increasing over time

  15. uNetworkingAB commented on Aug 16, 2026

    @uNetworkingAB
    Contributor

    145/159 👌 📈

    Have you seen these tests https://github.com/uNetworking/h1spec

  16. uNetworkingAB commented on Aug 16, 2026

    @uNetworkingAB
    Contributor

    You need this commit to pass the no-body test uNetworkingAB/Http11Probe@54b9fad

  17. MDA2AV commented on Aug 16, 2026

    @MDA2AV

    145/159 👌 📈

    Have you seen these tests https://github.com/uNetworking/h1spec

    I'll take a look if probe is missing any of those and add thanks.

    It doesn't currently test for tcp fragmentation as it isn't http protocol related, those type of tests are on httparena's validator.

  18. uNetworkingAB commented on Aug 18, 2026

    @uNetworkingAB
    Contributor

    I know from testing other servers that many of them lack fragmentation support (entirely). I was thinking, the simplest and most through test for fragmentation would probably be to just have 1 maybe 2 maybe 3 valid HTTP requests and then just run them through 1-byte-per-100ms sends and see if all of them return 200. That should cover all possible failures with very little code. Maybe you don't even need to wait at all, if TCP_NAGLE is off for the sender?

  19. uNetworkingAB commented on Aug 18, 2026

    @uNetworkingAB
    Contributor

    Ah wait I'm so stupid. What you do is - you take the valid HTTP request, you chop it up in 2 parts at all the byte offsets possible, then you send them all at the same time on its own connection, then wait 1 second, send the other chunk. That way the test finishes in 1 second and covers all breaking spots.

  20. uNetworkingAB commented on Aug 18, 2026

    @uNetworkingAB
    Contributor
  21. nigrosimone commented on Aug 19, 2026

    @nigrosimone
    Contributor

    Just a quick note: I’ve added many of the most famous HTTP servers, including yours, see:

  22. MDA2AV commented on Aug 19, 2026

    @MDA2AV

    I know from testing other servers that many of them lack fragmentation support (entirely). I was thinking, the simplest and most through test for fragmentation would probably be to just have 1 maybe 2 maybe 3 valid HTTP requests and then just run them through 1-byte-per-100ms sends and see if all of them return 200. That should cover all possible failures with very little code. Maybe you don't even need to wait at all, if TCP_NAGLE is off for the sender?

    If you want to test TCP frag properly, enable TLS and send a fragment with less than 5 byte, that is the minimum required for TLS 1.3, this will break the ankles of some frameworks. The 100ms probably help or the Kernel might rearrange the fragments into a single read. Chunked encoding is usually as good candidate for it too, hit the fragmentation in the chunk length.

    But if we really want to check an edge case, try QUIC connection migration with certificate rotation that invalidates a certificate the moment the connection is migrating.

  23. uNetworkingAB commented on Aug 19, 2026

    @uNetworkingAB
    Contributor

    That doesn't actually test the server's parser for all the (multiple various kinds of) failures from not handling fragmentation. A server can handle fragmentation just fine at some place but fail completely at another place. So you need to test all byte offsets like the https://github.com/uNetworking/uWebSockets/blob/master/tests/fragment_test.ts does (I added it to my CI testing).

  24. MDA2AV commented on Aug 19, 2026

    @MDA2AV

    That is a nice test, added an equivalent script to httparena validation suite, I added query params to it. Does roughly 1050 frag combinations total.
    https://www.http-arena.com/docs/test-profiles/h1/isolated/baseline/validation/

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    help wantedExtra attention is needed

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions