bench: add experimental node:bench module - #65606
Conversation
|
Review requested:
|
This comment was marked as outdated.
This comment was marked as outdated.
Codecov Report❌ Patch coverage is Additional details and impacted files@@ Coverage Diff @@
## main #65606 +/- ##
==========================================
+ Coverage 90.06% 90.14% +0.08%
==========================================
Files 754 764 +10
Lines 255747 258220 +2473
Branches 48318 48885 +567
==========================================
+ Hits 230327 232769 +2442
- Misses 16550 16557 +7
- Partials 8870 8894 +24
🚀 New features to boost your workflow:
|
|
@RafaelGSS probably nothing except that hopefully now it can progress. My question would be: why not vendor https://www.npmjs.com/package/bench-node? |
I initially considered vendoring one of the existing packages and opted against it in favor of a light weight, minimal, no-new-dependency approach that a more feature-rich tool like bench-node can build/iterate on. I don't consider these either/or options. Node.js' own benchmarks demonstrate that the full set of features provided by bench-node aren't necessary for a minimal bench suite.
We can provide stable measurement and structured result primitives without needing to ship the larger user-facing feature set and these existing tools can continue serving their current users while optionally adopting the built-in foundation. Just like That said, this is just a starting point. I've never been one to say "It must be done this specific way!" so all options are on the table. What I want is a built-in |
If I can't do it with
I'm with you. I just think it's easier to deliver what folks would actually want out of the module by bringing something like |
9176490 to
f5af443
Compare
different folks "actually want" different things. You prove this point yourself with the "If I can't do it with node:test I just don't worry about it" comment. Node.js' own benchmarks don't require any of the extended features of |
Maybe, but aren't standard library modules meant to provide utility to the community, not just the project itself? |
Why not both? I'm intentionally designing this for both cases. Standalone utility plus primitives that can be used by tools like |
f5af443 to
2c0ddc5
Compare
|
Large PR Review Guide: Given that this is a large PR for a new subsystem... some guidance for review:
|
This comment was marked as outdated.
This comment was marked as outdated.
This comment was marked as outdated.
This comment was marked as outdated.
|
How does this compare to mitata? |
Signed-off-by: James M Snell <jasnell@gmail.com> Assisted-by: Opencode
Signed-off-by: James M Snell <jasnell@gmail.com> Assisted-by: Opencode
Signed-off-by: James M Snell <jasnell@gmail.com> Assisted-by: Opencode
Signed-off-by: James M Snell <jasnell@gmail.com> Assisted-by: Opencode
Signed-off-by: James M Snel <jasnell@gmail.com> Assisted-by: Opencode
Signed-off-by: James M Snell <jasnell@gmail.com>
Makes it easier for benchmark tools to build on top of the bench runner primitives. Signed-off-by: James M Snell <jasnell@gmail.com> Assisted-by: Opencode
Signed-off-by: James M Snell <jasnell@gmail.com> Assisted-by: Opencode
Signed-off-by: James M Snell <jasnell@gmail.com> Assisted-by: Opencode
Signed-off-by: James M Snell <jasnell@gmail.com>
2c0ddc5 to
69ca552
Compare
|
@ronag ... I'm not familiar with mitata but I suspect the story is the same as |
This comment was marked as outdated.
This comment was marked as outdated.
|
Who's rushing to get anything merged? |
|
Oh, sorry, misunderstand the Marking the comment as resolved |
|
I plan to merge this on 2026-09-03 if there are no issues raised by then. With green CI, two TSC approvals, and the time that's passed, it is otherwise ready to land. Note that there is a second follow-up PR here #65631 that will be marked draft until this merges. |
H4ad
left a comment
There was a problem hiding this comment.
The current API goes in a different direction than any benchmark lib I saw so far (mitata, bench-node, benchmark.js, deno.bench) by not managing the benchmark runs automatically, making the user to explicitly define their own amount of runs.
As a base structure to be used to extend/implement other benchmarks on top of it, it's actually a good API surface to be this minimal and managed.
This avoid questions like "should we inline the benchmark", "how we handle async fn", "what is the minimum amount of runs", or "how we handle dead-code elimination".
A new
node:benchmodule, modeled closely after thenode:testarchitecture.The core functionality is here but there are still improvements that are needed (see #65631). The focus was on minimal functionality, not absolutely optimal functionality. But this gives a solid starting point for improvements.
Architecturally, this is intended to be a minimal benchmark primitive. It's useful on it's own but provides a base set of primitives that more full-featured benchmark tools can build on. Specifically, this is not a full replacement for things like
bench-node, etc. It shouldn't be judged as such.