POC: Memory profiler allocation labels

[rudolf]

This is a POC for initial feedback. If we can get alignment within Node.js I could try to contribute the V8 changes upstream.

Summary

Adds the ability to tag sampling heap profiler allocations with string labels that propagate through async context (via CPED). This enables attributing memory usage to specific HTTP routes, tenants, or operations.

V8 changes

The V8 surface is intentionally minimal and avoids committing to string types.

• AllocationProfile::Sample::label_id (uint32_t, behind V8_HEAP_PROFILER_SAMPLE_LABELS) — a compact 4-byte identifier instead of a vector<pair<string,string>>. Sample remains a plain aggregate.
• LabelInternTable — process-wide intern table mapping Global<Value> to uint32_t label_id with refcount. One reference per sample; the table releases the Global when the last sample referencing it is collected.
• HeapProfiler::SetHeapProfileSampleLabelsKey — the on/off switch and the CPED Map lookup key. When unset, zero cost in the allocation path.
• HeapProfiler::InternLabelValue / ReleaseLabelValue / ResolveLabelValue — embedder API for allocation trackers that need to share the intern table.

No callback registration. The design associates any JS value (not just string pairs) with each sample and lets the profile consumer interpret it. The same mechanism is reusable for CPU profile samples (Szegedi's v8-dev proposal targets that use-case).

Attila Szegedi proposed a similar label mechanism for CPU profiling on v8-dev (July 2025). V8 team (Leszek Swirski) indicated they would review non-invasive patches behind #ifdefs.

Node.js changes

• v8.withHeapProfileLabels(labels, fn) — runs fn with labels propagating across await
• v8.setHeapProfileLabels(labels) — sets labels for the current async scope (for framework middleware patterns)
• v8.getAllocationProfile() returns samples[].labels (frozen, shared across samples with the same label set) and per-label externalBytes.
• ProfilingArrayBufferAllocator tracks external allocations per label_id (one atomic null-check per allocation/free when disabled, predicted not-taken).

The JS-facing API is unchanged. Label resolution (label_id to flat array to labels object) happens in the Node.js binding with a per-call cache, not in V8.

#62273 landed the SyncHeapProfileHandle API with Symbol.dispose support. The labels API proposed here is complementary.

Motivation

In multi-tenant or multi-route Node.js servers, a memory spike today tells you how much memory grew but not what caused it. With labeled heap/external memory profiling, you can answer "route /api/search accounts for 400 MB of the 1.2 GB heap" directly from production telemetry (e.g. via OTel).

This mirrors Go's pprof.Labels capability.

Read-time performance

The label_id design moves getAllocationProfile() from O(total samples) to O(distinct label sets): each unique label_id is resolved once and the frozen object is shared across all samples with that id. Benchmark results (5000 retained samples, n=5 calls per run):

samples/unique set	unique sets	pre-label_id ops/s	label_id ops/s	speedup
1	5000	115	160	1.4x
10	500	213	326	1.5x
100	50	225	383	1.7x
1000	5	223	376	1.7x

Overhead

20-run benchmark (two-server realistic HTTP workload):

• Sampling profiler alone: 0.6% (not statistically significant)
• Sampling + labels: 2.2% total (p<0.01)
• When disabled: one atomic null-check per Allocate/Free (predicted not-taken)

Test plan

• V8 cctests: label internment, dedup, refcount lifecycle, concurrent release
• Node.js cctests: label resolution, GC interaction
• JS tests: label propagation across await, concurrent contexts, setHeapProfileLabels, external memory tracking, GC cleanup, HTTP regression
• Benchmarks: benchmark/v8/heap-profiler-labels-resolution.js, benchmark/http/heap-profiler-realistic.js

[nodejs-github-bot]

Review requested:

• @nodejs/gyp
• @nodejs/performance
• @nodejs/security-wg
• @nodejs/v8-update

[Qard]

Storing in binding data by the CPED address won't work at all. Because all AsyncLocalStorage contexts are combined into a single AsyncContextFrame map, any changes to any contexts will change what this value is, even if the particular store you are interested in has not changed at all within that map frame.

You would need to have V8 capture the CPED value at the time of the sample and store that on the heap profile itself alongside the samples, then use that actual AsyncContextFrame instance to look up what the corresponding data was in that frame for the label store.

[szegedi]

Very interesting, I had to take a look since I was mentioned in the PR description itself 😄. I generally like the direction this is going in, long term we can probably replace Datadog's heap profiler that directly wraps V8 heap profiler with this and reduce our maintenance surface. This solution indeed looks like it can only be implemented in Node.js itself, and not as an add-on since the v8::ArrayBuffer::Allocator instance is a global Isolate::CreateParams setting so it's controlled by the embedder, that is, Node.js.

[szegedi]

Can't you get the value associated with the AsyncLocalStorage from the AsyncContextFrame here and only store that? Essentially the additional step from BuildSamples. You'll be retaining in memory all ALSes this ACF references as keys, some might have large retained set themselves. If you can safely call GetContinuationPreservedEmbedderData (which creates a v8::Local) I'd think you can also safely get a local to the ALS key from its global, and call v8::Map::Get on context too? (Or direct V8 hashmap reading like I suggested in that other comment in ProfilingArrayBufferAllocator::FindCurrentLabels)

[rudolf]

Great point about memory retention. You're right that storing the entire CPED keeps all ALS stores alive as long as the sample exists, not just the labels store 💣 💥

OrderedHashMap::FindEntry is a great suggestion!

[szegedi]

What's the memory requirement of this AsArray call? It sounds like it'd have to construct a whole new array?
You're lucky that you have a whole embedded copy of V8, so you can use existing internals, something like this roughly sketched might work:

#include "src/objects/js-collection.h"
#include "src/objects/ordered-hash-table.h"

// Given a v8::Local<v8::Map>, get to the internal table:
i::Tagged<i::JSMap> js_map = *Utils::OpenDirectHandle(*frame);
i::Tagged<i::OrderedHashMap> table = i::Cast<i::OrderedHashMap>(js_map->table());

// no-JS lookup in the table:
i::InternalIndex entry = table->FindEntry(isolate, *Utils::OpenDirectHandle(*als_key));
if (entry.is_found()) {
  i::Tagged<i::Object> value = table->ValueAt(entry);
  // go back from Tagged to Local:
  v8::Local<v8::Value> val = Utils::ToLocal(i::direct_handle(value, i_isolate));

  // use val as before...
}

[rudolf]

yeah AsArray() allocates a full JS Array and copies the Map backing store. I moved away from AsArray() to Map::Get() in an unpushed revision, but your OrderedHashMap::FindEntry approach would be even better.

Since we're already modifying V8 source in deps/v8/src/profiler/ and have access to internals from src/ as well, I can adopt this pattern in both locations:

1. SampleObject (allocation time) — extract ALS value, store as Global on Sample
2. ProfilingArrayBufferAllocator::TrackAllocate — same pattern for ArrayBuffer tracking

The read-time callback in GetAllocationProfile then receives the already-extracted flat array, making it trivial (just string conversion).

[szegedi]

So I presume this'll need upstreaming, right?

[rudolf]

yeah, I'm hoping if we can show that Nodejs would definitely use this feature they'd be more open to accepting it

[rudolf]

Very interesting, I had to take a look since I was mentioned in the PR description itself 😄. I generally like the direction this is going in, long term we can probably replace Datadog's heap profiler that directly wraps V8 heap profiler with this and reduce our maintenance surface.

@szegedi Thanks for popping by! Your thread sparked this idea and made me think maybe it's not all that hard (at least for memory profiling, sounds like CPU profiles might be a different beast).

[rudolf]

Great point about memory retention. You're right that storing the entire CPED keeps all ALS stores alive as long as the sample exists, not just the labels store 💣 💥

OrderedHashMap::FindEntry is a great suggestion!

[rudolf]

yeah AsArray() allocates a full JS Array and copies the Map backing store. I moved away from AsArray() to Map::Get() in an unpushed revision, but your OrderedHashMap::FindEntry approach would be even better.

Since we're already modifying V8 source in deps/v8/src/profiler/ and have access to internals from src/ as well, I can adopt this pattern in both locations:

1. SampleObject (allocation time) — extract ALS value, store as Global on Sample
2. ProfilingArrayBufferAllocator::TrackAllocate — same pattern for ArrayBuffer tracking

The read-time callback in GetAllocationProfile then receives the already-extracted flat array, making it trivial (just string conversion).

[rudolf]

yeah, I'm hoping if we can show that Nodejs would definitely use this feature they'd be more open to accepting it

[IlyasShabi]

This means we are storing the label value per sample, even when many samples share the same label set. I think this is likely expensive in memory and in GetAllocationProfile()

IMO it's better to store a small uint32_t label_id = 0 on each sample instead, and keep a shared map for labels. I believe that with this GetAllocationProfile() could resolve labels once per unique context rather than once per sample.

[rudolf]

I agree in principle but wonder if the memory overhead is really an issue. We only retain samples and memory that have survived GC. Starting with per sample overhead: ~120 B base + ~24 B label handle slot = ~144 B. And assuming Poisson sampling with 512kb intervals:

• 200 MB live heap would have ~400 retained samples with 58 KB / 0.029% sample overhead (48 KB base, 10 KB labels)
• 1GB live heap would have ~2000 samples with 288kb sample overhead (240KB base, 48KB labels)

There's additional overhead like the shared ALS array but that's a function of number of unique lables not a per sample overhead so would be present in both designs. Feels worth profiling/benchmarking to make sure my math adds up, but the label_id solution comes at the cost of additional complexity for a small memory gain.

It makes a lot of sense to add a local cache during GetAllocationProfile to avoid the expensive callbacks for labels we have already resolved. I think that can make a noticeable difference to CPU overhead.

[IlyasShabi]

We only retain samples and memory that have survived GC

Probably not if you call it like:

hp->StartSamplingHeapProfiler(
             64, 16,
             static_cast<v8::HeapProfiler::SamplingFlags>(
                   v8::HeapProfiler::kSamplingIncludeObjectsCollectedByMinorGC |
                   v8::HeapProfiler::kSamplingIncludeObjectsCollectedByMajorGC
              )
       );

[rudolf]

@IlyasShabi I'm stuck trying to measure this. The labels work adds bookkeeping that lives in C++ heap so process.memoryUsage().rss and v8.getHeapStatistics().used_heap_size don't directly capture it.

What I've tried so far:

• v8.getHeapStatistics().used_global_handles_size surprisingly doesn't track Sample::label_value Globals
• RSS deltas across 3-iteration matrix runs at 30 s each has a noise band of +/-60 MB

Is there a methodology you'd use to measure this kind of work?

[rudolf]

Did some measurement work on this. With a realistic HTTP workload (~2,500 rps, ~750 MB/s V8 alloc churn), 30 s with includeCollectedObjects: true at64 KB sampling interval, ~300K retained samples: aggregate libc-malloc shows the labels feature adds 6-9 MB on top of ~29 MB of profiler-only bookkeeping. Below 5% of the underlying sampler cost at this workload, scales linearly with sample count.

However, while running the load test I hit a segfault in ProfilingArrayBufferAllocator::TrackFree: V8's ArrayBufferSweeper calls it on a background worker thread, and the per-allocation Global label_value runs ~Global off-thread which trips the node->IsInUse() CHECK in GlobalHandles::NodeSpace::Release.

The fix it I think we need the same machinery as what you suggested for Sample::label_value so will take a stab at that

[IlyasShabi]

We should call ensureHeapProfileLabelsALS() before this line to make sure the ALS store exists and its key is registered before sampling begins.

[szegedi]

@szegedi Thanks for popping by! Your thread sparked this idea and made me think maybe it's not all that hard (at least for memory profiling, sounds like CPU profiles might be a different beast).

Oh yeah, I could probably talk for hours about CPU sample labeling :-)

Reading labels to associate with samples is the tricky part with CPU profiles. Since it happens in a signal handler, we we can't use basically any V8 APIs, as even getting a Local<Value> from GetContinuationPreservedEmbedderData violates signal safety. So in our profiler we end up with code like this as well as a reimplementation of V8 logic for reading hashmaps.

There's also some other fun details, like needing to have a ringbuffer for samples around as you can't allocate memory in the signal handler. And we're also using shared_ptr<Global<Value>> to store the reference to the current label set as we can safely copy a shared_ptr but couldn't be allocating a different global (globals can't be copied either.) FWIW, in our implementation, we aren't converting labels up front, we just take a regular JS object representing them and store a reference to it on sampling, and then only generate labels from its entries when we build the profiles.

Associating labels in that ringbuffer with samples is also somewhat tricky, we do it by correlating monotonic clock values – we invoke v8::base::TimeTicks::Now in the signal handler and store the value in the ring buffer struct and then later correlate them with samples using v8::CpuProfiler::GetSampleTimestamp

As I said, I can probably go on for at least an hour. I might need to write a talk :-)

[Qard]

Reading labels to associate with samples is the tricky part with CPU profiles. Since it happens in a signal handler, we we can't use basically any V8 APIs, as even getting a Local<Value> from GetContinuationPreservedEmbedderData violates signal safety. So in our profiler we end up with code like this as well as a reimplementation of V8 logic for reading hashmaps.

Yeah, this is why I wanted to have V8 itself capture CPED pointer state in each sample internally so that could be read back as actual local values later when you process the profile tree. Never got around to that before I changed teams though.

[szegedi]

As far as design decisions go, have you considered not committing to having a set of string labels, but have a more generic facility for associating any JS value with samples? You wouldn't have to deal with string serialization and would just need to pass around a shareable global reference to a value (std::shared_ptr<v8::Global<v8::Value>> or a semantic equivalent in V8 internal structures.) Interpretation of the value would be up to the code using the resulting profile, and it can itself serialize the values in any way it wants to, the profiler code itself would carry zero performance overhead for string serialization as it would be doing no such thing.

This way you only implement a very generic facility in Node.js/V8 for associating values with samples, without restricting what those values can be. E.g. an OTEL tracing implementation might want to use a bigint for trace ID. An implementation might want to use numeric values for some labels. And so on.

[github-actions]

This pull request has been marked as stale due to 90 days of inactivity.
It will be automatically closed in 30 days if no further activity occurs. If this is still relevant, please leave a comment or update it to keep it open.