[TRTLLM-17239][feat] Locality domain support for the C++ KV cache manager v2 and two-stream attention by farazkh80 · Pull Request #20229 · NVIDIA/TensorRT-LLM

[None][feat] Per-locality-domain KV cache storage, allocation, eviction and resize in the C++ KV cache manager v2

farazkh80

committed

ba463cf

[None][feat] Locality domains in the Python KVCacheManagerV2 and request path

farazkh80

committed

890ce6b

[None][feat] Run attention once per locality domain on that domain's stream

farazkh80

committed

4a56b78

[None][perf] Size the MLA decode kernel's KV split for the SMs of its locality domain

farazkh80

committed

0ba4e1a

[None][feat] Transfer localized KV pools in disaggregated serving

farazkh80

committed

e593ea6

[None][doc] Add a short page on when to enable locality domains

farazkh80

committed

78cc94a

[None][chore] Derive a KV page's locality domain from its slot id in KVCM2

farazkh80

committed

a445406

[None][feat] Place new requests on the locality domain that holds their cached prefix

farazkh80

committed

3321e16

[None][fix] Give a draft KV cache manager a valid locality domain when its domain count differs from the target's

farazkh80

committed

efff447

[None][fix] Give each locality domain its own CuTe DSL MLA split plan

farazkh80

committed

4047607

[None][chore] Drop the unused request id from pick_locality_domain

farazkh80

committed

e78fba3

Navigation Menu

Repository navigation

[TRTLLM-17239][feat] Locality domain support for the C++ KV cache manager v2 and two-stream attention - #20229

[TRTLLM-17239][feat] Locality domain support for the C++ KV cache manager v2 and two-stream attention#20229
farazkh80 wants to merge 11 commits into
NVIDIA:mainfrom
farazkh80:faraz/ld-cpp-kvcm

Commits on Oct 29, 2026

[None][feat] Per-locality-domain KV cache storage, allocation, eviction and resize in the C++ KV cache manager v2

[None][feat] Locality domains in the Python KVCacheManagerV2 and request path

[None][feat] Run attention once per locality domain on that domain's stream

[None][perf] Size the MLA decode kernel's KV split for the SMs of its locality domain

[None][feat] Transfer localized KV pools in disaggregated serving

[None][doc] Add a short page on when to enable locality domains

[None][chore] Derive a KV page's locality domain from its slot id in KVCM2

[None][feat] Place new requests on the locality domain that holds their cached prefix

[None][fix] Give a draft KV cache manager a valid locality domain when its domain count differs from the target's

[None][fix] Give each locality domain its own CuTe DSL MLA split plan

[None][chore] Drop the unused request id from pick_locality_domain