Skip to content

Adjust work spliting to subtasks, about 1.5x faster prefill on zen5 machines, and +- neutral on older ones. Importantly to fully utilize this it's important to use high prefill token_batch like 4k#953

Merged
copybara-service[bot] merged 1 commit into
devfrom
test_948310576
Jul 15, 2026