[aw471]

Summary

Add a private tensor-native helper for selecting variable-length payloads with a statically bounded output capacity. This prepares string sorting for fullgraph compilation.

Description of the change

Preserve exact logical offsets, null masks and element order while allowing unused padding after the last payload byte. The helper avoids reading the compact byte count back to Python; general public indexing remains unchanged.

Stacked on shared UTF-8 packing and byte bounds. Incremental production change: 36 added lines in one file, excluding tests and the dependency.

Validation

Focused variable-length and packing tests: 78 passed locally; the combined preprocessing stack passed 141 focused CPU/CUDA tests. Full pre-commit checks passed. Coverage includes repeated, reordered, negative, empty, strided and broadcast selections, UTF-8 and nonempty null payloads.

On one RTX PRO 6000 Blackwell GPU with PyTorch 2.14.0+cu130, selecting 5,000 payloads from 40,000 Bank Marketing job strings took 0.135 ms native compiled versus 0.473 ms for registration of main's exact-compaction implementation, a 3.5× improvement. Eager execution was approximately tied at 0.366 versus 0.365 ms.

Both compiled variants used Inductor with fullgraph=True; native capture produced one graph without the registered selection operation. Times are warm synchronized wall-clock medians from 25 alternating samples, excluding compilation. Native storage includes padding, so physical allocation can exceed exact compaction.

[copy-pr-bot]

This pull request requires additional validation before any workflows can run on NVIDIA's runners.

Pull request vetters can view their responsibilities here.

Contributors can view more details about this message here.

[codecov]

Codecov Report

❌ Patch coverage is 96.66667% with 1 line in your changes missing coverage. Please review.
⚠️ Please upload report for BASE (compile/kumotabular-native-string-keys@cf8fe3b). Learn more about missing BASE report.
✅ All tests successful. No failed tests found.

Files with missing lines	Patch %	Lines
sdm/tensor/var_len.py	96.66%	1 Missing ⚠️

Additional details and impacted files

@@                            Coverage Diff                            @@
##             compile/kumotabular-native-string-keys    #1271   +/-   ##
=========================================================================
  Coverage                                          ?   83.10%           
=========================================================================
  Files                                             ?      177           
  Lines                                             ?    11461           
  Branches                                          ?        0           
=========================================================================
  Hits                                              ?     9525           
  Misses                                            ?     1936           
  Partials                                          ?        0           

☔ View full report in Codecov by Harness.
📢 Have feedback on the report? Share it here.