[aw471]

Summary

Use native PyTorch string-category matching on CUDA; keep CPU on Arrow.

Description of the change

Compare packed keys directly for small vocabularies and group sorted keys for larger ones. There is no fixed byte-length cap. Native dispatch requires known length bounds; raw/cuDF inputs with unknown bounds retain the ordinary eager join, which is not fullgraph-capable.

The 65,536-element comparison budget switches between two native algorithms; it does not restrict accepted input size. This changes AlignCategories matching, not the general join API. Depends on #1274.

Validation

One RTX PRO 6000, balanced original/native/native/original processes. Warm vocabulary-mapping timings include packing but exclude query-row code gathering, input construction and compilation startup. Torch peaks are per-call; cuDF/RMM peaks are shown separately and are not summed with Torch peaks.

Vocabulary profile	Main eager, ms	Native eager, ms	Native compiled, ms	Torch peak MiB: main → native eager → compiled
35/6/4 categories, 128 bytes	6.157	1.348	0.138	0.032 → 0.146 → 0.062
35/6/4 categories, 256 bytes	6.171	1.354	0.140	0.054 → 0.277 → 0.116
300 categories, 128 bytes	5.887	1.313	0.723	0.159 → 1.164 → 0.469

Original cuDF separately used 0.026, 0.048 and 0.173 MiB RMM respectively; native paths used zero. Wide lookup trades temporary memory for latency, especially in eager mode at 300 categories. Independent allocator peaks are not summed, and no universal memory improvement is claimed. Exact outputs, unknown raw/cuDF fallback, known-width pickle restoration and static/dynamic CUDA fullgraph lookup passed; the final combined run passed 11 focused CUDA lookup cases.

[copy-pr-bot]

This pull request requires additional validation before any workflows can run on NVIDIA's runners.

Pull request vetters can view their responsibilities here.

Contributors can view more details about this message here.

[codecov]

Codecov Report

❌ Patch coverage is 95.83333% with 1 line in your changes missing coverage. Please review.
✅ Project coverage is 83.10%. Comparing base (e053cc3) to head (28d242f).
✅ All tests successful. No failed tests found.

Files with missing lines	Patch %	Lines
sdm/processing/categorical/align.py	95.83%	1 Missing ⚠️

Additional details and impacted files

@@                           Coverage Diff                           @@
##           compile/kumotabular-native-string-keys    #1275   +/-   ##
=======================================================================
  Coverage                                   83.09%   83.10%           
=======================================================================
  Files                                         177      177           
  Lines                                       11447    11453    +6     
=======================================================================
+ Hits                                         9512     9518    +6     
  Misses                                       1935     1935           

☔ View full report in Codecov by Harness.
📢 Have feedback on the report? Share it here.