Skip to content

simd: two-column GROUP BY — GroupKeyAddr::Pair + five _pair keyed reductions - #324

Merged
AdaWorldAPI merged 2 commits into
masterfrom
claude/fold-distillation-pr-wave-s57uj7
Sep 23, 2026
Merged

AdaWorldAPI merged 2 commits into
masterfrom
claude/fold-distillation-pr-wave-s57uj7

Conversation

@AdaWorldAPI

Copy link
Copy Markdown
Owner

What

Adds multi-key GROUP BY a, b to the keyed-reduction family as a third group address, not a third walker. The group of row i is the composite hi[i] * stride + lo[i], fused into group_walk, so no composite key lane is ever materialised. Every fold closure is copied unchanged from its Resident sibling.

  • GroupKeyAddr::Pair { hi, lo, stride }:
    • A minor key lo >= stride names no group and is dropped. This follows the family's zero-fallback contract; it is never an error.
    • The composite is formed in u64, where (2^32-1)^2 + (2^32-2) < 2^64, so it cannot overflow. It then meets the usual k < groups drop.
  • New public functions, re-exported from ndarray::simd: masked_group_{count_u32, sum_i32, sum_sym_i32, min_i32, max_i32}_pair.

This is the ndarray half of lance-graph's parity item W-D (multi-key GROUP BY). The mask-risc / quack lowering follows in lance-graph, whose CI resolves ndarray through the local path dep.

Gates (local, toolchain 1.98.1, debug=0)

  • group_family_tests: 14/14 passing, 4 of them new. Doctests: 4/4. clippy --lib -D warnings clean; cargo fmt --check clean.

  • Differential: all five _pair folds equal their Resident sibling fed the precomputed composite key. Fixture: n = 1000 with a dirty tail word; anti-vacuity requires ≥ 20 non-empty groups.

  • Disable runs. I committed first, broke each guard, and all three went red:

    • Delete the lo >= stride guard → pair_drops_a_minor_key_at_or_past_stride fails.
    • Transpose hi/lo → three tests fail.
    • u32 wrapping arithmetic instead of u64 → the no-overflow test fails.

    The first version of the stride test was vacuous: its dropped rows also fell past out.len(), so the universe check dropped them anyway. It is rewritten so each dropped row would otherwise land in another group's slot ((0, 4) → slot 4 = (1, 0)), and it has been re-verified red → green.

🤖 Generated with Claude Code

https://claude.ai/code/session_01GXUahz73MZxtxWcfpHp9dG


Generated by Claude Code

…uctions

A multi-key `GROUP BY a, b` addressed as ONE composite group
`hi[i] * stride + lo[i]`, fused into the existing walker: no composite key
lane is ever materialised, and every fold closure is the Resident sibling's,
unchanged. `group_walk` stays the one bit loop; this is a third address, not
a third walker.

- `GroupKeyAddr::Pair { hi, lo, stride }`. A minor key at or past `stride`
  names no group and is DROPPED (the family's zero-fallback contract, never
  an error). The composite is formed in u64, so it cannot overflow:
  (2^32-1)^2 + (2^32-2) < 2^64. It then meets the same `k < groups` drop.
- Public: `masked_group_{count_u32,sum_i32,sum_sym_i32,min_i32,max_i32}_pair`,
  each directly after its `_via` sibling, re-exported from `ndarray::simd`.
  Row population = `hi.len()`; `hi.len() != lo.len()` panics, and so does a
  values length mismatch.

Tests (group_family_tests): all five `_pair` folds equal their Resident
sibling fed the precomputed composite key (n = 1000 with a dirty tail
word; anti-vacuity >= 20 non-empty groups); `lo == stride - 1` kept while
`lo == stride` and `lo == u32::MAX` are dropped; hi = stride = u32::MAX
drops past the group universe without overflow, while a small composite
lands; the mismatched-length panic. Four doctests.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01GXUahz73MZxtxWcfpHp9dG
…ias real slots)

The first version dropped rows whose composites also fell past out.len(),
so the group-universe check dropped them too and deleting the lo >= stride
guard left it green. Now out has 12 slots: (hi=0, lo=4) composes to 4,
which is (1, 0)'s slot, and (hi=1, lo=5) composes to 9, which is (2, 1)'s.
Only the stride guard keeps them out.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01GXUahz73MZxtxWcfpHp9dG
@coderabbitai

coderabbitai Bot commented Sep 23, 2026 •

Copy link
Copy Markdown
Contributor

Review in Change Stack →

Navigate logical layers of code changes, visualize relationships, and explore their blast radius.

Note

Currently processing new changes in this PR. This may take a few minutes, please wait...

⚙️ Run configuration

Configuration used: Organization UI

Review profile: CHILL

Plan: Essentials

Run ID: e227b663-b175-40d3-bc31-4a88f68d4d1e

📥 Commits

Reviewing files that changed from the base of the PR and between 6ae40cd and 2456a84.

📒 Files selected for processing (2)
  • src/simd.rs
  • src/simd_masking_ops.rs
 _________________________________________
< Time zones: the final boss of software. >
 -----------------------------------------
  \
   \   (\__/)
       (•ㅅ•)
       /   づ
✨ Finishing Touches
📝 Generate docstrings
  • Commit to this branch
  • Create a new PR

Comment @coderabbitai help to get the list of available commands.

@cursor

cursor Bot commented Sep 23, 2026

Copy link
Copy Markdown

Bugbot couldn't run - usage limit reached

Bugbot is counted against Cursor usage for this user or team, and this run hit a usage or spend limit.

A user or team admin can review and increase usage limits in the Cursor dashboard.

(requestId: serverGenReqId_7141b239-e6c3-467a-b193-2a2cbcef02b9)

@AdaWorldAPI
AdaWorldAPI marked this pull request as ready for review September 23, 2026 20:24
@AdaWorldAPI
AdaWorldAPI merged commit 2c91538 into master Sep 23, 2026
27 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants