simd: two-column GROUP BY — GroupKeyAddr::Pair + five _pair keyed reductions - #324
Merged
Merged
Conversation
…uctions
A multi-key `GROUP BY a, b` addressed as ONE composite group
`hi[i] * stride + lo[i]`, fused into the existing walker: no composite key
lane is ever materialised, and every fold closure is the Resident sibling's,
unchanged. `group_walk` stays the one bit loop; this is a third address, not
a third walker.
- `GroupKeyAddr::Pair { hi, lo, stride }`. A minor key at or past `stride`
names no group and is DROPPED (the family's zero-fallback contract, never
an error). The composite is formed in u64, so it cannot overflow:
(2^32-1)^2 + (2^32-2) < 2^64. It then meets the same `k < groups` drop.
- Public: `masked_group_{count_u32,sum_i32,sum_sym_i32,min_i32,max_i32}_pair`,
each directly after its `_via` sibling, re-exported from `ndarray::simd`.
Row population = `hi.len()`; `hi.len() != lo.len()` panics, and so does a
values length mismatch.
Tests (group_family_tests): all five `_pair` folds equal their Resident
sibling fed the precomputed composite key (n = 1000 with a dirty tail
word; anti-vacuity >= 20 non-empty groups); `lo == stride - 1` kept while
`lo == stride` and `lo == u32::MAX` are dropped; hi = stride = u32::MAX
drops past the group universe without overflow, while a small composite
lands; the mismatched-length panic. Four doctests.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01GXUahz73MZxtxWcfpHp9dG
…ias real slots) The first version dropped rows whose composites also fell past out.len(), so the group-universe check dropped them too and deleting the lo >= stride guard left it green. Now out has 12 slots: (hi=0, lo=4) composes to 4, which is (1, 0)'s slot, and (hi=1, lo=5) composes to 9, which is (2, 1)'s. Only the stride guard keeps them out. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01GXUahz73MZxtxWcfpHp9dG
Contributor
|
Navigate logical layers of code changes, visualize relationships, and explore their blast radius. Note Currently processing new changes in this PR. This may take a few minutes, please wait... ⚙️ Run configurationConfiguration used: Organization UI Review profile: CHILL Plan: Essentials Run ID: 📒 Files selected for processing (2)
✨ Finishing Touches📝 Generate docstrings
Comment |
Bugbot couldn't run - usage limit reachedBugbot is counted against Cursor usage for this user or team, and this run hit a usage or spend limit. A user or team admin can review and increase usage limits in the Cursor dashboard. (requestId: serverGenReqId_7141b239-e6c3-467a-b193-2a2cbcef02b9) |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What
Adds multi-key
GROUP BY a, bto the keyed-reduction family as a third group address, not a third walker. The group of rowiis the compositehi[i] * stride + lo[i], fused intogroup_walk, so no composite key lane is ever materialised. Every fold closure is copied unchanged from its Resident sibling.GroupKeyAddr::Pair { hi, lo, stride }:lo >= stridenames no group and is dropped. This follows the family's zero-fallback contract; it is never an error.(2^32-1)^2 + (2^32-2) < 2^64, so it cannot overflow. It then meets the usualk < groupsdrop.ndarray::simd:masked_group_{count_u32, sum_i32, sum_sym_i32, min_i32, max_i32}_pair.This is the ndarray half of lance-graph's parity item W-D (multi-key GROUP BY). The mask-risc / quack lowering follows in lance-graph, whose CI resolves ndarray through the local path dep.
Gates (local, toolchain 1.98.1, debug=0)
group_family_tests: 14/14 passing, 4 of them new. Doctests: 4/4. clippy--lib -D warningsclean;cargo fmt --checkclean.Differential: all five
_pairfolds equal their Resident sibling fed the precomputed composite key. Fixture: n = 1000 with a dirty tail word; anti-vacuity requires ≥ 20 non-empty groups.Disable runs. I committed first, broke each guard, and all three went red:
lo >= strideguard →pair_drops_a_minor_key_at_or_past_stridefails.hi/lo→ three tests fail.The first version of the stride test was vacuous: its dropped rows also fell past
out.len(), so the universe check dropped them anyway. It is rewritten so each dropped row would otherwise land in another group's slot ((0, 4)→ slot 4 =(1, 0)), and it has been re-verified red → green.🤖 Generated with Claude Code
https://claude.ai/code/session_01GXUahz73MZxtxWcfpHp9dG
Generated by Claude Code