[WIP] Robin Hood Hashing

[aterenin]

This PR implements Robin Hood hashing, through introducing a new probing scheme, as per #817. An initial-pass implementation for get and insert, and tests, are done.

The implementation works by adding if constexpr logic throughout to handle Robin Hood specifics as they come up:

• The big question: is this the right structure? As an alternative, I could refactor the code to write separate insert methods for Robin Hood vs. others, and dispatch according to table type.

Some more details about this question, with an example, are given at the very end of #817.

[copy-pr-bot]

This pull request requires additional validation before any workflows can run on NVIDIA's runners.

Pull request vetters can view their responsibilities here.

Contributors can view more details about this message here.

[PointKernel]

/ok to test 1f3d64e

[PointKernel]

@aterenin Just wanted to let you know we haven't forgotten about this work! @sleeepyjack and I are discussing the new design and will align on a concrete proposal before moving forward to minimize unnecessary iterations. We'll keep you posted.

[aterenin]

Sounds good!

The current WIP implements everything needed, along with tests. I've carefully gone over all the core logic, and I think is reasonably close to ready-to-review.

The main remaining item I was planning to think about more is whether there is a more maintenance-friendly way to organize and structure the code, which is what the "big question" in the PR centers on. I also have a list of a bunch of subtleties worth mentioning, but these should wait until a review.

If discussion would help, I'd be happy to jump on a Zoom call - just let me know!

[PointKernel]

We've had a chance to evaluate the proposal and would be interested in discussing it further over a video call.

@aterenin Could you send your availability and preferred contact details to yunsongw@nvidia.com? Wednesday through Friday next week would work well on our end. Once we have a few time slots, we can coordinate and send out a meeting invite.

[aterenin]

Email headed your way!

[aterenin]

@PointKernel / @sleeepyjack Would you mind giving an OK to test on 628889a?

(Additional note: this is not quite ready for review yet - stay tuned for that)

[PointKernel]

/ok to test 628889a

[aterenin]

@PointKernel And can we try again with another OK to test?

(Sorry for long delay - just got back from conference travel.)

[sleeepyjack]

/ok to test 40187dd

[aterenin]

I finally have access to my GPU box again, and have ran some initial benchmarks. The goal here was to successfully reproduce the results from the GPURHH repo in a manner fully contained inside cuCollections, and I believe this has been successful.

Setup:

• 4090
• static_map_insert_unique_occupancy
• 1GB hash table
• Occupancy: 0.75, 0.85, 0.95, 0.99
• NumInputs: calculated from occupancy to ensure table is the correct target size
• MatchingRate: calculated to be approximately-equivalent to GPURHH's original test, see below.

Note that the resulting MatchingRate is set relatively small. This needs some explanation: the GPURHH benchmark does independent uniform insertion and retrieval. In cuCo, a different setup is used: first, the system first generates a list of N keys by shuffling 0,..,N-1 into a random order. The same stream is reused for insertion and retrieval, except that, with probability 1 - MatchingRate, for retrieval each key is replaced with a new one that re-drawn from N,..,max_value. I have set MatchingRate to ensure the number of collisions is approximately the same as GPURHH in expectation. The effects here are big enough that I think the fine details here are unlikely to affects results much.

The exact numbers are 0.023, 0.027, 0.03, 0.031 depending on load factor. As a result, for small tables, most keys will not be present, which is a pretty favorable evaluation from Robin Hood's perspective - if I had thought more carefully about this at blog post time, I might have picked something a bit more balanced. Though, there are certainly use cases out there where this will likely be realistic.

This makes me think I should benchmark some more instances to get a broader view - I'd be curious what other benchmarking scenarios you'd potentially like to see? (@PointKernel / @sleeepyjack)

Otherwise, I think now would be a good time to speak again on Zoom. The current implementation should mostly match what was discussed, with the main caveat that the hard-wire required quite a bit of stuff to be commented out, because either they don't make sense for RHH (different atomic pattern, etc.) or for related reasons. A diff against the non-Robin-Hood open_addressing_ref_impl should be pretty informative of the state of the code. On my end, I'll probably take a minor clean-up pass before meeting to fully remind myself what's there, but do not anticipate I will change much if anything.

[aterenin]

The plot from yesterday left me feeling a little bit unsatisfied, so I looked into this a bit more today. Two updates:

• It turns out we can also use Robin Hood in erase to exit early if we try to erase an element which does not exist in the table anyway. This is now also implemented, so RH will potentially be faster in erase too.
• Benchmarked a wider range of MatchingRate values, to get a less-narrow view of what happens.

Core result: even for larger MatchingRate below one, we still see noticeable performance gains, but where the magnitude of the gains themselves is potentially a bit less.

[aterenin]

Finally: sorry for many messages - I should have previously wrote but forgot - can I get an /ok to test?

[PointKernel]

/ok to test 2b658a4

[PointKernel]

@aterenin Thanks for the follow-up, and we truly appreciate your contribution here.

We’ll take a look at the code and benchmark results and get back to you. Nowadays, with agent-assisted coding, there’s a lot more code to review, and we already have quite a few PRs in the pipeline.

[aterenin]

Thanks! No hurry from my end, just let me know, and I'll try my best to be responsive.

In the meantime, for some reason it looks like my fork never actually picked up upstream changes, even though I thought I had merged them in. Very strange - this should be fixed now, everything compiles again on my end. Do you mind giving a second /ok to test?