Keep PostStore records when a sibling index evicts - #57
Open
Pitchfork-and-Torch wants to merge 1 commit into
Open
Keep PostStore records when a sibling index evicts#57Pitchfork-and-Torch wants to merge 1 commit into
Pitchfork-and-Torch wants to merge 1 commit into
Conversation
Capacity eviction from video_posts_by_user (or original/secondary) unconditionally removed the CompactPost. An original video is indexed in both lists, so overflowing the video list with retweets dropped a still-indexed original from serving. Co-authored-by: Jon Bailey <Pitchfork-and-Torch@users.noreply.github.com>
Pitchfork-and-Torch
marked this pull request as ready for review
August 14, 2026 22:55
This was referenced Aug 15, 2026
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Bug
PostStore::insert_posts_internalindexes an original video in bothoriginal_posts_by_userandvideo_posts_by_user. When either deque hitsMAX_POSTING_LIST_SIZE(5000), the oldest id is popped andposts.removeruns unconditionally.get_all_posts_by_users/get_videos_by_userslook upCompactPostby that id. If the video list overflows first — a user who retweets a lot of video, which is the common fill path — the original is still on the original index, but the canonical record is gone.filter_mapdrops it. The viewer following that author no longer sees a post that is still inside the original posting list and inside retention.This is not the same-second insert drop (#48). It is a cross-index consistency hole on capacity eviction.
Fix
Only remove from
postswhen the author no longer indexes the id in original, secondary, or video. Overflow of one list leaves the other list servable. An original evicted from its only index still leavesposts.Tests
postsand is returned byget_all_posts_by_users; it is not returned byget_videos_by_users.posts.Standalone harness (same insert/get control flow, cap 4): buggy path drops the still-indexed original; fixed path serves it; single-index overflow still GC's.
thunder/has no Cargo manifest in this snapshot, so the crate tests cannot run here.Lane:
thunder/posts/post_store.rsonly. Based onxai-org/x-algorithmmain (c65aa179).