Keep posts created in the same second they are ingested - #48
Open
huxint wants to merge 1 commit into
Open
Conversation
insert_posts drops any post whose created_at is not strictly less than the current wall-clock second. A post whose create event is applied in the same second it was created — normal for a low-latency Kafka pipeline — fails the strict comparison and is silently discarded. The event is not revisited for the lifetime of that serving process: only a restart, which re-reads the topic under a fresh consumer group, would pick the post up again. Until then the freshest in-network content is simply missing from that instance's store. The check exists to reject future-dated timestamps; created_at equal to the current second is not future-dated. Use <= so only genuinely future-dated posts are dropped. The retention arithmetic is unaffected: current_time - created_at is 0 at the boundary, which passes the retention check and cannot underflow in trim_old_posts.
This was referenced Aug 14, 2026
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
PostStore::insert_postsuses a strict<when comparingcreated_atagainst the current wall-clock second, so a post whose create event is applied in the same second it was created is silently discarded and never enters the in-network store.Root cause
thunder/posts/post_store.rs:134:Both timestamps are epoch seconds, so for a low-latency pipeline
created_at == current_timeis a normal case, not a future-dated timestamp. The event is consumed once and not revisited for the lifetime of the process — only a restart, which re-reads the topic under a fresh consumer group, would recover it — so the freshest in-network content is simply missing from that instance's store.Fix
Use
<=so only genuinely future-dated posts are rejected. Boundary arithmetic is unaffected: at equalitycurrent_time - created_at == 0, which passes the retention check here, and theu64subtraction intrim_old_postsstill cannot underflow sincecreated_at <= current_timecontinues to hold at insert time.Verification
thunder/ships no Cargo manifest, so the crate cannot be compiled from this snapshot;rustfmtparses the file cleanly.created_atvalues ofcurrent_time - 1and older, so they do not exercise this boundary and are unaffected by the change.